Design mithril-common split & re-organization in repository

[jpraynaud]

Why

In order to reduce build times, binary sizes and number of features, and also to provide more structured libraries, we want to split the mithril-common crate in sub-crates.

What

Propose a split for the mithril-common crate and a plan of deployment.

How

• Create a new mapping for mithril-common into sub-crates (mention responsibility of the new sub-crate, propose names and locations in the repository -internal, root, ...-)
• Identify the new sub-crates that will be need to be published on crates.io
• Prepare a roll-out plan which details the order of migrations and sub-crates that need to be migrated together

[Alenar]

Current state overview

Tip

The graphs have been generated using cargo module.

Note

For all generated graph, their code have been edited to remove a mithril-common node that was connected to every items, polluting the resulting graph (this dependency exist because of the StdResult alias that's broadly used).

Top modules inter-dependencies

Graph code

Command

Note: Since the cfg_xxx features gate macros where hindering cargo module analisys capabilities, the cfg_fs! macro arround the digesters, block scanner, etc ... was temporary removed in the lib.rs before running the command

cargo modules dependencies -p mithril-common --lib --features all --no-fns --no-traits --no-owns --no-types --no-externs --max-depth 1

Generated graphviz

digraph {

    graph [
        label="mithril_common",
        labelloc=t,

        pad=0.4,

        // Consider rendering the graph using a different layout algorithm, such as:
        // [dot, neato, twopi, circo, fdp, sfdp]
        layout=neato,
        overlap=false,
        splines="line",
        rankdir=LR,

        fontname="Helvetica", 
        fontsize="36",
    ];

    node [
        fontname="monospace",
        fontsize="10",
        shape="record",
        style="filled",
    ];

    edge [
        fontname="monospace",
        fontsize="10",
    ];

    "mithril_common::api_version" [label="pub mod|api_version", fillcolor="#81c169"]; // "mod" node
    "mithril_common::cardano_block_scanner" [label="pub mod|cardano_block_scanner", fillcolor="#81c169"]; // "mod" node
    "mithril_common::cardano_transactions_preloader" [label="pub mod|cardano_transactions_preloader", fillcolor="#81c169"]; // "mod" node
    "mithril_common::certificate_chain" [label="pub mod|certificate_chain", fillcolor="#81c169"]; // "mod" node
    "mithril_common::chain_observer" [label="pub mod|chain_observer", fillcolor="#81c169"]; // "mod" node
    "mithril_common::chain_reader" [label="pub mod|chain_reader", fillcolor="#81c169"]; // "mod" node
    "mithril_common::crypto_helper" [label="pub mod|crypto_helper", fillcolor="#81c169"]; // "mod" node
    "mithril_common::digesters" [label="pub mod|digesters", fillcolor="#81c169"]; // "mod" node
    "mithril_common::entities" [label="pub mod|entities", fillcolor="#81c169"]; // "mod" node
    "mithril_common::era" [label="pub mod|era", fillcolor="#81c169"]; // "mod" node
    "mithril_common::logging" [label="pub mod|logging", fillcolor="#81c169"]; // "mod" node
    "mithril_common::messages" [label="pub mod|messages", fillcolor="#81c169"]; // "mod" node
    "mithril_common::protocol" [label="pub mod|protocol", fillcolor="#81c169"]; // "mod" node
    "mithril_common::resource_pool" [label="pub mod|resource_pool", fillcolor="#81c169"]; // "mod" node
    "mithril_common::signable_builder" [label="pub mod|signable_builder", fillcolor="#81c169"]; // "mod" node
    "mithril_common::signed_entity_type_lock" [label="pub mod|signed_entity_type_lock", fillcolor="#81c169"]; // "mod" node
    "mithril_common::ticker_service" [label="pub(crate) mod|ticker_service", fillcolor="#f8c04c"]; // "mod" node

    "mithril_common::api_version" -> "mithril_common::era" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::cardano_block_scanner" -> "mithril_common::chain_reader" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::cardano_block_scanner" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::cardano_block_scanner" -> "mithril_common::logging" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::cardano_transactions_preloader" -> "mithril_common::chain_observer" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::cardano_transactions_preloader" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::cardano_transactions_preloader" -> "mithril_common::logging" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::cardano_transactions_preloader" -> "mithril_common::signed_entity_type_lock" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::certificate_chain" -> "mithril_common::crypto_helper" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::certificate_chain" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::certificate_chain" -> "mithril_common::logging" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::certificate_chain" -> "mithril_common::protocol" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::chain_observer" -> "mithril_common::crypto_helper" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::chain_observer" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::chain_reader" -> "mithril_common::cardano_block_scanner" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::chain_reader" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::chain_reader" -> "mithril_common::logging" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::crypto_helper" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::crypto_helper" -> "mithril_common::resource_pool" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::digesters" -> "mithril_common::crypto_helper" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::digesters" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::digesters" -> "mithril_common::logging" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::entities" -> "mithril_common::crypto_helper" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::entities" -> "mithril_common::messages" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::entities" -> "mithril_common::protocol" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::entities" -> "mithril_common::signable_builder" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::era" -> "mithril_common::chain_observer" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::era" -> "mithril_common::crypto_helper" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::era" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::messages" -> "mithril_common::crypto_helper" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::messages" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::messages" -> "mithril_common::era" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::protocol" -> "mithril_common::crypto_helper" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::protocol" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::signable_builder" -> "mithril_common::crypto_helper" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::signable_builder" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::signable_builder" -> "mithril_common::era" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::signable_builder" -> "mithril_common::logging" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::ticker_service" -> "mithril_common::chain_observer" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::ticker_service" -> "mithril_common::digesters" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge
    "mithril_common::ticker_service" -> "mithril_common::entities" [label="uses", color="#7f7f7f", style="dashed"] [constraint=false]; // "uses" edge

}

Modules analysis

signed_entity_type_lock

Dependency of: cardano_transactions_preloader

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Nothing depends on it and it don't depends on anything either, it could be easily extracted.

certificate_chain

Dependency of: Nothing

Depends on	Reason	Remark	Could be lifted off
crypto_helper	for the multiples keys it needs	/	❌
logging	To get its logger name	/	❌
entities	Mostly for the Certificate and protocol parameters	/	❌
protocol	To convert an input to a message when creating a genesis certificate	used in a test only method	❔

Api version

Dependency of: Nothing

Depends on	Reason	Remark	Could be lifted off
era	Need the current era to fetch the appropriate openapi file	/	❌

Logging

Dependency of: signable_builder, certificate_chain, cardano_block_scanner, chain_reader, digesters, cardano_transactions_preloader

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Resource pool

Dependency of: crypto_helper

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

The crypto helper only need them to implement the reset trait to the MKMap type.

Era

Dependency of: api_version, messages, signable_builder

Depends on	Reason	Remark	Could be lifted off
entities	needs the Epoch type for the era marker	/	❌
crypto_helper		/	✔️
chain_observer	/	/	✔️

This module consist of two parts: one part to check the current era and one to read them from different providers. The read part is the one that needs the dependency to crypto_helpers and chain_observer and could be partly closer to those dependencies.

Chain observer

Dependency of: era, cardano_transactions_preloader, ticker_service

Depends on	Reason	Remark	Could be lifted off
crypto_helper	mostly for the KESPeriod and the OpCert	/	❌
entities	For multiple types (epoch, stake distribution, blocknumbers, ...)	/	❌

Signable builder

Dependency of:

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Ticker service

Dependency of:

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Protocol

Dependency of:

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Cardano block scanner

Dependency of:

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Chain reader

Dependency of:

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Messages

Dependency of:

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Crypto helper

Dependency of:

Depends on	Reason	Remark	Could be lifted off
/	/	/	/

Entities

Dependency of:

Depends on	Reason	Remark	Could be lifted off
/	/	/	/