Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flesh out merkle root construction page #38

Merged
merged 7 commits into from
Oct 26, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/core-blockchain-concepts/blockchain.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ Below is a high-level diagram of the Decred blockchain.

A block of one or more new transactions (Block 1) is collected into
the transaction data part of a block. These transactions are then
hashed and rehashed to create the [merkle root](../developer-guides/merkle-root-construction.md).
hashed and rehashed to create the [Merkle root](../developer-guides/merkle-root-construction.md).

The [block header](https://docs.decred.org/advanced/block-header-specifications/)
contains metadata about the block, including
the merkle root and hash of the previous block header.
the Merkle root and hash of the previous block header.
Including the hash of the previous block header makes it impossible to change
a transaction in a block without also modifying all subsequent blocks
in the chain.
Expand Down
89 changes: 45 additions & 44 deletions docs/developer-guides/merkle-root-construction.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,47 +2,48 @@

---

#### Merkle Trees

The merkle root is constructed using all the TXIDs of transactions in
this block, but first the TXIDs are placed in order as required by the
consensus rules:

* The coinbase transaction's TXID is always placed first.

* Any input within this block can spend an output which also appears in
this block (assuming the spend is otherwise valid). However, the TXID
corresponding to the output must be placed at some point before the
TXID corresponding to the input. This ensures that any program parsing
block chain transactions linearly will encounter each output before it
is used as an input.

If a block only has a coinbase transaction, the coinbase TXID is used as
the merkle root hash.

If a block only has a coinbase transaction and one other transaction,
the TXIDs of those two transactions are placed in order, concatenated as
64 raw bytes, and then SHA256(SHA256()) hashed together to form the
merkle root.

If a block has three or more transactions, intermediate merkle tree rows
are formed. The TXIDs are placed in order and paired, starting with the
coinbase transaction's TXID. Each pair is concatenated together as 64
raw bytes and SHA256(SHA256()) hashed to form a second row of
hashes. If there are an odd (non-even) number of TXIDs, the last TXID is
concatenated with a copy of itself and hashed. If there are more than
two hashes in the second row, the process is repeated to create a third
row (and, if necessary, repeated further to create additional rows).
Once a row is obtained with only two hashes, those hashes are concatenated and
hashed to produce the merkle root.

<!-- built block 170's merkle root with Python to confirm left-to-right order
for A|B concatenation demonstrated below:
sha256(sha256("82501c1178fa0b222c1f3d474ec726b832013f0a532b44bb620cce8624a5feb1169e1e83e930853391bc6f35f605c6754cfead57cf8387639d3b4096c54f18f4".decode("hex")).digest()).digest().encode("hex_codec")
-->

![Example Merkle Tree Construction](/img/dev/en-merkle-tree-construction.svg)

TXIDs and intermediate hashes are always in internal byte order when they're
concatenated, and the resulting merkle root is also in internal byte
order when it's placed in the block header.
This page explains what Merkle trees are, how Decred uses them, and provides step-by-step instructions for constructing compliant Merkle trees.

### Merkle Trees


A Merkle tree is a data structure that can be used to store hashes of arbitrary data. In the context of Decred, this might be transaction or ticket data.

Merkle trees are created by repeatedly hashing pairs of nodes until there is only one hash left, the Merkle Root. They are constructed from the bottom up, from hashes of the leaf nodes (hashes of the raw input data). The diagram below shows a Merkle root constructed from eight leaf nodes.

![Merkle Tree](/img/core-blockchain-concepts/merkle_tree.svg)

Merkle trees are particularly useful for blockchains, as they efficiently and compactly summarize larger data sets.


### Merkle Trees in Decred

Merkle roots are used to create compact summaries of all transactions in a block. These Merkle roots can then be queried to quickly and efficiently verify whether or not a transaction is included in a block.

Each block header contains the Merkle roots of two Merkle trees, the `MerkleRoot` tree and `StakeRoot` tree. The `MerkleRoot` tree contains hashes of all regular transaction IDs. The `StakeRoot` tree contains hashes of stake-based transaction IDs (ticket purchases, votes, revocations, etc.).

#### Hash Function

Unless otherwise specified, all hashing operations **must** be performed with the BLAKE-256 hash function with 14 rounds.

#### Merkle Root Construction

All Merkle trees **must** be constructed according to the unique Merkle tree construct originally implemented by Satoshi, which relies on construction of the tree from an ordered list as follows:

1. Calculate list of BLAKE-256 hashes of original data (e.g. transaction IDs)
1. If the list is empty, return a 32-byte root hash of all zeroes; otherwise
jholdstock marked this conversation as resolved.
Show resolved Hide resolved
1. While the list contains two or more items:
- If the number of items remaining in the list is odd, duplicate the final hash
- Combine each pair of adjacent entries with the BLAKE-256 hash of the two entries concatenated together. The list will have ceil(N/2) entries remaining after combining the adjacent entries
1. Return the final remaining item in the list as the Merkle root

!!! warning "Warning"
The requirement for duplicating the final hash for internal tree levels that have an odd number of nodes must be carefully considered by applications making use of the resulting Merkle root because it means the final calculated Merkle root for a tree that internally duplicated a hash and one that actually included a duplicate hash at that position will be indistinguishable.

The below diagram illustrates these steps.

![Merkle Tree Calculation](/img/core-blockchain-concepts/merkle_root_calc.svg)




86 changes: 86 additions & 0 deletions docs/img/core-blockchain-concepts/merkle-tree-construction.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading