diff --git a/docs/core-blockchain-concepts/blockchain.md b/docs/core-blockchain-concepts/blockchain.md index 71e7ba9..00e5128 100644 --- a/docs/core-blockchain-concepts/blockchain.md +++ b/docs/core-blockchain-concepts/blockchain.md @@ -15,11 +15,11 @@ Below is a high-level diagram of the Decred blockchain. A block of one or more new transactions (Block 1) is collected into the transaction data part of a block. These transactions are then -hashed and rehashed to create the [merkle root](../developer-guides/merkle-root-construction.md). +hashed and rehashed to create the [Merkle root](../developer-guides/merkle-root-construction.md). The [block header](https://docs.decred.org/advanced/block-header-specifications/) contains metadata about the block, including -the merkle root and hash of the previous block header. +the Merkle root and hash of the previous block header. Including the hash of the previous block header makes it impossible to change a transaction in a block without also modifying all subsequent blocks in the chain. diff --git a/docs/developer-guides/merkle-root-construction.md b/docs/developer-guides/merkle-root-construction.md index ef7b57b..b5c305f 100644 --- a/docs/developer-guides/merkle-root-construction.md +++ b/docs/developer-guides/merkle-root-construction.md @@ -2,47 +2,48 @@ --- -#### Merkle Trees - -The merkle root is constructed using all the TXIDs of transactions in -this block, but first the TXIDs are placed in order as required by the -consensus rules: - -* The coinbase transaction's TXID is always placed first. - -* Any input within this block can spend an output which also appears in - this block (assuming the spend is otherwise valid). However, the TXID - corresponding to the output must be placed at some point before the - TXID corresponding to the input. This ensures that any program parsing - block chain transactions linearly will encounter each output before it - is used as an input. - -If a block only has a coinbase transaction, the coinbase TXID is used as -the merkle root hash. - -If a block only has a coinbase transaction and one other transaction, -the TXIDs of those two transactions are placed in order, concatenated as -64 raw bytes, and then SHA256(SHA256()) hashed together to form the -merkle root. - -If a block has three or more transactions, intermediate merkle tree rows -are formed. The TXIDs are placed in order and paired, starting with the -coinbase transaction's TXID. Each pair is concatenated together as 64 -raw bytes and SHA256(SHA256()) hashed to form a second row of -hashes. If there are an odd (non-even) number of TXIDs, the last TXID is -concatenated with a copy of itself and hashed. If there are more than -two hashes in the second row, the process is repeated to create a third -row (and, if necessary, repeated further to create additional rows). -Once a row is obtained with only two hashes, those hashes are concatenated and -hashed to produce the merkle root. - - - -![Example Merkle Tree Construction](/img/dev/en-merkle-tree-construction.svg) - -TXIDs and intermediate hashes are always in internal byte order when they're -concatenated, and the resulting merkle root is also in internal byte -order when it's placed in the block header. +This page explains what Merkle trees are, how Decred uses them, and provides step-by-step instructions for constructing compliant Merkle trees. + +### Merkle Trees + + +A Merkle tree is a data structure that can be used to store hashes of arbitrary data. In the context of Decred, this might be transaction or ticket data. + +Merkle trees are created by repeatedly hashing pairs of nodes until there is only one hash left, the Merkle Root. They are constructed from the bottom up, from hashes of the leaf nodes (hashes of the raw input data). The diagram below shows a Merkle root constructed from eight leaf nodes. + +![Merkle Tree](/img/core-blockchain-concepts/merkle_tree.svg) + +Merkle trees are particularly useful for blockchains, as they efficiently and compactly summarize larger data sets. + + +### Merkle Trees in Decred + +Merkle roots are used to create compact summaries of all transactions in a block. These Merkle roots can then be queried to quickly and efficiently verify whether or not a transaction is included in a block. + +Each block header contains the Merkle roots of two Merkle trees, the `MerkleRoot` tree and `StakeRoot` tree. The `MerkleRoot` tree contains hashes of all regular transaction IDs. The `StakeRoot` tree contains hashes of stake-based transaction IDs (ticket purchases, votes, revocations, etc.). + +#### Hash Function + +Unless otherwise specified, all hashing operations **must** be performed with the BLAKE-256 hash function with 14 rounds. + +#### Merkle Root Construction + +All Merkle trees **must** be constructed according to the unique Merkle tree construct originally implemented by Satoshi, which relies on construction of the tree from an ordered list as follows: + +1. Calculate list of BLAKE-256 hashes of original data (e.g. transaction IDs) +1. If the list is empty, return a 32-byte root hash of all zeroes; otherwise +1. While the list contains two or more items: + - If the number of items remaining in the list is odd, duplicate the final hash + - Combine each pair of adjacent entries with the BLAKE-256 hash of the two entries concatenated together. The list will have ceil(N/2) entries remaining after combining the adjacent entries +1. Return the final remaining item in the list as the Merkle root + +!!! warning "Warning" + The requirement for duplicating the final hash for internal tree levels that have an odd number of nodes must be carefully considered by applications making use of the resulting Merkle root because it means the final calculated Merkle root for a tree that internally duplicated a hash and one that actually included a duplicate hash at that position will be indistinguishable. + +The below diagram illustrates these steps. + +![Merkle Tree Calculation](/img/core-blockchain-concepts/merkle_root_calc.svg) + + + + diff --git a/docs/img/core-blockchain-concepts/merkle-tree-construction.svg b/docs/img/core-blockchain-concepts/merkle-tree-construction.svg new file mode 100644 index 0000000..93bcb73 --- /dev/null +++ b/docs/img/core-blockchain-concepts/merkle-tree-construction.svg @@ -0,0 +1,86 @@ + + + + + + +_anonymous_0 + + Example Merkle Tree Construction  [Hash function H() = SHA256(SHA256())] + +txids +Row 1: Transaction hashes (TXIDs) +(A is coinbase; C can spend output from B) + + +row2 +Row 2: Hashes of paired TXIDs + + + +rootrow +Merkle root + + + +txid_a + +A + + +row2_ab + +H(A|B) + + +txid_a->row2_ab + + + + +txid_b + +B + + +txid_b->row2_ab + + + + + +row2_cc + +H(C|C) + + + +txid_c + +C + + +txid_c->row2_cc + + + + +root + +H(H(A|B)|H(C|C)) + + +row2_ab->root + + + + +row2_cc->root + + + + + diff --git a/docs/img/core-blockchain-concepts/merkle_root_calc.svg b/docs/img/core-blockchain-concepts/merkle_root_calc.svg new file mode 100644 index 0000000..87b28eb --- /dev/null +++ b/docs/img/core-blockchain-concepts/merkle_root_calc.svg @@ -0,0 +1,149 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + List Empty? + + + + + + + Accept list of + BLAKE-256 hashes + + + + + + + Return 32-byte + zero hash + + + + + + + Merkle Root + Calculation + + + + + + + List Size >= 2? + + + + + + + Duplicate final list hash + + + + + + + Odd List Size? + + + + + + + Concatenate and hash + adjacent list entries: + BLAKE-256(left || right) + + + + + + + Return final + remaining list item + + + + + + + + + Yes + + + + + No + + + Yes + + + Yes + + + No + + + + + + + No + + + diff --git a/docs/img/core-blockchain-concepts/merkle_tree.svg b/docs/img/core-blockchain-concepts/merkle_tree.svg new file mode 100644 index 0000000..b8cd44b --- /dev/null +++ b/docs/img/core-blockchain-concepts/merkle_tree.svg @@ -0,0 +1,457 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Data Items + + + + + + + D3 + + + + + + + + + D2 + + + + + + + + + D1 + + + + + + + + + D0 + + + + + + + + + D7 + + + + + + + + + D6 + + + + + + + + + D5 + + + + + + + + + D4 + + + + + + + + + + Merkle Tree with N=8 + + + + Level 0 + + + + + + + + + L0:3 + + + + + + + L0:1 + + + + + + + L0:0 + + + + + + + L0:7 + + + + + + + L0:6 + + + + + + + L0:5 + + + + + + + L0:4 + + + + + + + L0:2 + + + + + + + hash(D2) + + + + + + + + + + hash(D0) + + + + + + + + + + hash(D1) + + + + + + + + + + hash(D3) + + + + + + + + + + hash(D4) + + + + + + + + + + hash(D5) + + + + + + + + + + hash(D6) + + + + + + + + + + hash(D7) + + + + + + + + + + + + + + + + + + + + + + + + + + + Level 1 + + + + + + + + + L1:3 + + + + + + + L1:2 + + + + + + + L1:1 + + + + + + + L1:0 + + + + + + + hash(L0:0 || L0:1) + + + + + + + + + + hash(L0:2 || L0:3) + + + + + + + + + + hash(L0:4 || L0:5) + + + + + + + + + + hash(L0:6 || L0:7) + + + + + + + + + + + + + + + + + + + + + + + + + + + Level 2 + + + + + + + + + L2:0 + + + + + + + L2:1 + + + + + + + hash(L1:0 || L1:1) + + + + + + + + + + hash(L1:2 || L1:3) + + + + + + + + + + + + + + + + + + + Level 3 + + + + + + + + + Root Hash + + + + + + + + + hash(L2:0 || L2:1) + + + + + + + + + + + + + + + diff --git a/docs/projects.md b/docs/projects.md index 1af37b6..f41bcb0 100644 --- a/docs/projects.md +++ b/docs/projects.md @@ -102,7 +102,7 @@ inspired by the work of Peter Todd’s [OpenTimestamps](https://petertodd.org/2016/opentimestamps-announcement). dcrtime allows a nearly unlimited number of hashes to be timestamped onchain with the inclusion of a -single merkle root in a transaction. +single Merkle root in a transaction. dcrtimegui implements a front-end for dcrtime, enabling files to be timestamped through a web-browser.