diff --git a/bip-0114.mediawiki b/bip-0114.mediawiki
index aee8646b..cb9aea7c 100644
--- a/bip-0114.mediawiki
+++ b/bip-0114.mediawiki
@@ -24,71 +24,172 @@ The [[bip-0016.mediawiki|BIP16]] (Pay-to-script-hash, "P2SH") fixes the first 3
The [[bip-0141.mediawiki|BIP141]] defines 2 new types of scripts that support segregated witness. The pay-to-witness-script-hash (P2WSH) is similar to P2SH is many ways. By supplying the script in witness, P2WSH restores the original 10,000 byte script limit. However, it still requires publishing of unexecuted branches.
===Merkelized Abstract Syntax Tree===
-The idea of Merkelized Abstract Syntax Tree (MAST) is to use a Merkle tree to encode mutually exclusive branches in a script. When spending, the redeemer may provide only the branch they are executing, and hashes that connect the branch to the fixed size Merkel root. This reduces the size of redemption stack from O(n) to O(log n) (n as the number of mutually exclusive branches). This enables complicated redemption conditions that is currently not possible due to the script size and op code limit, improves privacy by hiding unexecuted branches, and allows inclusion of non-consensus enforced data with very low or no additional cost.
+The idea of Merkelized Abstract Syntax Tree (MAST) is to use a Merkle tree to encode branches in a script. When spending, users may provide only the branches they are executing, and hashes that connect the branches to the fixed size Merkel root. This reduces the size of redemption stack from O(n) to O(log n) (n as the number of branches). This enables complicated redemption conditions that is currently not possible due to the script size and opcode limit, improves privacy by hiding unexecuted branches, and allows inclusion of non-consensus enforced data with very low or no additional cost.
==Specification==
-In [[bip-0141.mediawiki|BIP141]], witness programs with a version byte of 1 or larger are considered to be anyone-can-spend scripts. The following new validation rules are applied if the witness program version byte is 1 and the program size is 32 bytes. The witness program is the MAST Root.
+In [[bip-0141.mediawiki|BIP141]], witness programs with a version byte of 1 or larger are considered to be anyone-can-spend scripts. The following new validation rules are applied if the witness program version byte is 1 and the program size is 32 bytes.If the version byte is 1, but the witness program is not 32 bytes, no further interpretation of the witness program or witness stack happens. This is reserved for future extensions. The witness program is the MAST Root.
-To redeem an output of this kind, the witness must consist of an input stack to feed to the script, followed by a Postion value, a serialized Merkle path (Path), and a serialized script (MAST Script).
+To redeem an output of this kind, the witness must consist of the following items:
-The Position, Path, and MAST Script are popped off the initial witness stack.
-The double-SHA256 of the MAST Script (≤ TBD bytes) must be correctly connected to the MAST Root with the ComputeMerkleRootFromBranch function, with the specified Path and Position.
+ Script_stack_1
+ Script_stack_2
+ .
+ .
+ Script_stack_X (X ≥ 0)
+ Subscript_1
+ Subscript_2
+ .
+ .
+ Subscript_Y (1 ≤ Y ≤ 255)
+ Position
+ Path
+ Metadata (Y|MAST Version)
-Path is the serialized Merkle path for the MAST Script. Size of Path must be a multiple of 32 bytes, and not more than 1024 bytes (which allows 32 levels). Each 32 byte word is a double-SHA256 merkle node in the merkle branch connecting to the MAST Root. If the size of Path is zero, the double-SHA256 of the MAST Script must match the MAST Root.
-Position indicates the location of the MAST Script in the Merkle tree, with zero indicating the leftmost position. It is an unsigned little-endian integer with not more than 4 bytes. It must be encoded in the most parsimonious way possible, with no leading zero and not larger than the maximum number of items allowed by the depth of the tree (as implied by the size of Path).
+Metadata is the last witness item. It is a vector of 1 to 5 bytes. The first byte is an unsigned integer between 1 to 255 denoting the number of Subscript (defined hereinafter). The following 0 to 4 byte(s) is an unsigned little-endian integer denoting the MAST version. MAST Version must be minimally encoded (the most significant byte must not be 0).
-The MAST Script is then deserialized, and executed after normal script evaluation with the remaining witness stack (≤ TBD bytes for each stack item). The script must not fail, and result in exactly a single TRUE on the stack.
+Path is the second last witness item. It is a serialized Merkle path of the Script Hash (defined hereinafter). Size of Path must be a multiple of 32 bytes, and not more than 1024 bytes. Each 32 byte word is a double-SHA256 merkle node in the merkle branch connecting to the Script Root (defined hereinafter). Depth of the tree (0 to 32) is the size of Path divided by 32.
-Sigops in MAST program are counted to the block sigop limit in the same way as the version 0 witness program (see BIP141).
+Position is the third last witness item. It indicates the location of the Script Hash in the Merkle tree, with zero indicating the leftmost position. It is an unsigned little-endian integer with not more than 4 bytes. It must be minimally encoded: the value must not be larger than the maximum number of items allowed by the Depth of the tree, and the most significant byte must not be 0. For example, if Depth is 4, the valid range of Position is 0 to 15 (24-1).
-If the version byte is 1, but the witness program is not 32 bytes, the script must fail.
+Depends on the first byte of Metadata, there should be 1 to 255 Subscript witness item(s) before Position.
+
+Script Hash is defined as:
+
+ Script Hash = H(Y|H(Subscript_1)|H(Subscript_2)|...|H(Subscript_Y))
+ H() = SHA256(SHA256())
+
+where Y is a 1-byte value denoting number of Subscript, followed by the hash of each Subscript
+
+Script Root is the Merkle root calculated by the ComputeMerkleRootFromBranch function, using Script Hash, Path and Position.
+
+MAST Root is H(MAST Version|Script Root). The pre-image has a fixed size of 36 bytes: 4 bytes for MAST Version (unsigned little-endian integer) and 32 bytes for Script Root.
+
+The script evaluation fails if MAST Root does not match the witness program.
+
+If the MAST Root matches the witness program and MAST Version is greater than 0, the script returns a success without further evaluation. SigOpsCost is counted as 0. This is reserved for future script upgrades.
+
+If the MAST Version is 0, the Subscript(s) are serialized to form the final MAST Script, beginning with Subscript_1. The unused witness item(s) before the Subscript_1 are used as Input Stack to feed to the MASTScript. (Similar to P2WSH in BIP141)
+
+The script fails with one of the following conditions:
+* MAST Script is malformed (i.e. not enough data provided for the last push operation). Individual Subscript might be malformed, as long as they are serialized into a valid MAST Script
+* Size of MAST Script is larger than 10,000 bytes
+* Size of any one of the Input Stack item is larger than 520 bytes
+* Number of non-push operations (nOpCount) is more than 201. nOpCount is the sum of the number of non-push operations in MAST Script (counted in the same way as P2WSH witnessScript), number of Subscript (Y), and Depth of the Merkle tree.
+
+The MAST Script is then evaluated with the Input Stack (with some new or redefined opcodes described in BIPXXX). The evaluation must not fail, and result in an exactly empty stack.
+
+Counting of SigOpsCost is based on the MAST Script, described in BIPYYY.
+
+== Rationale ==
+=== MAST Structure ===
+This proposal is a restricted case of more general MAST. In a general MAST design, users may freely assign one or more script branches for execution. In this proposal, only one branch is allowed for execution, and users are required to transform a complicated condition into several mutually exclusive branches. For example, if the desired redeem condition is:
+
+ (A or B) and (C or D or E) and (F or G)
+
+In a general MAST design, the 7 branches (A to G) will form a 3-level Merkle tree, plus an "overall condition" describing the relationship of different branches. In redemption, the "overall condition", executed branches (e.g. B, D, F), and Merkle path data will be provided for validation.
+
+In the current proposal, the user has to transform the redeem condition into 12 mutually exclusive branches and form a 4-level Merkle tree, and present only one branch in redemption:
+
+
+ A and C and F
+ B and C and F
+ A and D and F
+ .
+ .
+ B and E and G
+
+One way to implement the general MAST design is using a combination of OP_EVAL, OP_CAT, and OP_HASH256. However, that will suffer from the problems of OP_EVAL, including risks of indefinite program loop and inability to do static program analysis. A complicated implementation is required to fix these problems and is difficult to review.
+
+The advantages of the current proposal are:
+* Subscript are located at a fixed position in the witness stack. This allows static program analysis, such as static SigOpsCost counting and early termination of scripts with disabled opcodes.
+* If different parties in a contract do not want to expose their scripts to each other, they may provide only H(Subscript) and keep the Subscript private until redemption.
+* If they are willing to share the actual scripts, they may combine them into one Subscript for each branch, saving some nOpCount and a few bytes of witness space.
+
+The are some disadvantages, but only when the redemption condition is very complicated:
+* It may require more branches than a general MAST design (as shown in the previous example) and take more witness space in redemption
+* Creation and storage of the MAST structure may take more time and space. However, such additional costs affect only the related parties in the contract but not any other Bitcoin users.
+
+=== MAST Version ===
+This proposal allows users to indicate the version of scripting language in the witness, which is cheaper than doing that in scriptPubKey or scriptSig. Undefined versions remain anyone-can-spend and are reserved for future expansions. A new version could be used for relaxing constraints (e.g. the 10,000 bytes size limit of MAST Script), adding or redefining opcodes, or even introducing a completely novel scripting system.
+
+=== nOpCount limit ===
+In version 0 MAST, the extra hashing operations in calculating the MAST Root are counted towards the 201 nOpCount limit to prevent abusive use. This limitation is not applied to undefined MAST Version for flexibility, but it is constrained by the 255 Subscript and 32 Depth limits.
+
+=== Script evaluation ===
+This proposal requires script evaluation resulting in an empty stack, instead of a single TRUE value as in P2WSH. This allows each party in a contract to provide its own Subscript, and demonstrate the required Input Stack to clean up its own Subscript. In this case, order of the Subscript is not important since the overall objective is to clean up the stack after evaluation.
== Examples ==
=== Calculation of MAST Root ===
-To calculate the MAST Root for 4 branches, the double-SHA256 of each branch is first calculated:
- <1> EQUAL (0x5187), HASH256=3b647cb856a965fa6feffb4621eb2a4f4c2453693b2f64e021383ccbd80a1abb
- <2> EQUAL (0x5287), HASH256=37772654bdce9b3d59e1169ea16ddbaa8a2ae8ee265db64863d0b76f02c882fa
- <3> EQUAL (0x5387), HASH256=2e972642436151cd96e4b0868077b6362ffb5eb30b420a6f1c5e1c6fff02bc33
- <4> EQUAL (0x5487), HASH256=4c954fc1e635ce8417341465f85b59d700806f6e57bb96b2a25bec5ca3f9f154
-Serialize the hashes of the first pair and calculate the hash. Same for the other pair:
- HASH256(HASH256(5187)|HASH256(5287)) = 64fbdfc0a82ecc3b33434bfea63440e9a5275fa5e533200d2eaf18281e8b28b6
- HASH256(HASH256(5387)|HASH256(5487)) = aeadea837d5e640a1444208f7aca3be63bc8ab3c6b28a19878a00cc9c631ac31
+
+
+
+ Subscript:
+ SA = 1 EQUALVERIFY (0x5188)
+ SB = 2 EQUALVERIFY (0x5288)
+ SC = 3 EQUALVERIFY (0x5388)
+ SD = 4 EQUALVERIFY (0x5488)
+ SE = 5 EQUALVERIFY (0x5588)
+ SF = 6 EQUALVERIFY (0x5688)
+ SG = 7 EQUALVERIFY (0x5788)
+ SH = 8 EQUALVERIFY (0x5888)
+ M = RETURN "Hello" (0x6a0548656c6c6f)
+ Hash:
+ HA = H(0x01|H(SA)) = H(0x015acb54166e0db370cd1b05a29120373568dacea2abc3748459ec3da2106e4b4e) = 0xd385d7268ad7e1ec51660f833d54787d2d8d79b6b1809d9c1d06c9e71f7be204
+ HB = H(0x02|H(SB)|H(SC)) = 0x7cbfa08e44ea9f4f996873be95d9bffd97d4b91a5af32cc5f64efb8461727cdd
+ HF = H(0x03|H(SD)|H(SE)|H(SF)) = 0x4611414355945a7c2fcc62a53a0004821b87e68f93048ffba7a55a3cb1e9783b
+ HG = H(0x01|H(SG)) = 0xaa5fbdf58264650eadec33691ba1e7606d0a62f570eea348a465c55bc86ffc10
+ HC = H(0x01|H(M)) = 0x70426d480d5b28d93c5be54803681f99abf4e8df4eab4dc87aaa543f0d138159
+ HD = H(0x0x|H(SH)) = 0x8482f6c9c3fe90dd4d533b4efedb6a241b95ec9267d1bd5aaaee36d2ce2dd6da
+ HE = H(HA|HB) = 0x049b9f2f94f0a9bdea624e39cd7d6b27a365c6a0545bf0e9d88d86eff4894210
+ HH = H(HC|HD) = 0xc709fdc632f370f3367da45378d1cf430c5fda6805e731ad5761c213cf2d276e
+ HI = H(HE|HF) = 0xead5e1a1e7e41b77b794f091df9be3f0e9f41d47304eb43dece90688f69843b7
+ HJ = H(HG|HH) = 0xd00fc690c4700d0f983f9700740066531ea826b21a4cbc62f80317261723d477
+ Script Root = H(HI|HJ) = 0x26d5235d20daf1440a15a248f5b5b4f201392128072c55afa64a26ccc6f56bd9
+ MAST Root = H(MAST Version|Script Root) = H(0x0000000026d5235d20daf1440a15a248f5b5b4f201392128072c55afa64a26ccc6f56bd9) = 0xb4b706e0c02eab9aba58419eb7ea2a286fb1c01d7406105fc12742bf8a3f97c9
-Serialize the 2 hashes from the previous step and calculate the hash, which is the MAST Root:
- MAST Root = 6746003b5c9d342b2c210d406802c351e7eb5943412dcfc4718be625a8a59c0e
The scriptPubKey with native witness program is:
- <1> <0x6746003b5c9d342b2c210d406802c351e7eb5943412dcfc4718be625a8a59c0e>
- (0x51206746003b5c9d342b2c210d406802c351e7eb5943412dcfc4718be625a8a59c0e)
-To redeem with the <4> EQUAL branch, the witness is
- 04 (Stack for evaluation)
- 03 (Position)
- 2e972642436151cd96e4b0868077b6362ffb5eb30b420a6f1c5e1c6fff02bc3364fbdfc0a82ecc3b33434bfea63440e9a5275fa5e533200d2eaf18281e8b28b6 (Path)
- 5487 (MAST Script)
+ 1 <0xb4b706e0c02eab9aba58419eb7ea2a286fb1c01d7406105fc12742bf8a3f97c9>
+ (0x5120b4b706e0c02eab9aba58419eb7ea2a286fb1c01d7406105fc12742bf8a3f97c9)
+
+
+To redeem with the SD|SE|SF branch, the witness is
+
+ Script_stack_1: 0x06
+ Script_stack_2: 0x05
+ Script_stack_3: 0x04
+ Subscript_1: 0x5488
+ Subscript_2: 0x5588
+ Subscript_3: 0x5688
+ Position: 0x01 (HF is the second hash in its level)
+ Path (HE|HJ): 0x049b9f2f94f0a9bdea624e39cd7d6b27a365c6a0545bf0e9d88d86eff4894210d00fc690c4700d0f983f9700740066531ea826b21a4cbc62f80317261723d477
+ Metadata: 0x03 (3 Subscript)
+
+
=== Imbalance MAST ===
-When constructing a MAST, if the user believes that some of the branches are more likely to be executed, they may put them closer to the MAST Root. It will save some witness space when the preferred branches are actually executed.
+When constructing a MAST, if the user believes that some of the branches are more likely to be executed, they may put them closer to the Script Root. It will save some witness space when the preferred branches are actually executed.
=== Escrow with Timeout ===
The following is the "Escrow with Timeout" example in [[bip-0112.mediawiki|BIP112]]:
IF
- 2 3 CHECKMULTISIGVERIFY
+ 2 3 CHECKMULTISIG
ELSE
"30d" CHECKSEQUENCEVERIFY DROP
- CHECKSIGVERIFY
+ CHECKSIG
ENDIF
Using compressed public key, the size of this script is 150 bytes.
-With MAST, this script could be broken down into 2 mutually exclusive branches:
+With MAST, this script could be broken down into 2 mutually exclusive branches:In BIPXXX, it is proposed that CHECKLOCKTIMEVERIFY and CHECKSEQUENCEVERIFY will pop the top stack item
2 3 CHECKMULTISIGVERIFY (105 bytes)
- "30d" CHECKSEQUENCEVERIFY DROP CHECKSIGVERIFY (42 bytes)
+ "30d" CHECKSEQUENCEVERIFY CHECKSIGVERIFY (42 bytes)
-With 2 branches, the Path will be 32 bytes (as the hash of the unexecuted branch), and the Position will be 1 byte as either 0 or 1. Since only one branch will be published, it is more difficult for a blockchain analyst to determine the details of the escrow.
+Since only one branch will be published, it is more difficult for a blockchain analyst to determine the details of the escrow.
=== Hashed Time-Lock Contract ===
The following is the "Hashed TIme-Lock Contract" example in [[bip-0112.mediawiki|BIP112]]:
@@ -107,16 +208,16 @@ The following is the "Hashed TIme-Lock Contract" example in [[bip-0112.mediawiki
CHECKSIG
To create a MAST Root, it is flattened to 3 mutually exclusive branches:
- HASH160 EQUALVERIFY "24h" CHECKSEQUENCEVERIFY DROP CHECKSIG
- HASH160 EQUALVERIFY CHECKSIG
- "Timestamp" CHECKLOCKTIMEVERIFY DROP CHECKSIG
+ HASH160 EQUALVERIFY "24h" CHECKSEQUENCEVERIFY CHECKSIGVERIFY
+ HASH160 EQUALVERIFY CHECKSIGVERIFY
+ "Timestamp" CHECKLOCKTIMEVERIFY CHECKSIGVERIFY
which significantly improves readability and reduces the witness size when it is redeemed.
=== Large multi-signature constructs ===
-The current CHECKMULTISIG supports up to 20 public keys. Although it is possible to extend it beyond 20 keys by using multiple CHECKSIGs and IF/ELSE conditions, the construction could be very complicated and soon use up the 10,000 bytes and 201 op codes limit.
+The current CHECKMULTISIG supports up to 20 public keys. Although it is possible to extend it beyond 20 keys by using multiple CHECKSIGs and IF/ELSE conditions, the construction could be very complicated and soon use up the 10,000 bytes and 201 nOpCount limit.
-With MAST, large and complex multi-signature constructs could be flattened to many simple CHECKMULTISIG conditions. For example, a 3-of-2000 multi-signature scheme could be expressed as 1,331,334,000 3-of-3 CHECKMULTISIGs, which forms a 31-level MAST. The scriptPubKey still maintains a fixed size of 34 bytes, and the redemption witness will be very compact, with less than 1,500 bytes.
+With MAST, large and complex multi-signature constructs could be flattened to many simple CHECKMULTISIGVERIFY conditions. For example, a 3-of-2000 multi-signature scheme could be expressed as 1,331,334,000 3-of-3 CHECKMULTISIGVERIFY, which forms a 31-level MAST. The scriptPubKey still maintains a fixed size of 34 bytes, and the redemption witness will be very compact, with less than 1,500 bytes.
=== Commitment of non-consensus enforced data ===
Currently, committing non-consensus enforced data in the scriptPubKey requires the use of OP_RETURN which occupies additional block space. With MAST, users may commit such data as a branch. Depends on the number of executable branches, inclusion of such a commitment may incur no extra witness space, or 32 bytes at most.
@@ -133,76 +234,121 @@ This BIP depends on [[bip-0141.mediawiki|BIP141]] and will be deployed by versio
The idea of MAST originates from Russell O’Connor, Pieter Wuille, and [https://bitcointalk.org/index.php?topic=255145.msg2757327#msg2757327 Peter Todd].
== Reference Implementation ==
-https://github.com/jl2012/bitcoin/tree/segwit_mast
+https://github.com/jl2012/bitcoin/tree/bip114v2 (WIP)
diff --git a/bip-0114/mastexample.png b/bip-0114/mastexample.png
new file mode 100644
index 00000000..9bff7b99
Binary files /dev/null and b/bip-0114/mastexample.png differ