In this commit, we implement the new BIP 341+342 taproot sighash digest
computation. The digest is similar, but re-orders some fragments and
also starts to commit to the input values of all the transactions in the
SIGHASH_ALL case. A new implicit sighash flag, SIGHASH_DEFAULT has been
added that allows signatures to always be 64-bytes for the common case.
The hashcache has been updated as well to store both the v0 and v1 mid
state hashes. The v0 hashes are a double-sha of the contents, while the
v1 hash is a single sha. As a result, if a transaction spends both v0
and v1 inputs, then we 're able to re-use all the intermediate hashes.
As the sighash computation needs the input values and scripts, we create
an abstraction: the PrevOutFetcher to give the caller flexibility w.r.t
how this is done. We also create a `CannedPrevOutputFetcher` that holds
the information in a map for a single input.
A series of function options are also added to allow re-use of the same
base sig hash calculation for both BIP 341 and 342.
This implements an efficient and zero-allocation script tokenizer that
is exported to both provide a new capability to tokenize scripts to
external consumers of the API as well as to serve as a base for
refactoring the existing highly inefficient internal code.
It is important to note that this tokenizer is intended to be used in
consensus critical code in the future, so it must exactly follow the
existing semantics.
The current script parsing mechanism used throughout the txscript module
is to fully tokenize the scripts into an array of internal parsed
opcodes which are then examined and passed around in order to implement
virtually everything related to scripts.
While that approach does simplify the analysis of certain scripts and
thus provide some nice properties in that regard, it is both extremely
inefficient in many cases, and makes it impossible for external
consumers of the API to implement any form of custom script analysis
without manually implementing a bunch of error prone tokenizing code or,
alternatively, the script engine exposing internal structures.
For example, as shown by profiling the total memory allocations of an
initial sync, the existing script parsing code allocates a total of
around 295.12GB, which equates to around 50% of all allocations
performed. The zero-alloc tokenizer this introduces will allow that to
be reduced to virtually zero.
The following is a before and after comparison of tokenizing a large
script with a high opcode count using the existing code versus the
tokenizer this introduces for both speed and memory allocations:
benchmark old ns/op new ns/op delta
BenchmarkScriptParsing-8 63464 677 -98.93%
benchmark old allocs new allocs delta
BenchmarkScriptParsing-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkScriptParsing-8 311299 0 -100.00%
The following is an overview of the changes:
- Introduce new error code ErrUnsupportedScriptVersion
- Implement zero-allocation script tokenizer
- Add a full suite of tests to ensure the tokenizer works as intended
and follows the required consensus semantics
- Add an example of using the new tokenizer to count the number of
opcodes in a script
- Update README.md to include the new example
- Update script parsing benchmark to use the new tokenizer
