Commit Graph

3544 Commits

Author SHA1 Message Date
Olaoluwa Osuntokun
87e8fe92c9
btcec: convert package into go module, alias to dcrec
In this commit, we turn the package into a new Go module (version 2),
and then port over the current set of types and functions to mainly
alias to the more optimized and maintained dcrec variant.

Taking a look at the benchmarks, most operations other than
normalization (which IIRC is a bit slower now due to constant time
fixes) enjoy some nice speeds up:
```
benchcmp is deprecated in favor of benchstat: https://pkg.go.dev/golang.org/x/perf/cmd/benchstat
benchmark                            old ns/op     new ns/op     delta
BenchmarkAddJacobian-8               464           328           -29.20%
BenchmarkAddJacobianNotZOne-8        1138          372           -67.27%
BenchmarkScalarBaseMult-8            47336         31531         -33.39%
BenchmarkScalarBaseMultLarge-8       42465         32057         -24.51%
BenchmarkScalarMult-8                123355        117579        -4.68%
BenchmarkNAF-8                       582           168           -71.12%
BenchmarkSigVerify-8                 175414        120794        -31.14%
BenchmarkFieldNormalize-8            23.8          24.4          +2.39%
BenchmarkParseCompressedPubKey-8     24282         10907         -55.08%
```
2022-01-26 16:10:14 -08:00
Olaoluwa Osuntokun
588c0714c3
Merge pull request #1700 from Roasbeef/bip-8-prep
chaincfg+blockchain: abstract/refactor BIP 9 version bits implementation to work w/ BIP 8 block heights
2022-01-26 12:53:38 -08:00
Olaoluwa Osuntokun
0b245cca4f
btcjson+rpc: add min activation height to soft fork RPC response 2022-01-25 15:27:52 -08:00
Olaoluwa Osuntokun
54f6fa948e
blockchain: refactor new thresholdState method, test BIP9 transitions
In this commit, we extract the BIP 9 state transition logic from the
thresholdState method into a new thresholdStateTransition function that
allows us to test all the defined state transitions, including the
modified "speedy trial" logic.
2022-01-25 15:27:48 -08:00
Olaoluwa Osuntokun
c6b66ee79c
blockchain+integration: add support for min activation height and custom thresholds
In this commit, we extend the existing version bits state machine to add
support for the new minimum activation height and custom block threshold
for activation. We then extend the existing BIP 9 tests (tho this isn't
really BIP 9 anymore...) to exercise the new min activation height
logic.
2022-01-25 15:27:46 -08:00
Olaoluwa Osuntokun
38737a8ae3
chainparams: add new DeploymentTestDummyMinActivation
In this commit, we add a new "dummy" deployment that adds the new params
used to activate taproot. We chose to add a new deployment as unlike the
bitcoind codebase, we don't currently "bury" soft forks that have
happened in the past (hard code an activation height).

The old taproot deployment has been removed as with the way the array
works, a deployment needs to be defined for _all_ networks.
2022-01-25 15:27:44 -08:00
Olaoluwa Osuntokun
0556c7084f
build: don't run the integration tests w/ -race 2022-01-25 15:27:42 -08:00
Olaoluwa Osuntokun
298d6165be
blockchain: update version bits logic to use HasStarted/HasEnded for deployments
In this commit, we update our version bits logic to use the newly added
HasStarted and HasEnded methods for consensus deployments. Along the
way, wee modify the thresholdConditionChecker` interface to be based off
the new chaincfg interfaces. In addition, we add a new method
`PastMedianTime`, in order to allow the chain itself to be used as a
`chaincfg.BlockClock`.

This serves to make the logic more generic in order to support both
block height and time based soft fork timeouts.
2022-01-25 15:27:40 -08:00
Olaoluwa Osuntokun
2b6370dfd7
chaincfg: use DeploymentStarter/DeploymentEnder instead of start/end times
In this commit, we utilize the recently added ConsensusDeploymentStarter
and ConsensusDeploymentEnder interfaces. Concrete implementations of
this interface based on the median time past comparison are used now in
the ConsensusDeployment struct instead of hard coded start/end times.

Along the way, we had to switch to using the "zero time": time.Time{},
in place of 0 and math.MaxInt64 as comparison (After/Before) seems to be
broken in the Go stdlib for times very far in the future. It appears Go
isn't ready to handle the heat death of the universe.
2022-01-25 15:27:37 -08:00
Olaoluwa Osuntokun
dc4dc15d8c
chaincfg: create new abstract deployment starter/ender interfaces
In this commit, we create a series of new interfaces that'll allow us to
abstract "when" exactly a deployment starts and ends. As is, all
deployments start/end based on a unix timestamp, which is compared
against the MTP of a given block to determine if a new deployment has
started or ended. This works fine for BIP 9 which uses time based
timeouts, but not so much for BIP 8. In order to prep a future refactor
that allows our version bits implementation to support both time and
block based start/end times, this new abstraction has been introduced.
2022-01-25 15:27:30 -08:00
Olaoluwa Osuntokun
c5e6d3cf5f
Merge pull request #1788 from Roasbeef/btcutil-tag-updates
btcutil: update modules to replace to top-level btcd repo
2022-01-11 14:26:28 -08:00
Olaoluwa Osuntokun
161863e7fc
btcutil: update modules to replace to top-level btcd repo
Now that the new `btcutil/v1.0.0` tag has been pushed, we update the new
internal modules to point to the top-level `btcd` repo via replace
directives.
2022-01-11 14:21:39 -08:00
Olaoluwa Osuntokun
97732e5281
Merge pull request #1785 from Roasbeef/btcutil-move
multi: move the btcutil repo into btcd as a sub-module
2022-01-10 19:27:46 -08:00
Olaoluwa Osuntokun
3cacbd489e
build: adapt btcutil Makefile for usage in project
In this commit, we adapt the Makefile that was being used for the
btcutil project to work for btcd as well. The Makefile is pretty simple,
and is just a series of templated commands. Overtime, we can pull in
some of the `lnd` additions as well, which we use to handle our
reproducible build and verification system.
2022-01-10 18:45:03 -08:00
Olaoluwa Osuntokun
caac0f821a
multi: update btcutil imports to point to new sub-module
In this commit, we update all the btcutil imports to point to the new
sub-module.

In the same commit, we also modify the recently added `btcutil/go.mod`
file as we need to continue pointing to the _old_ version of btcd, until
we merge this PR and push a new tag.
2022-01-10 18:44:58 -08:00
Olaoluwa Osuntokun
5cd3533e2b
btcutil: move btcutil into new sub-module
In this commit, we move `btcutil` as well as its sub-module, the `psbt`
package into the `btcd` repo itself.
2022-01-10 18:44:56 -08:00
Matthew Bajorek
a1f43e4d84 integration: Add unit tests for all three GetNetworkHashPS client methods 2021-12-14 10:05:52 -05:00
Matthew Bajorek
ec9ca7d782 rpcclient+rpcserver: Change getnetworkhashps return type to be a float to be in line with bitcoin core 2021-12-14 10:05:52 -05:00
Tomasz Ziolkowski
780cc0889f reduce redundant memory allocatio - resolves btcsuite/btcd#1699
Signed-off-by: Tomasz Ziolkowski <tomasz.ziolkowski@allegro.pl>
2021-11-30 09:39:38 -05:00
Calvin Kim
3be166e3ae go.mod, go.sum: Update goleveldb
Goleveldb recently had a PR in where memory allocation was reduced
drastically (github.com/syndtr/goleveldb/pull/367).  Update goleveldb
to use that PR.
2021-11-30 09:35:11 -05:00
Olaoluwa Osuntokun
7070d53e09
Merge pull request #1769 from Roasbeef/txscript_zero_alloc_optimization_refactor
txscript: backport tokenizer from dcrd
2021-11-16 19:14:28 -08:00
Dave Collins
b95ba0ac95
txscript: Make op callbacks take opcode and data.
This converts the callback function defined on the internal opcode
struct to accept the opcode and data slice instead of a parsed opcode as
the final step towards removing the parsed opcode struct and associated
supporting code altogether.

It also updates all of the callbacks and tests accordingly and finally
removes the now unused parsedOpcode struct.

The final results for the raw script analysis and tokenizer
optimizations are as follows:

benchmark                                       old ns/op     new ns/op     delta
BenchmarkIsPayToScriptHash-8                    62393         0.51          -100.00%
BenchmarkIsPubKeyHashScript-8                   62228         0.56          -100.00%
BenchmarkGetSigOpCount-8                        61051         658           -98.92%
BenchmarkExtractPkScriptAddrsLarge-8            60713         17.2          -99.97%
BenchmarkExtractPkScriptAddrs-8                 289           17.9          -93.81%
BenchmarkIsWitnessPubKeyHash-8                  61688         0.42          -100.00%
BenchmarkIsUnspendable-8                        656           520           -20.73%
BenchmarkExtractAtomicSwapDataPushesLarge-8     61332         44.0          -99.93%
BenchmarkExtractAtomicSwapDataPushes-8          990           260           -73.74%
BenchmarkDisasmString-8                         102902        39754         -61.37%
BenchmarkGetPreciseSigOpCount-8                 130223        715           -99.45%
BenchmarkScriptParsing-8                        63464         681           -98.93%
BenchmarkIsMultisigScriptLarge-8                64166         5.83          -99.99%
BenchmarkIsMultisigScript-8                     630           58.5          -90.71%
BenchmarkPushedData-8                           64837         1779          -97.26%
BenchmarkCalcSigHash-8                          3627895       3605459       -0.62%
BenchmarkIsPubKeyScript-8                       62323         2.83          -100.00%
BenchmarkIsPushOnlyScript-8                     62412         569           -99.09%
BenchmarkIsWitnessScriptHash-8                  61243         0.56          -100.00%
BenchmarkGetScriptClass-8                       61515         16.4          -99.97%
BenchmarkIsNullDataScript-8                     62495         2.53          -100.00%
BenchmarkIsMultisigSigScriptLarge-8             69328         2.52          -100.00%
BenchmarkIsMultisigSigScript-8                  2375          141           -94.06%
BenchmarkGetWitnessSigOpCountP2WKH-8            504           72.0          -85.71%
BenchmarkGetWitnessSigOpCountNested-8           1158          136           -88.26%
BenchmarkIsWitnessPubKeyHash-8                  68927         0.53          -100.00%
BenchmarkIsWitnessScriptHash-8                  62774         0.63          -100.00%

benchmark                                       old allocs     new allocs     delta
BenchmarkIsPayToScriptHash-8                    1              0              -100.00%
BenchmarkIsPubKeyHashScript-8                   1              0              -100.00%
BenchmarkGetSigOpCount-8                        1              0              -100.00%
BenchmarkExtractPkScriptAddrsLarge-8            1              0              -100.00%
BenchmarkExtractPkScriptAddrs-8                 1              0              -100.00%
BenchmarkIsWitnessPubKeyHash-8                  1              0              -100.00%
BenchmarkIsUnspendable-8                        1              0              -100.00%
BenchmarkExtractAtomicSwapDataPushesLarge-8     1              0              -100.00%
BenchmarkExtractAtomicSwapDataPushes-8          2              1              -50.00%
BenchmarkDisasmString-8                         46             51             +10.87%
BenchmarkGetPreciseSigOpCount-8                 3              0              -100.00%
BenchmarkScriptParsing-8                        1              0              -100.00%
BenchmarkIsMultisigScriptLarge-8                1              0              -100.00%
BenchmarkIsMultisigScript-8                     1              0              -100.00%
BenchmarkPushedData-8                           7              6              -14.29%
BenchmarkCalcSigHash-8                          1335           712            -46.67%
BenchmarkIsPubKeyScript-8                       1              0              -100.00%
BenchmarkIsPushOnlyScript-8                     1              0              -100.00%
BenchmarkIsWitnessScriptHash-8                  1              0              -100.00%
BenchmarkGetScriptClass-8                       1              0              -100.00%
BenchmarkIsNullDataScript-8                     1              0              -100.00%
BenchmarkIsMultisigSigScriptLarge-8             5              0              -100.00%
BenchmarkIsMultisigSigScript-8                  3              0              -100.00%
BenchmarkGetWitnessSigOpCountP2WKH-8            2              0              -100.00%
BenchmarkGetWitnessSigOpCountNested-8           4              0              -100.00%
BenchmarkIsWitnessPubKeyHash-8                  1              0              -100.00%
BenchmarkIsWitnessScriptHash-8                  1              0              -100.00%

benchmark                                       old bytes     new bytes     delta
BenchmarkIsPayToScriptHash-8                    311299        0             -100.00%
BenchmarkIsPubKeyHashScript-8                   311299        0             -100.00%
BenchmarkGetSigOpCount-8                        311299        0             -100.00%
BenchmarkExtractPkScriptAddrsLarge-8            311299        0             -100.00%
BenchmarkExtractPkScriptAddrs-8                 768           0             -100.00%
BenchmarkIsWitnessPubKeyHash-8                  311299        0             -100.00%
BenchmarkIsUnspendable-8                        1             0             -100.00%
BenchmarkExtractAtomicSwapDataPushesLarge-8     311299        0             -100.00%
BenchmarkExtractAtomicSwapDataPushes-8          3168          96            -96.97%
BenchmarkDisasmString-8                         389324        130552        -66.47%
BenchmarkGetPreciseSigOpCount-8                 623367        0             -100.00%
BenchmarkScriptParsing-8                        311299        0             -100.00%
BenchmarkIsMultisigScriptLarge-8                311299        0             -100.00%
BenchmarkIsMultisigScript-8                     2304          0             -100.00%
BenchmarkPushedData-8                           312816        1520          -99.51%
BenchmarkCalcSigHash-8                          1373812       1290507       -6.06%
BenchmarkIsPubKeyScript-8                       311299        0             -100.00%
BenchmarkIsPushOnlyScript-8                     311299        0             -100.00%
BenchmarkIsWitnessScriptHash-8                  311299        0             -100.00%
BenchmarkGetScriptClass-8                       311299        0             -100.00%
BenchmarkIsNullDataScript-8                     311299        0             -100.00%
BenchmarkIsMultisigSigScriptLarge-8             330035        0             -100.00%
BenchmarkIsMultisigSigScript-8                  9472          0             -100.00%
BenchmarkGetWitnessSigOpCountP2WKH-8            1408          0             -100.00%
BenchmarkGetWitnessSigOpCountNested-8           3200          0             -100.00%
BenchmarkIsWitnessPubKeyHash-8                  311299        0             -100.00%
BenchmarkIsWitnessScriptHash-8                  311299        0             -100.00%
2021-11-16 18:49:46 -08:00
Dave Collins
ef3d06e62b
txscript: Make executeOpcode take opcode and data.
This converts the executeOpcode function defined on the engine to accept
an opcode and data slice instead of a parsed opcode as a step towards
removing the parsed opcode struct and associated supporting code altogether.

It also updates all callers accordingly.
2021-11-16 18:49:41 -08:00
Dave Collins
595d379fa6
txscript: Remove unused parseScriptTemplate func.
Also remove tests associated with the func accordingly.
2021-11-16 18:49:38 -08:00
Dave Collins
ca044fefcb
txscript: Remove unused parsedOpcode.bytes func. 2021-11-16 18:49:36 -08:00
Dave Collins
491b7b59fc
txscript: Remove unused unparseScript func.
Also remove tests associated with unparsing opcodes accordingly.
2021-11-16 18:49:33 -08:00
Conner Fromknecht
e06b11a999
txscript: Remove unused calcWitnessSignatureHash 2021-11-16 18:49:31 -08:00
Dave Collins
6e5fbf8ea8
txscript: Remove unused parseScript func. 2021-11-16 18:49:28 -08:00
Conner Fromknecht
7533672993
txscript/pkscript: Use finalOpcodeData to extract redeem script 2021-11-16 18:49:25 -08:00
Conner Fromknecht
69f3a39c1c
txscript/sign: Use calcWitnessSigHashRaw for witness sigs 2021-11-16 18:49:23 -08:00
Conner Fromknecht
94e99cf6b7
txscript: Rename calcSignatureHashRaw 2021-11-16 18:49:20 -08:00
Dave Collins
911db90858
txscript: Rename removeOpcodeByDataRaw func.
This renames the removeOpcodeByDataRaw to removeOpcodeByData now that
the old version has been removed.
2021-11-16 18:49:18 -08:00
Dave Collins
07ab66b790
txscript: Remove unused removeOpcodeByData func. 2021-11-16 18:49:15 -08:00
Conner Fromknecht
03d1fb0f86
txscript: Remove unused isWitnessProgram 2021-11-16 18:49:13 -08:00
Conner Fromknecht
06c8bea6c7
txscript: Remove unused calcSignatureHash 2021-11-16 18:49:10 -08:00
Dave Collins
d6b968c3ea
txscript: Refactor engine to use raw scripts.
This refactors the script engine to store and step through raw scripts
by making using of the new zero-allocation script tokenizer as opposed
to the less efficient method of storing and stepping through parsed
opcodes.  It also improves several aspects while refactoring such as
optimizing the disassembly trace, showing all scripts in the trace in
the case of execution failure, and providing additional comments
describing the purpose of each field in the engine.

It should be noted that this is a step towards removing the parsed
opcode struct and associated supporting code altogether, however, in
order to ease the review process, this retains the struct and all
function signatures for opcode execution which make use of an individual
parsed opcode.  Those will be updated in future commits.

The following is an overview of the changes:

- Modify internal engine scripts slice to use raw scripts instead of
  parsed opcodes
- Introduce a tokenizer to the engine to track the current script
- Remove no longer needed script offset parameter from the engine since
  that is tracked by the tokenizer
- Add an opcode index counter for disassembly purposes to the engine
- Update check for valid program counter to only consider the script
  index
  - Update tests for bad program counter accordingly
- Rework the NewEngine function
  - Store the raw scripts
  - Setup the initial tokenizer
  - Explicitly check against version 0 instead of DefaultScriptVersion
    which would break consensus if changed
  - Check the scripts parse according to version 0 semantics to retain
    current consensus rules
  - Improve comments throughout
- Rework the Step function
  - Use the tokenizer and raw scripts
  - Create a parsed opcode on the fly for now to retain existing
    opcode execution function signatures
  - Improve comments throughout
- Update the Execute function
  - Explicitly check against version 0 instead of DefaultScriptVersion
    which would break consensus if changed
  - Improve the disassembly tracing in the case of error
- Update the CheckErrorCondition function
  - Modify clean stack error message to make sense in all cases
  - Improve the comments
- Update the DisasmPC and DisasmScript functions on the engine
  - Use the tokenizer
  - Optimize construction via the use of strings.Builder
- Modify the subScript function to return the raw script bytes since the
  parsed opcodes are no longer stored
- Update the various signature checking opcodes to use the raw opcode
  data removal and signature hash calculation functions since the
  subscript is now a raw script
  - opcodeCheckSig
  - opcodeCheckMultiSig
  - opcodeCheckSigAlt
2021-11-16 18:49:08 -08:00
Dave Collins
54036e8bab
txscript: Convert to use non-parsed opcode disasm.
This converts the engine's current program counter disasembly to make
use of the standalone disassembly function to remove the dependency on
the parsed opcode struct.

It also updates the tests accordingly.
2021-11-16 18:49:05 -08:00
Dave Collins
710bd5646e
txscript: Make min push accept raw opcode and data.
This converts the checkMinimalDataPush function defined on a parsed
opcode to a standalone function which accepts an opcode and data slice
instead in order to make it more flexible for raw script analysis.

It also updates all callers accordingly.
2021-11-16 18:49:03 -08:00
Dave Collins
62c608f265
txscript: Make isConditional accept raw opcode.
This converts the isConditional function defined on a parsed opcode to a
standalone function named isOpcodeConditional which accepts an opcode as
a byte instead in order to make it more flexible for raw script
analysis.

It also updates all callers accordingly.
2021-11-16 18:49:00 -08:00
Dave Collins
c6410257eb
txscript: Make alwaysIllegal accept raw opcode.
This converts the alwaysIllegal function defined on a parsed opcode to a
standalone function named isOpcodeAlwaysIllegal which accepts an opcode
as a byte instead in order to make it more flexible for raw script
analysis.

It also updates all callers accordingly.
2021-11-16 18:48:58 -08:00
Dave Collins
484f7b1fef
txscript: Make isDisabled accept raw opcode.
This converts the isDisabled function defined on a parsed opcode to a
standalone function which accepts an opcode as a byte instead in order
to make it more flexible for raw script analysis.

It also updates all callers accordingly.
2021-11-16 18:48:55 -08:00
Conner Fromknecht
a2ab5b6681
txscript: Use removeOpcodeRaw for CODESEP in calcSigHash 2021-11-16 18:48:52 -08:00
Conner Fromknecht
2ddcdb91f5
txscript: Remove unused removeOpcode 2021-11-16 18:48:50 -08:00
Conner Fromknecht
a4720f30e5
txscript: Optimize removeOpcodeRaw 2021-11-16 18:48:47 -08:00
Dave Collins
30874ff76b
txscript: Implement efficient opcode data removal.
This introduces a new function named removeOpcodeByDataRaw which accepts
the raw scripts and data to remove versus requiring the parsed opcodes
to both significantly optimize it as well as make it more flexible for
working with raw scripts.

There are several places in the rest of the code that currently only
have access to the parsed opcodes, so this only introduces the function
for use in the future and deprecates the existing one.

Note that, in practice, the script will never actually contain the data
that is intended to be removed since the function is only used during
signature verification to remove the signature itself which would
require some incredibly non-standard code to create.

Thus, as an optimization, it avoids allocating a new script unless there
is actually a match that needs to be removed.

Finally, it updates the tests to use the new function.
2021-11-16 18:48:45 -08:00
Dave Collins
f3354beb12
txscript: Use raw scripts in SignTxOutput.
This converts SignTxOutput and supporting funcs, namely sign,
mergeScripts and mergeMultiSig, to make use of the new tokenizer as well
as some recently added funcs that deal with raw scripts in order to
remove the reliance on parsed opcodes as a step towards utlimately
removing them altogether and updates the comments to explicitly call out
the script version semantics.

It is worth noting that this has the side effect of optimizing the
function as well, however, since this change is not focused on the
optimization aspects, no benchmarks are provided.
2021-11-16 18:48:42 -08:00
Conner Fromknecht
e00fec1557
txscript: Use optimized calcWitnessSignatureHashRaw w/o parsing 2021-11-16 18:48:40 -08:00
Conner Fromknecht
ed9e17a043
txscript: Remove unused isWitnessPubKeyHash 2021-11-16 18:48:37 -08:00
Conner Fromknecht
dd609d6e36
txscript: Introduce calcWitnessSignatureHashRaw 2021-11-16 18:48:35 -08:00
Dave Collins
7ad3a10442
txscript: mergeMultiSig function def order cleanup.
This moves the function definition for mergeMultiSig so it is more
consistent with the preferred order used through the codebase.  In
particular, the functions are defined before they're first used and
generally as close as possible to the first use when they're defined in
the same file.
2021-11-16 18:48:32 -08:00