mirror of
https://github.com/bitcoin/bitcoin.git
synced 2025-02-22 23:07:59 +01:00
78c312c983
This replaces the current benchmarking framework with nanobench [1], an MIT licensed single-header benchmarking library, of which I am the autor. This has in my opinion several advantages, especially on Linux: * fast: Running all benchmarks takes ~6 seconds instead of 4m13s on an Intel i7-8700 CPU @ 3.20GHz. * accurate: I ran e.g. the benchmark for SipHash_32b 10 times and calculate standard deviation / mean = coefficient of variation: * 0.57% CV for old benchmarking framework * 0.20% CV for nanobench So the benchmark results with nanobench seem to vary less than with the old framework. * It automatically determines runtime based on clock precision, no need to specify number of evaluations. * measure instructions, cycles, branches, instructions per cycle, branch misses (only Linux, when performance counters are available) * output in markdown table format. * Warn about unstable environment (frequency scaling, turbo, ...) * For better profiling, it is possible to set the environment variable NANOBENCH_ENDLESS to force endless running of a particular benchmark without the need to recompile. This makes it to e.g. run "perf top" and look at hotspots. Here is an example copy & pasted from the terminal output: | ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 2.52 | 396,529,415.94 | 0.6% | 25.42 | 8.02 | 3.169 | 0.06 | 0.0% | 0.03 | `bench/crypto_hash.cpp RIPEMD160` | 1.87 | 535,161,444.83 | 0.3% | 21.36 | 5.95 | 3.589 | 0.06 | 0.0% | 0.02 | `bench/crypto_hash.cpp SHA1` | 3.22 | 310,344,174.79 | 1.1% | 36.80 | 10.22 | 3.601 | 0.09 | 0.0% | 0.04 | `bench/crypto_hash.cpp SHA256` | 2.01 | 496,375,796.23 | 0.0% | 18.72 | 6.43 | 2.911 | 0.01 | 1.0% | 0.00 | `bench/crypto_hash.cpp SHA256D64_1024` | 7.23 | 138,263,519.35 | 0.1% | 82.66 | 23.11 | 3.577 | 1.63 | 0.1% | 0.00 | `bench/crypto_hash.cpp SHA256_32b` | 3.04 | 328,780,166.40 | 0.3% | 35.82 | 9.69 | 3.696 | 0.03 | 0.0% | 0.03 | `bench/crypto_hash.cpp SHA512` [1] https://github.com/martinus/nanobench * Adds support for asymptotes This adds support to calculate asymptotic complexity of a benchmark. This is similar to #17375, but currently only one asymptote is supported, and I have added support in the benchmark `ComplexMemPool` as an example. Usage is e.g. like this: ``` ./bench_bitcoin -filter=ComplexMemPool -asymptote=25,50,100,200,400,600,800 ``` This runs the benchmark `ComplexMemPool` several times but with different complexityN settings. The benchmark can extract that number and use it accordingly. Here, it's used for `childTxs`. The output is this: | complexityN | ns/op | op/s | err% | ins/op | cyc/op | IPC | total | benchmark |------------:|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|----------:|:---------- | 25 | 1,064,241.00 | 939.64 | 1.4% | 3,960,279.00 | 2,829,708.00 | 1.400 | 0.01 | `ComplexMemPool` | 50 | 1,579,530.00 | 633.10 | 1.0% | 6,231,810.00 | 4,412,674.00 | 1.412 | 0.02 | `ComplexMemPool` | 100 | 4,022,774.00 | 248.58 | 0.6% | 16,544,406.00 | 11,889,535.00 | 1.392 | 0.04 | `ComplexMemPool` | 200 | 15,390,986.00 | 64.97 | 0.2% | 63,904,254.00 | 47,731,705.00 | 1.339 | 0.17 | `ComplexMemPool` | 400 | 69,394,711.00 | 14.41 | 0.1% | 272,602,461.00 | 219,014,691.00 | 1.245 | 0.76 | `ComplexMemPool` | 600 | 168,977,165.00 | 5.92 | 0.1% | 639,108,082.00 | 535,316,887.00 | 1.194 | 1.86 | `ComplexMemPool` | 800 | 310,109,077.00 | 3.22 | 0.1% |1,149,134,246.00 | 984,620,812.00 | 1.167 | 3.41 | `ComplexMemPool` | coefficient | err% | complexity |--------------:|-------:|------------ | 4.78486e-07 | 4.5% | O(n^2) | 6.38557e-10 | 21.7% | O(n^3) | 3.42338e-05 | 38.0% | O(n log n) | 0.000313914 | 46.9% | O(n) | 0.0129823 | 114.4% | O(log n) | 0.0815055 | 133.8% | O(1) The best fitting curve is O(n^2), so the algorithm seems to scale quadratic with `childTxs` in the range 25 to 800. |
||
|---|---|---|
| .. | ||
| check-doc.py | ||
| check-rpc-mappings.py | ||
| commit-script-check.sh | ||
| extended-lint-all.sh | ||
| extended-lint-cppcheck.sh | ||
| git-subtree-check.sh | ||
| lint-all.sh | ||
| lint-assertions.sh | ||
| lint-circular-dependencies.sh | ||
| lint-filenames.sh | ||
| lint-format-strings.py | ||
| lint-format-strings.sh | ||
| lint-include-guards.sh | ||
| lint-includes.sh | ||
| lint-locale-dependence.sh | ||
| lint-logs.sh | ||
| lint-python-mutable-default-parameters.sh | ||
| lint-python-utf8-encoding.sh | ||
| lint-python.sh | ||
| lint-qt.sh | ||
| lint-rpc-help.sh | ||
| lint-shebang.sh | ||
| lint-shell-locale.sh | ||
| lint-shell.sh | ||
| lint-spelling.ignore-words.txt | ||
| lint-spelling.sh | ||
| lint-submodule.sh | ||
| lint-tests.sh | ||
| lint-whitespace.sh | ||
| README.md | ||
This folder contains lint scripts.
check-doc.py
Check for missing documentation of command line options.
commit-script-check.sh
Verification of scripted diffs. Scripted diffs are only assumed to run on the latest LTS release of Ubuntu. Running them on other operating systems might require installing GNU tools, such as GNU sed.
git-subtree-check.sh
Run this script from the root of the repository to verify that a subtree matches the contents of the commit it claims to have been updated to.
To use, make sure that you have fetched the upstream repository branch in which the subtree is maintained:
- for
src/secp256k1: https://github.com/bitcoin-core/secp256k1.git (branch master) - for
src/leveldb: https://github.com/bitcoin-core/leveldb.git (branch bitcoin-fork) - for
src/univalue: https://github.com/bitcoin-core/univalue.git (branch master) - for
src/crypto/ctaes: https://github.com/bitcoin-core/ctaes.git (branch master) - for
src/crc32c: https://github.com/google/crc32c.git (branch master)
Usage: git-subtree-check.sh DIR (COMMIT)
COMMIT may be omitted, in which case HEAD is used.
lint-all.sh
Calls other scripts with the lint- prefix.