Bitcoin Core integration/staging tree
Go to file
Wladimir J. van der Laan 4ebe2f6e75
Merge #18011: Replace current benchmarking framework with nanobench
78c312c983 Replace current benchmarking framework with nanobench (Martin Ankerl)

Pull request description:

  Replace current benchmarking framework with nanobench

  This replaces the current benchmarking framework with nanobench [1], an
  MIT licensed single-header benchmarking library, of which I am the
  autor. This has in my opinion several advantages, especially on Linux:

  * fast: Running all benchmarks takes ~6 seconds instead of 4m13s on
    an Intel i7-8700 CPU @ 3.20GHz.

  * accurate: I ran e.g. the benchmark for SipHash_32b 10 times and
    calculate standard deviation / mean = coefficient of variation:

    * 0.57% CV for old benchmarking framework
    * 0.20% CV for nanobench

    So the benchmark results with nanobench seem to vary less than with
    the old framework.

  * It automatically determines runtime based on clock precision, no need
    to specify number of evaluations.

  * measure instructions, cycles, branches, instructions per cycle,
    branch misses (only Linux, when performance counters are available)

  * output in markdown table format.

  * Warn about unstable environment (frequency scaling, turbo, ...)

  * For better profiling, it is possible to set the environment variable
    NANOBENCH_ENDLESS to force endless running of a particular benchmark
    without the need to recompile. This makes it to e.g. run "perf top"
    and look at hotspots.

  Here is an example copy & pasted from the terminal output:

  |             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |                2.52 |      396,529,415.94 |    0.6% |           25.42 |            8.02 |  3.169 |           0.06 |    0.0% |      0.03 | `bench/crypto_hash.cpp RIPEMD160`
  |                1.87 |      535,161,444.83 |    0.3% |           21.36 |            5.95 |  3.589 |           0.06 |    0.0% |      0.02 | `bench/crypto_hash.cpp SHA1`
  |                3.22 |      310,344,174.79 |    1.1% |           36.80 |           10.22 |  3.601 |           0.09 |    0.0% |      0.04 | `bench/crypto_hash.cpp SHA256`
  |                2.01 |      496,375,796.23 |    0.0% |           18.72 |            6.43 |  2.911 |           0.01 |    1.0% |      0.00 | `bench/crypto_hash.cpp SHA256D64_1024`
  |                7.23 |      138,263,519.35 |    0.1% |           82.66 |           23.11 |  3.577 |           1.63 |    0.1% |      0.00 | `bench/crypto_hash.cpp SHA256_32b`
  |                3.04 |      328,780,166.40 |    0.3% |           35.82 |            9.69 |  3.696 |           0.03 |    0.0% |      0.03 | `bench/crypto_hash.cpp SHA512`

  [1] https://github.com/martinus/nanobench

ACKs for top commit:
  laanwj:
    ACK 78c312c983

Tree-SHA512: 9e18770b18b6f95a7d0105a4a5497d31cf4eb5efe6574f4482f6f1b4c88d7e0946b9a4a1e9e8e6ecbf41a3f2d7571240677dcb45af29a6f0584e89b25f32e49e
2020-07-30 15:34:17 +02:00
.github doc: Add redirect for GUI issues and pull requests 2020-06-08 10:06:02 -04:00
.tx tx: Bump transifex slug to 020x 2020-03-16 10:52:55 +01:00
build_msvc Updates msvc build to use ISO standard C++17. 2020-07-04 16:03:18 +01:00
build-aux/m4 Merge #18297: build: Use pkg-config in BITCOIN_QT_CONFIGURE for all hosts including Windows 2020-06-13 15:41:39 +08:00
ci Merge #19519: ci: Increase CCACHE_SIZE in some builds on Travis 2020-07-28 10:25:11 +02:00
contrib Merge #18011: Replace current benchmarking framework with nanobench 2020-07-30 15:34:17 +02:00
depends Merge #19530: depends: build LTO support into Apple's ld64 2020-07-28 16:01:26 +08:00
doc Merge #18011: Replace current benchmarking framework with nanobench 2020-07-30 15:34:17 +02:00
share doc: Use precise permission flags where possible 2020-07-10 15:37:42 +02:00
src Merge #18011: Replace current benchmarking framework with nanobench 2020-07-30 15:34:17 +02:00
test Merge #18011: Replace current benchmarking framework with nanobench 2020-07-30 15:34:17 +02:00
.appveyor.yml Merge #18011: Replace current benchmarking framework with nanobench 2020-07-30 15:34:17 +02:00
.cirrus.yml ci: Run tsan ci config on cirrus 2020-07-02 12:22:39 -04:00
.fuzzbuzz.yml ci: Add fuzzbuzz integration 2020-04-14 16:38:26 +00:00
.gitattributes Separate protocol versioning from clientversion 2014-10-29 00:24:40 -04:00
.gitignore Fix .gitignore for src/test/fuzz directory 2020-07-27 00:56:37 +03:00
.python-version .python-version: Specify full version 3.5.6 2019-03-02 12:06:26 -05:00
.style.yapf test: .style.yapf: Set column_limit=160 2019-03-04 18:28:13 -05:00
.travis.yml travis: Re-enable s390x 2020-07-28 16:01:53 +02:00
autogen.sh scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
configure.ac build: add -Wl,-z,separate-code to hardening flags 2020-07-28 12:57:35 +08:00
CONTRIBUTING.md doc: CONTRIBUTING.md improvements 2020-07-12 07:52:28 +02:00
COPYING doc: Update license year range to 2020 2019-12-26 23:11:21 +01:00
INSTALL.md Update INSTALL landing redirection notice for build instructions. 2016-10-06 12:27:23 +13:00
libbitcoinconsensus.pc.in build: remove libcrypto as internal dependency in libbitcoinconsensus.pc 2019-11-19 15:03:44 +01:00
Makefile.am tests: run test-security-check.py in CI 2020-06-16 19:52:30 +08:00
README.md doc: Mention repo split in the READMEs 2020-06-08 10:06:14 -04:00
SECURITY.md doc: Remove explicit mention of version from SECURITY.md 2019-06-14 06:39:17 -04:00

Bitcoin Core integration/staging tree

https://bitcoincore.org

What is Bitcoin?

Bitcoin is an experimental digital currency that enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network. Bitcoin Core is the name of open source software which enables the use of this currency.

For more information, as well as an immediately usable, binary version of the Bitcoin Core software, see https://bitcoincore.org/en/download/, or read the original whitepaper.

License

Bitcoin Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is regularly built (see doc/build-*.md for instructions) and tested, but it is not guaranteed to be completely stable. Tags are created regularly from release branches to indicate new official, stable release versions of Bitcoin Core.

The https://github.com/bitcoin-core/gui repository is used exclusively for the development of the GUI. Its master branch is identical in all monotree repositories. Release branches and tags do not exist, so please do not fork that repository unless it is for development reasons.

The contribution workflow is described in CONTRIBUTING.md and useful hints for developers can be found in doc/developer-notes.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python, that are run automatically on the build server. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Bitcoin Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.

Translators should also subscribe to the mailing list.