bitcoin/doc/benchmarking.md
merge-script 502d47203e
Merge bitcoin/bitcoin#31161: cmake: Set top-level target output locations
568fcdddae scripted-diff: Adjust documentation per top-level target output location (Hennadii Stepanov)
026bb226e9 cmake: Set top-level target output locations (Hennadii Stepanov)

Pull request description:

  This PR sets the target output locations to the `bin` and `lib` subdirectories within the build tree, creating a directory structure that mirrors that of the installed targets.

  This approach is widely adopted by the large projects, such as [LLVM](e146c1867e/lldb/cmake/modules/LLDBStandalone.cmake (L128-L130)):
  ```cmake
  set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
  set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX})
  set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX})
  ```

  The `libsecp256k1` project has also recently [adopted](https://github.com/bitcoin-core/secp256k1/pull/1553) this approach.

  With this PR, all binaries are conveniently located. For example, run:
  ```
  $ ./build/bin/fuzz
  ```
  instead of:
  ```
  $ ./build/src/test/fuzz/fuzz
  ```

  On Windows, all required DLLs are now located in the same directory as the executables, allowing to run `bitcoin-chainstate.exe` (which loads `bitcoinkernel.dll`) without the need to copy DLLs or modify the `PATH` variable.

  The idea was briefly discussed among the build team during the recent CoreDev meeting.

  ---

  **Warning**: This PR changes build locations of newly built executables like `bitcoind` and `test_bitcoin` from `src/` to `bin/` without deleting previously built executables. A clean build is recommended to avoid accidentally running old binaries.

ACKs for top commit:
  theStack:
    Light re-ACK 568fcdddae
  ryanofsky:
    Code review ACK 568fcdddae. Only change since last review was rebasing. I'm ok with this PR in its current form if other developers are happy with it. I just personally think it is inappropriate to \*silently\* break an everyday developer workflow like `git pull; make bitcoind`. I wouldn't have a problem with this PR if it triggered an explicit error, or if the problem was limited to less common workflows like changing cmake options in an existing build.
  TheCharlatan:
    Re-ACK 568fcdddae
  theuni:
    ACK 568fcdddae

Tree-SHA512: 1aa5ecd3cd49bd82f1dcc96c8e171d2d19c58aec8dade4bc329df89311f9e50cbf6cf021d004c58a0e1016c375b0fa348ccd52761bcdd179c2d1e61c105e3b9f
2025-03-12 11:19:00 +08:00

3 KiB

Benchmarking

Bitcoin Core has an internal benchmarking framework, with benchmarks for cryptographic algorithms (e.g. SHA1, SHA256, SHA512, RIPEMD160, Poly1305, ChaCha20), rolling bloom filter, coins selection, thread queue, wallet balance.

Running

For benchmarking, you only need to compile bench_bitcoin. The bench runner warns if you configure with -DCMAKE_BUILD_TYPE=Debug, but consider if building without it will impact the benchmark(s) you are interested in by unlatching log printers and lock analysis.

cmake -B build -DBUILD_BENCH=ON
cmake --build build -t bench_bitcoin

After compiling bitcoin-core, the benchmarks can be run with:

build/bin/bench_bitcoin

The output will look similar to:

|               ns/op |                op/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|       57,927,463.00 |               17.26 |    3.6% |      0.66 | `AddrManAdd`
|          677,816.00 |            1,475.33 |    4.9% |      0.01 | `AddrManGetAddr`

...

|             ns/byte |              byte/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|              127.32 |        7,854,302.69 |    0.3% |      0.00 | `Base58CheckEncode`
|               31.95 |       31,303,226.99 |    0.2% |      0.00 | `Base58Decode`

...

Help

build/bin/bench_bitcoin -h

To print the various options, like listing the benchmarks without running them or using a regex filter to only run certain benchmarks.

Notes

Benchmarks help with monitoring for performance regressions and can act as a scope for future performance improvements. They should cover components that impact performance critical functions of the system. Functions are performance critical if their performance impacts users and the cost associated with a degradation in performance is high. A non-exhaustive list:

  • Initial block download (Cost: slow IBD results in full node operation being less accessible)
  • Block template creation (Cost: slow block template creation may result in lower fee revenue for miners)
  • Block propagation (Cost: slow block propagation may increase the rate of orphaned blocks and mining centralization)

A change aiming to improve the performance may be rejected when a clear end-to-end performance improvement cannot be demonstrated. The change might also be rejected if the code bloat or review/maintenance burden is too high to justify the improvement.

Benchmarks are ill-suited for testing denial-of-service issues as they are restricted to the same input set (introducing bias). Fuzz tests are better suited for this purpose, as they are specifically aimed at exploring the possible input space.

Going Further

To monitor Bitcoin Core performance more in depth (like reindex or IBD): https://github.com/bitcoin-dev-tools/benchcoin