bitcoin/src/bench
Andrew Chow 5aa0c82ccd
Merge bitcoin/bitcoin#25325: Add pool based memory resource
9f947fc3d4 Use PoolAllocator for CCoinsMap (Martin Leitner-Ankerl)
5e4ac5abf5 Call ReallocateCache() on each Flush() (Martin Leitner-Ankerl)
1afca6b663 Add PoolResource fuzzer (Martin Leitner-Ankerl)
e19943f049 Calculate memory usage correctly for unordered_maps that use PoolAllocator (Martin Leitner-Ankerl)
b8401c3281 Add pool based memory resource & allocator (Martin Leitner-Ankerl)

Pull request description:

  A memory resource similar to `std::pmr::unsynchronized_pool_resource`, but optimized for node-based containers. The goal is to be able to cache more coins with the same memory usage, and allocate/deallocate faster.

  This is a reimplementation of #22702. The goal was to implement it in a way that is simpler to review & test

  * There is now a generic `PoolResource` for allocating/deallocating memory. This has practically the same API as `std::pmr::memory_resource`. (Unfortunately I cannot use std::pmr because libc++ simply doesn't implement that API).
  * Thanks to sipa there is now a fuzzer for PoolResource! On a fast machine I ran it for ~770 million executions without finding any issue.

  * The estimation of the correct node size is now gone, PoolResource now has multiple pools and just needs to be created large enough to have space for the unordered_map nodes.

  I ran benchmarks with #22702, mergebase, and this PR. Frequency locked Intel i7-8700, clang++ 13.0.1 to reindex up to block 690000.

  ```sh
  bitcoind -dbcache=5000 -assumevalid=00000000000000000002a23d6df20eecec15b21d32c75833cce28f113de888b7 -reindex-chainstate -printtoconsole=0 -stopatheight=690000
  ```

  The performance is practically identical with #22702, just 0.4% slower. It's ~21% faster than master:

  ![Progress in Million Transactions over Time(2)](https://user-images.githubusercontent.com/14386/173288685-91952ade-f304-4825-8bfb-0725a71ca17b.png)

  ![Size of Cache in MiB over Time](https://user-images.githubusercontent.com/14386/173291421-e6b410be-ac77-479b-ad24-5fafcebf81eb.png)
  Note that on cache drops mergebase's memory doesnt go so far down because it does not free the `CCoinsMap` bucket array.

  ![Size of Cache in Million tx over Time(1)](https://user-images.githubusercontent.com/14386/173288703-a80c9c9e-93c8-4a16-9df8-610c89c61cc4.png)

ACKs for top commit:
  LarryRuane:
    ACK 9f947fc3d4
  achow101:
    re-ACK 9f947fc3d4
  john-moffett:
    ACK 9f947fc3d4
  jonatack:
    re-ACK 9f947fc3d4

Tree-SHA512: 48caf57d1775875a612b54388ef64c53952cd48741cacfe20d89049f2fb35301b5c28e69264b7d659a3ca33d4c714d47bafad6fd547c4075f08b45acc87c0f45
2023-04-20 16:20:15 -04:00
..
data Add deserialize + CheckBlock benchmarks, and a full block hex 2016-11-09 11:27:59 -08:00
.gitignore Ignore bench_bitcoin binary. 2015-10-06 17:46:12 +02:00
addrman.cpp bench: test select for a new table with only one address 2023-03-17 18:02:40 -07:00
base58.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
bech32.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
bench.cpp refactor: Move fs.* to util/fs.* 2023-03-23 12:55:18 +01:00
bench.h refactor: Move fs.* to util/fs.* 2023-03-23 12:55:18 +01:00
bench_bitcoin.cpp refactor: Move fs.* to util/fs.* 2023-03-23 12:55:18 +01:00
block_assemble.cpp Merge bitcoin/bitcoin#26695: bench: BlockAssembler on a mempool with packages 2023-01-11 18:11:11 -05:00
ccoins_caching.cpp Merge bitcoin/bitcoin#26691: Update secp256k1 subtree to libsecp256k1 version 0.2.0 2023-01-13 09:40:57 +00:00
chacha20.cpp Only support 32-byte keys in ChaCha20{,Aligned} 2023-01-30 18:12:21 -05:00
chacha_poly_aead.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
checkblock.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
checkqueue.cpp refactor: Use move semantics in CCheckQueue::Loop 2023-03-21 13:04:21 +00:00
coin_selection.cpp wallet: single output groups filtering and grouping process 2023-03-06 09:45:40 -03:00
crypto_hash.cpp Merge bitcoin/bitcoin#15294: refactor: Extract RipeMd160 2023-01-30 09:49:01 +01:00
data.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
data.h bench: Move generated data to a dedicated translation unit 2019-07-02 18:11:15 +01:00
descriptors.cpp Merge bitcoin/bitcoin#26691: Update secp256k1 subtree to libsecp256k1 version 0.2.0 2023-01-13 09:40:57 +00:00
duplicate_inputs.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
examples.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
gcs_filter.cpp Fix clang-tidy readability-const-return-type violations 2023-02-01 11:33:35 +01:00
hashpadding.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
load_external.cpp Use DataStream where possible 2023-01-26 10:44:05 +01:00
lockedpool.cpp clang-tidy: Add performance-inefficient-vector-operation check 2023-03-26 20:17:55 +01:00
logging.cpp bench: document expected results in logging benchmarks 2023-03-07 09:32:55 -08:00
mempool_eviction.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
mempool_stress.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
merkle_root.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
nanobench.cpp Replace current benchmarking framework with nanobench 2020-06-13 12:24:18 +02:00
nanobench.h Update nanobench to version v4.3.10 2023-02-03 07:08:28 +01:00
peer_eviction.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
poly1305.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
pool.cpp Add pool based memory resource & allocator 2023-03-23 19:38:38 +01:00
prevector.cpp Use DataStream where possible 2023-01-26 10:44:05 +01:00
rollingbloom.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
rpc_blockchain.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
rpc_mempool.cpp refactor: add kernel/cs_main.* 2023-01-05 09:05:14 +00:00
strencodings.cpp bench: explicitly make all current benchmarks "high" priority 2022-10-20 10:21:04 -03:00
util_time.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
verify_script.cpp Merge bitcoin/bitcoin#26691: Update secp256k1 subtree to libsecp256k1 version 0.2.0 2023-01-13 09:40:57 +00:00
wallet_balance.cpp refactor: wallet, remove global 'ArgsManager' access 2023-02-15 15:49:45 -03:00
wallet_create_tx.cpp gui: bugfix, getAvailableBalance skips selected coins 2023-04-03 17:23:42 -03:00
wallet_loading.cpp Fix clang-tidy readability-const-return-type violations 2023-02-01 11:33:35 +01:00