Commit graph

935 commits

Author SHA1 Message Date
Fabian Jahr
0930654e1c
build: Add embedded asmap data
The data embedded is from the latest ASMap file from the asmap-data
repository: https://github.com/fjahr/asmap-data/blob/main/1724248800_asmap.dat
2025-03-03 18:07:19 +01:00
merge-script
9491676438
Merge bitcoin/bitcoin#31157: Cleanups to port mapping module post UPnP drop
70398ae05b mapport: make ProcessPCP void (Antoine Poinsot)
9e6cba2988 mapport: remove unnecessary 'g_mapport_enabled' (Antoine Poinsot)
8fb45fcda0 mapport: remove unnecessary 'g_mapport_current' variable (Antoine Poinsot)
1b223cb19b mapport: merge DispatchMapPort into StartMapPort (Antoine Poinsot)
9bd936fa34 mapport: drop unnecessary function (Antoine Poinsot)
2a6536ceda mapport: rename 'use_pcp' to 'enable' (Antoine Poinsot)
c4e82b854c mapport: make 'enabled' and 'current' bool (Antoine Poinsot)

Pull request description:

  Followup to #31130, this does a couple cleanups to `src/mapport.*` to clarify the logic now that there is a single protocol option for port mapping.

ACKs for top commit:
  laanwj:
    Code review ACK 70398ae05b
  TheCharlatan:
    ACK 70398ae05b

Tree-SHA512: d9a3ab4fcd59a7cf4872415c40cc7ac3a98dfc5aa25e195d4df880bb588bac286c30c3471e9d9499de379a75f45dcd0a82019eba3cb9f342004ae1482d0ba075
2025-02-14 11:15:53 +01:00
glozow
6b165f5906
Merge bitcoin/bitcoin#31384: mining: bugfix: Fix duplicate coinbase tx weight reservation
386eecff5f doc: add release notes (ismaelsadeeq)
3eaa0a3b66 miner: init: add `-blockreservedweight` startup option (ismaelsadeeq)
777434a2cd doc: rpc: improve `getmininginfo` help text (ismaelsadeeq)
c8acd4032d init: fail to start when `-blockmaxweight` exceeds `MAX_BLOCK_WEIGHT` (ismaelsadeeq)
5bb31633cc test: add `-blockmaxweight` startup option functional test (ismaelsadeeq)
2c7d90a6d6 miner: bugfix: fix duplicate weight reservation in block assembler (ismaelsadeeq)

Pull request description:

  * This PR attempts to fix the duplicate coinbase weight reservation issue we currently have.
  * Fixes #21950

  We reserve 4000 weight units for coinbase transaction in `DEFAULT_BLOCK_MAX_WEIGHT`

  7590e93bc7/src/policy/policy.h (L23)

  And also reserve additional `4000` weight units in the default `BlockCreationOptions` struct.

  7590e93bc7/src/node/types.h (L36-L40)

  **Motivation**

  - This issue was first noticed during a review here https://github.com/bitcoin/bitcoin/pull/11100#discussion_r136157411)
  - It was later reported in issue #21950.
  - I also came across the bug while writing a test for building the block template. I could not create a block template above `3,992,000` in the block assembler, and this was not documented anywhere. It took me a while to realize that we were reserving space for the coinbase transaction weight twice.

  ---
  This PR fixes this by consolidating the reservation to be in a single location in the codebase.

  This PR then adds a new startup option `-blockreservedweight` whose default is `8000` that can be used to lower or increase the block reserved weight for block header, txs count, coinbase tx.

ACKs for top commit:
  Sjors:
    ACK 386eecff5f
  fjahr:
    Code review ACK 386eecff5f
  glozow:
    utACK 386eecff5f, nonblocking nits. I do think the release notes should be clarified more
  pinheadmz:
    ACK 386eecff5f

Tree-SHA512: f27efa1da57947b7f4d42b9322b83d13afe73dd749dd9cac49360002824dd41c99a876a610554ac2d67bad7485020b9dcc423a8e6748fc79d6a10de6d4357d4c
2025-02-10 08:26:01 -05:00
glozow
982ce10178 add orphanage byte accounting to TxDownloadManagerImpl::CheckIsEmpty() 2025-02-06 15:45:30 -05:00
ismaelsadeeq
3eaa0a3b66
miner: init: add -blockreservedweight startup option
- Prevent setting the value of `-blockreservedweight` below
  a safety value of 2000.
2025-02-04 11:53:11 -05:00
ismaelsadeeq
777434a2cd
doc: rpc: improve getmininginfo help text
- The reserved weight of the coinbase transaction is an estimate and
  may not reflect the exact value; it can be lower.

- It should be clear that `currentblockweight` includes the reserved coinbase transaction weight.
  whereas `currentblocktx` does not account for the coinbase transaction count.

- Also clarify `m_last_block_num_txs` and `m_last_block_weight`
2025-02-04 11:53:11 -05:00
ismaelsadeeq
2c7d90a6d6
miner: bugfix: fix duplicate weight reservation in block assembler
- This commit renamed coinbase_max_additional_weight to block_reserved_weight.

- Also clarify that the reservation is for block header, transaction count
  and coinbase transaction.
2025-02-04 11:53:03 -05:00
merge-script
94ca99ac51
Merge bitcoin/bitcoin#31666: multi-peer orphan resolution followups
7426afbe62 [p2p] assign just 1 random announcer in AddChildrenToWorkSet (glozow)
4c1fa6b28c test fix: make peer who sends MSG_TX announcement non-wtxidrelay (glozow)
2da46b88f0 pass P2PTxInvStore init args to P2PInterface init (glozow)
e3bd51e4b5 [doc] how unique_parents can be empty (glozow)
32eb6dc758 [refactor] assign local variable for wtxid (glozow)
18820ccf6b multi-announcer orphan handling test fixups (glozow)
c4cc61db98 [fuzz] GetCandidatePeers (glozow)
7704139cf0 [refactor] make GetCandidatePeers take uint256 and in-out vector (glozow)
6e4d392a75 [refactor] rename to OrphanResolutionCandidate to MaybeAdd* (glozow)
57221ad979 [refactor] move parent inv-adding to OrphanResolutionCandidate (glozow)

Pull request description:

  Followup to #31397.

  Addressing (in order):
  https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1906077380
  https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1881060842
  https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1905994963
  https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1905999581
  https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1906001592
  https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1905989913
  https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1905920861
  https://github.com/bitcoin/bitcoin/pull/31658#pullrequestreview-2551617694
  https://github.com/bitcoin/bitcoin/pull/31397#discussion_r1917559601

ACKs for top commit:
  instagibbs:
    reACK 7426afbe62
  marcofleon:
    reACK 7426afbe62
  mzumsande:
    Code Review ACK 7426afbe62
  dergoegge:
    Code review ACK 7426afbe62

Tree-SHA512: bca8f576873fdaa20b758e1ee9708ce94e618ff14726864b29b50f0f9a4db58136a286d2b654af569b09433a028901fe6bcdda68dcbfea71e2d1271934725503
2025-02-04 10:10:29 +00:00
Ava Chow
85f96b01b7
Merge bitcoin/bitcoin#30909: wallet, assumeutxo: Don't Assume m_chain_tx_count, Improve wallet RPC errors
9d2d9f7ce2 rpc: Include assumeutxo as a failure reason of rescanblockchain (Fabian Jahr)
595edee169 test, assumeutxo: import descriptors during background sync (Alfonso Roman Zubeldia)
d73ae603d4 rpc: Improve importdescriptor RPC error messages (Fabian Jahr)
27f99b6d63 validation: Don't assume m_chain_tx_count in GuessVerificationProgress (Fabian Jahr)
42d5d53363 interfaces: Add helper function for wallet on pruning (Fabian Jahr)

Pull request description:

  A test that is added as part of #30455 uncovered this issue: The `GuessVerificationProgress` function is used during during descriptor import and relies on `m_chain_tx_count`. In #29370 an [`Assume` was added](0fd915ee6b) expecting the `m_chaint_tx_count` to be set. However, as the test uncovered, `GuessVerificationProgress` is called with background sync blocks that have `m_chaint_tx_count = 0` when they have not been downloaded and processed yet.

  The simple fix is to remove the `Assume`. Users should not be thrown off by the `Internal bug detected` error. The behavior of `importdescriptor` is kept consistent with the behavior for blocks missing due to pruning.

  The test by alfonsoromanz is cherry-picked here to show that the [CI errors](https://cirrus-ci.com/task/5110045812195328?logs=ci#L2535) should be fixed by this change.

  This PR also improves error messages returned by the `importdescriptors` and `rescanblockchain` RPCs. The error message now changes depending on the situation of the node, i.e. if pruning is happening or an assumutxo backgroundsync is active.

ACKs for top commit:
  achow101:
    ACK 9d2d9f7ce2
  mzumsande:
    Code Review ACK 9d2d9f7ce2
  furszy:
    Code review ACK 9d2d9f7ce2

Tree-SHA512: b841a9b371e5eb8eb3bfebca35645ff2fdded7a3e5e06308d46a33a51ca42cc4c258028c9958fbbb6cda9bb990e07ab8d8504dd9ec6705ef78afe0435912b365
2025-01-31 15:45:14 -05:00
Ava Chow
601a6a6917
Merge bitcoin/bitcoin#30965: kernel: Move block tree db open to block manager
0cdddeb224 kernel: Move block tree db open to BlockManager constructor (TheCharlatan)
7fbb1bc44b kernel: Move block tree db open to block manager (TheCharlatan)
57ba59c0cd refactor: Remove redundant reindex check (TheCharlatan)

Pull request description:

  Before this change the block tree db was needlessly re-opened during startup when loading a completed snapshot. Improve this by letting the block manager open it on construction. This also simplifies the test code a bit.

  The change was initially motivated to make it easier for users of the kernel library to instantiate a BlockManager that may be used to read data from disk without loading the block index into a cache.

ACKs for top commit:
  maflcko:
    re-ACK 0cdddeb224 🏪
  achow101:
    ACK 0cdddeb224
  mzumsande:
    re-ACK 0cdddeb224

Tree-SHA512: fe3d557a725367e549e6a0659f64259cfef6aaa565ec867d9a177be0143ff18a2c4a20dd57e35e15f97cf870df476d88c05b03b6a7d9e8d51c568d9eda8947ef
2025-01-31 15:28:06 -05:00
glozow
7426afbe62 [p2p] assign just 1 random announcer in AddChildrenToWorkSet 2025-01-29 18:05:16 -05:00
glozow
e3bd51e4b5 [doc] how unique_parents can be empty 2025-01-29 18:05:16 -05:00
glozow
32eb6dc758 [refactor] assign local variable for wtxid 2025-01-29 18:05:16 -05:00
glozow
7704139cf0 [refactor] make GetCandidatePeers take uint256 and in-out vector
The txrequest fuzzer uses uint256s, not transactions, so it's best if
GetCandidatePeers takes that as an input.
2025-01-29 18:05:16 -05:00
glozow
6e4d392a75 [refactor] rename to OrphanResolutionCandidate to MaybeAdd* 2025-01-29 18:05:09 -05:00
glozow
57221ad979 [refactor] move parent inv-adding to OrphanResolutionCandidate
Deduplicate the logic of adding the parents as announcements to
txrequest. The function can return a bool (indicating whether we're
attempting orphan resolution) instead of the delay.
2025-01-29 18:02:49 -05:00
Sjors Provoost
0713548137
refactor: add GetMinimumTime() helper
Before bip94 there was an assumption that the minimum permitted
timestamp is GetMedianTimePast() + 1.

This commit splits a helper function out of UpdateTime() to
obtain the minimum time in a way that takes the
timewarp attack rule into account.
2025-01-29 09:39:32 +01:00
Ava Chow
9ecc7af41f
Merge bitcoin/bitcoin#31674: init: Lock blocksdir in addition to datadir
2656a5658c tests: add a test for the new blocksdir lock (Cory Fields)
bdc0a68e67 init: lock blocksdir in addition to datadir (Cory Fields)
cabb2e5c24 refactor: introduce a more general LockDirectories for init (Cory Fields)
1db331ba76 init: allow a new xor key to be written if the blocksdir is newly created (Cory Fields)

Pull request description:

  This probably should've been included in #12653 when `-blocksdir` was introduced. Credit TheCharlatan for noticing that it's missing.

  This guards against 2 processes running with separate datadirs but the same blocksdir. I didn't add `walletdir` as I assume sqlite has us covered there.

  It's not likely to happen currently, but may be more relevant in the future with applications using the kernel. Note that the kernel does not currently do any dir locking, but it should.

ACKs for top commit:
  maflcko:
    review ACK 2656a5658c 🏼
  kevkevinpal:
    ACK [2656a56](2656a5658c)
  achow101:
    ACK 2656a5658c
  tdb3:
    Code review and light test ACK 2656a5658c

Tree-SHA512: 3ba17dc670126adda104148e14d1322ea4f67d671c84aaa9c08c760ef778ca1936832c0dc843cd6367e09939f64c6f0a682b0fa23a5967e821b899dff1fff961
2025-01-24 18:15:00 -05:00
merge-script
59876b3ad7
Merge bitcoin/bitcoin#31376: Miner: never create a template which exploits the timewarp bug
733fa0b0a1 miner: never create a template which exploits the timewarp bug (Antoine Poinsot)

Pull request description:

  This check was introduced in #30681 but only enabled for testnet4. To avoid potentially creating an invalid block template if a soft fork to fix the timewarp attack were to activate in the future, we should have this check on all networks. It also seems wise for our miner to not support it whether or not a soft fork activates to fix it at the consensus level.

ACKs for top commit:
  Sjors:
    ACK 733fa0b0a1
  fjahr:
    utACK 733fa0b0a1
  TheCharlatan:
    ACK 733fa0b0a1

Tree-SHA512: 9b3bc8b26a57f93425b17dda80bcfac4ecb750a3d26bc3eb8df619135634e369ac15982fac0c9770b1df207bd2e418ffe02a98f37968f024e55262d97715a4f5
2025-01-23 10:54:47 +00:00
Ryan Ofsky
5d6f6fd00d
Merge bitcoin/bitcoin#31490: refactor: inline UndoWriteToDisk and WriteBlockToDisk to reduce serialization calls
223081ece6 scripted-diff: rename block and undo functions for consistency (Lőrinc)
baaa3b2846 refactor,blocks: remove costly asserts and modernize affected logs (Lőrinc)
fa39f27a0f refactor,blocks: deduplicate block's serialized size calculations (Lőrinc)
dfb2f9d004 refactor,blocks: inline `WriteBlockToDisk` (Lőrinc)
42bc491465 refactor,blocks: inline `UndoWriteToDisk` (Lőrinc)
86b85bb11f bench: add SaveBlockBench (Lőrinc)
34f9a0157a refactor,bench: rename bench/readblock.cpp to bench/readwriteblock.cpp (Lőrinc)

Pull request description:

  `UndoWriteToDisk` and `WriteBlockToDisk` were delegating a subset of their functionality to single-use methods that didn't optimally capture a meaningful chunk of the algorithm, resulting in calculating things twice (serialized size, header size).
  This change inlines the awkward methods (asserting that all previous behavior was retained), and in separate commits makes the usages less confusing.
  Besides making the methods slightly more intuitive, the refactorings reduce duplicate calculations as well.

  The speed difference is insignificant for now (~0.5% for the new `SaveBlockToDiskBench`), but are a cleanup for follow-ups such as https://github.com/bitcoin/bitcoin/pull/31539

ACKs for top commit:
  ryanofsky:
    Code review ACK 223081ece6. Since last review, "Save" was renamed to "Write", uint32_t references were dropped, some log statements and comments were improved as suggested, and a lot of tweaks made to commits and commit messages which should make this easier to review.
  hodlinator:
    ACK 223081ece6
  TheCharlatan:
    ACK 223081ece6
  andrewtoth:
    ACK 223081ece6

Tree-SHA512: 951bc8ad3504c510988afd95c561e3e259c6212bd14f6536fe56e8eb5bf5c35c32a368bbdb1d5aea1acc473d7e5bd9cdcde02008a148b05af1f955e413062d5c
2025-01-22 12:28:18 -05:00
TheCharlatan
0cdddeb224
kernel: Move block tree db open to BlockManager constructor
Make the block db open RAII style by calling it in the BlockManager
constructor.

Before this change the block tree db was needlessly re-opened during
startup when loading a completed snapshot. Improve this by letting the
block manager open it on construction. This also simplifies the test
code a bit.

The change was initially motivated to make it easier for users of the
kernel library to instantiate a BlockManager that may be used to read
data from disk without loading the block index into a cache.
2025-01-20 21:27:50 +01:00
TheCharlatan
7fbb1bc44b
kernel: Move block tree db open to block manager
This commit is done in preparation for the next commit. Here, the block
tree options are moved to the blockmanager options and the block tree is
instantiated through a helper method of the BlockManager, which is
removed again in the next commit.

Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2025-01-20 21:19:39 +01:00
Cory Fields
cabb2e5c24 refactor: introduce a more general LockDirectories for init
No functional change. This is in preparation for adding additional directory
locks on startup.
2025-01-16 21:06:21 +00:00
Cory Fields
1db331ba76 init: allow a new xor key to be written if the blocksdir is newly created
A subsequent commit will add a .lock file to this dir at startup, meaning that
the blocksdir is never empty by the time the xor key is being read/written.

Ignore all hidden files when determining if this is the first run.
2025-01-16 21:06:21 +00:00
TheCharlatan
57ba59c0cd
refactor: Remove redundant reindex check
The check for whether the block tree db has been wiped before calling
NeedsRedownload() is confusing. The boolean is set in case of a reindex.
It was originally introduced to guard NeedsRedownload in case of a
reindex in #21009. However NeedsRedownload already returns early if the
chain's tip is not loaded. Since that is the case during a reindex, the
pre-check is redundant.
2025-01-16 16:35:38 +01:00
merge-script
df8bf65745
Merge bitcoin/bitcoin#31483: kernel: Move kernel-related cache constants to kernel cache
2a92702baf init: Use size_t consistently for cache sizes (TheCharlatan)
65cde3621d kernel: Move default cache constants to caches (TheCharlatan)
8826cae285 kernel: Move non-kernel db cache size constants (TheCharlatan)
e758b26b85 kernel: Move kernel-specific cache size options to kernel (TheCharlatan)
d5e2c4a409 fuzz: Add fuzz test for checked and saturating add and left shift (TheCharlatan)
c03a2795a8 util: Add integer left shift helpers (TheCharlatan)
8bd5f8a38c [refactor] init: Simplify coinsdb cache calculation (TheCharlatan)
5db7d4d3d2 doc: Correct docstring describing max block tree db cache (TheCharlatan)

Pull request description:

  Carrying non-kernel related fields in the cache sizes for the indexes is confusing for kernel library users. The cache sizes are set currently with magic numbers in bitcoin-chainstate. The comments for the cache size calculations are not completely clear. The constants for the cache sizes are also currently in `txdb.h`, which is not an ideal place for holding all cache size related constants.

  Solve these things by moving the kernel-specific cache size fields to their own struct and moving the constants to either the node or the kernel cache sizes.

  This slightly changes the way the cache is allocated if (and only if) the txindex and/or blockfilterindex is used. Since they are now given precedence over the block tree db cache, this results in a bit less cache being allocated to the block tree db, coinsdb and coins caches. The effect is negligible though, i.e. cache sizes with default dbcache reported through the logs are:

  master:
  ```
  Cache configuration:
  * Using 2.0 MiB for block index database
  * Using 56.0 MiB for transaction index database
  * Using 49.0 MiB for basic block filter index database
  * Using 8.0 MiB for chain state database
  * Using 335.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
  ```

  this PR:
  ```
  Cache configuration:
  * Using 2.0 MiB for block index database
  * Using 56.2 MiB for transaction index database
  * Using 49.2 MiB for basic block filter index database
  * Using 8.0 MiB for chain state database
  * Using 334.5 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
  ```

  ---
  This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).

ACKs for top commit:
  stickies-v:
    re-ACK 2a92702baf
  ryanofsky:
    Code review ACK 2a92702baf. Changes since last review are fixing size options to use size_t instead of int64_t again, simplifying CheckedLeftShift more, and making other minor suggested cleanups
  hodlinator:
    re-ACK 2a92702baf

Tree-SHA512: 98376eaa0660b1b8c096a5ce1f3e7c8c30e7cd6644de36856c2d3e573108cfc9473c93ebb3952b7881047b5ae6c85c5b096e6726f30f35be58b98eca07c8c785
2025-01-16 15:04:58 +00:00
merge-script
335798c496
Merge bitcoin/bitcoin#31397: p2p: track and use all potential peers for orphan resolution
86d7135e36 [p2p] only attempt 1p1c when both txns provided by the same peer (glozow)
f7658d9b14 [cleanup] remove p2p_inv from AddTxAnnouncement (glozow)
063c1324c1 [functional test] getorphantxs reflects multiple announcers (glozow)
0da693f7e1 [functional test] orphan handling with multiple announcers (glozow)
b6ea4a9afe [p2p] try multiple peers for orphan resolution (glozow)
1d2e1d709c [refactor] move creation of unique_parents to helper function (glozow)
c6893b0f0b [txdownload] remove unique_parents that we already have (glozow)
163aaf285a [fuzz] orphanage multiple announcer functions (glozow)
22b023b09d [unit test] multiple orphan announcers (glozow)
96c1a822a2 [unit test] TxOrphanage EraseForBlock (glozow)
04448ce32a [txorphanage] add GetTx so that orphan vin can be read (glozow)
e810842acd [txorphanage] support multiple announcers (glozow)
62a9ff1870 [refactor] change type of unique_parents to Txid (glozow)
6951ddcefd [txrequest] GetCandidatePeers (glozow)

Pull request description:

  Part of #27463.

  (Transaction) **orphan resolution** is a process that kicks off when we are missing UTXOs to validate an unconfirmed transaction. We currently request missing parents by txid; BIP 331 also defines a way to [explicitly request ancestors](https://github.com/bitcoin/bips/blob/master/bip-0331.mediawiki#handle-orphans-better).

  Currently, when we find that a transaction is an orphan, we only try to resolve it with the peer who provided the `tx`. If this doesn't work out (e.g. they send a `notfound` or don't respond), we do not try again. We actually can't, because we've already forgotten who else could resolve this orphan (i.e. all the other peers who announced the transaction).

  What is wrong with this? It makes transaction download less reliable, particularly for 1p1c packages which must go through orphan resolution in order to be downloaded.

  Can we fix this with BIP 331 / is this "duct tape" before the real solution?
  BIP 331 (receiver-initiated ancestor package relay) is also based on the idea that there is an orphan that needs resolution, but it's just a new way of communicating information. It's not inherently more honest; you can request ancestor package information and get a `notfound`. So ancestor package relay still requires some kind of procedure for retrying when an orphan resolution attempt fails. See the #27742 implementation which builds on this orphan resolution tracker to keep track of what packages to download (it just isn't rebased on this exact branch). The difference when using BIP 331 is that we request `ancpkginfo` and then `pkgtxns` instead of the parent txids.

  Zooming out, we'd like orphan handling to be:
  - Bandwidth-efficient: don't have too many requests out at once. As already implemented today, transaction requests for orphan parents and regular download both go through the `TxRequestTracker` so that we don't have duplicate requests out.
  - Not vulnerable to censorship: don't give up too easily, use all candidate peers. See e.g. https://bitcoincore.org/en/2024/07/03/disclose_already_asked_for/
  - Load-balance between peers: don't overload peers; use all peers available. This is also useful for when we introduce per-peer orphan protection, since each peer will have limited slots.

  The approach taken in this PR is to think of each peer who announces an orphan as a potential "orphan resolution candidate." These candidates include:
  - the peer who sent us the orphan tx
  - any peers who announced the orphan prior to us downloading it
  - any peers who subsequently announce the orphan after we have started trying to resolve it
  For each orphan resolution candidate, we treat them as having "announced" all of the missing parents to us at the time of receipt of this orphan transaction (or at the time they announced the tx if they do so after we've already started tracking it as an orphan). We add the missing parents as entries to `m_txrequest`, incorporating the logic of typical txrequest processing, which means we prefer outbounds, try not to have duplicate requests in flight, don't overload peers, etc.

ACKs for top commit:
  marcofleon:
    Code review ACK 86d7135e36
  instagibbs:
    reACK 86d7135e36
  dergoegge:
    Code review ACK 86d7135e36
  mzumsande:
    ACK 86d7135e36

Tree-SHA512: 618d523b86e60c3ea039e88326d50db4e55e8e18309c6a20e8f2b10ed9e076f1de0315c335fd3b8abdabcc8b53cbceb66fb59147d05470ea25b83a2b4bd9c877
2025-01-16 13:42:26 +00:00
TheCharlatan
2a92702baf
init: Use size_t consistently for cache sizes
This avoids having to rely on implicit casts when passing them to the
various functions allocating the caches.

This also ensures that if the requested amount of db_cache does not fit
in a size_t, it is clamped to the maximum value of a size_t.

Also take this opportunity to make the total amounts of cache in the
chainstate manager a size_t too.
2025-01-15 15:44:56 +01:00
TheCharlatan
65cde3621d
kernel: Move default cache constants to caches
They are not related to the txdb, so a better place for them is the
new kernel and node cache file. Re-use the default amount of kernel
cache for the default node cache.
2025-01-15 15:44:55 +01:00
TheCharlatan
8826cae285
kernel: Move non-kernel db cache size constants
These have nothing to do with the txdb, so move them out and into the
node caches.
2025-01-15 15:44:44 +01:00
TheCharlatan
e758b26b85
kernel: Move kernel-specific cache size options to kernel
Carrying non-kernel related fields in the cache sizes for the indexes is
confusing for kernel library users. The cache sizes also are set
currently with magic numbers in bitcoin-chainstate. The comments for the
cache size calculations are also not completely clear.

Solve these things by moving the kernel-specific cache size fields to
their own struct.

This slightly changes the way the cache is allocated if the txindex
and/or blockfilterindex is used. Since they are now given precedence
over the block tree db cache, this results in a bit less cache being
allocated to the block tree db, coinsdb and coins caches. The effect is
negligible though, i.e. cache sizes with default dbcache reported
through the logs are:

master:
Cache configuration:
* Using 2.0 MiB for block index database
* Using 56.0 MiB for transaction index database
* Using 49.0 MiB for basic block filter index database
* Using 8.0 MiB for chain state database
* Using 335.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)

this branch:
Cache configuration:
* Using 2.0 MiB for block index database
* Using 56.2 MiB for transaction index database
* Using 49.2 MiB for basic block filter index database
* Using 8.0 MiB for chain state database
* Using 334.5 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
2025-01-15 15:44:16 +01:00
Lőrinc
223081ece6 scripted-diff: rename block and undo functions for consistency
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>

-BEGIN VERIFY SCRIPT-
grep -r -wE 'WriteBlock|ReadRawBlock|ReadBlock|WriteBlockUndo|ReadBlockUndo' $(git ls-files src/ ':!src/leveldb') && \
    echo "Error: One or more target names already exist!" && exit 1
sed -i \
    -e 's/\bSaveBlockToDisk/WriteBlock/g' \
    -e 's/\bReadRawBlockFromDisk/ReadRawBlock/g' \
    -e 's/\bReadBlockFromDisk/ReadBlock/g' \
    -e 's/\bWriteUndoDataForBlock/WriteBlockUndo/g' \
    -e 's/\bUndoReadFromDisk/ReadBlockUndo/g' \
    $(git ls-files src/ ':!src/leveldb')
-END VERIFY SCRIPT-
2025-01-09 15:17:02 +01:00
Lőrinc
baaa3b2846 refactor,blocks: remove costly asserts and modernize affected logs
When the behavior was changes in a previous commit (caching `GetSerializeSize` and avoiding `AutoFile.tell`), (static)asserts were added to make sure the behavior was kept - to make sure reviewers and CI validates it.
We can safely remove them now.

Logs were also slightly modernized since they were trivial to do.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
2025-01-09 15:16:49 +01:00
Lőrinc
fa39f27a0f refactor,blocks: deduplicate block's serialized size calculations
For consistency `UNDO_DATA_DISK_OVERHEAD` was also extracted to avoid the constant's ambiguity.
Asserts were added to help with the review - they are removed in the next commit.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2025-01-09 15:16:28 +01:00
Lőrinc
dfb2f9d004 refactor,blocks: inline WriteBlockToDisk
Similarly, `WriteBlockToDisk` wasn't really extracting a meaningful subset of the `SaveBlockToDisk` functionality, it's tied closely to the only caller (needs the header size twice, recalculated block serializes size, returns multiple branches, mutates parameter).

The inlined code should only differ in these parts (modernization will be done in other commits):
* renamed `blockPos` to `pos` in `SaveBlockToDisk` to match the parameter name;
* changed `return false` to `return FlatFilePos()`.

Also removed remaining references to `SaveBlockToDisk`.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2025-01-09 13:24:53 +01:00
Lőrinc
42bc491465 refactor,blocks: inline UndoWriteToDisk
`UndoWriteToDisk` wasn't really extracting a meaningful subset of the `WriteUndoDataForBlock` functionality, it's tied closely to the only caller (needs the header size twice, recalculated undo serializes size, returns multiple branches, modifies parameter, needs documentation).

The inlined code should only differ in these parts (modernization will be done in other commits):
* renamed `_pos` to `pos` in `WriteUndoDataForBlock` to match the parameter name;
* inlined `hashBlock` parameter usage into `hasher << block.pprev->GetBlockHash()`;
* changed `return false` to `return FatalError`;
* capitalize comment.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2025-01-09 13:18:22 +01:00
TheCharlatan
8bd5f8a38c
[refactor] init: Simplify coinsdb cache calculation
(total_cache / 4) + (1 << 23) is at least 8 MiB and nMaxCoinsDBCache is
also 8 MiB, so the minimum between the two will always be
nMaxCoinsDBCache. This is just a simplification and not changing the
result of the calculation.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2025-01-09 10:21:49 +01:00
ismaelsadeeq
7c123c08dd
miner: add package feerate vector to CBlockTemplate
- The package feerates are ordered by the sequence in which
  packages are selected for inclusion in the block template.

- The commit also tests this new behaviour.

Co-authored-by: willcl-ark <will@256k1.dev>
2025-01-07 15:29:17 -05:00
glozow
86d7135e36 [p2p] only attempt 1p1c when both txns provided by the same peer
Now that we track all announcers of an orphan, it's not helpful to
consider an orphan provided by a peer that didn't send us this parent.
It can only hurt our chances of finding the right orphan when there are
multiple candidates.

Adapt the 2 tests in p2p_opportunistic_1p1c.py that looked at 1p1c
packages from different peers. Instead of checking that the right peer
is punished, we now check that the package is not submitted. We can't
use the functional test to see that the package was not considered
because the behavior is indistinguishable (except for the logs).
2025-01-06 09:02:05 -05:00
glozow
f7658d9b14 [cleanup] remove p2p_inv from AddTxAnnouncement
This param is no longer needed since orphan parent requests are added to
the TxRequestTracker directly.
2025-01-06 09:02:05 -05:00
glozow
b6ea4a9afe [p2p] try multiple peers for orphan resolution
Co-authored-by: dergoegge <n.goeggi@gmail.com>
2025-01-06 09:02:05 -05:00
glozow
1d2e1d709c [refactor] move creation of unique_parents to helper function
This function will be reused in a later commit.
2025-01-06 09:02:05 -05:00
glozow
c6893b0f0b [txdownload] remove unique_parents that we already have
This means we no longer return parents we already have in the
m_unique_parents result from MempoolRejectedTx.

We need to separate the loop that checks AlreadyHave parents from the
loop that adds parents as announcements, because we may do the latter
loop multiple times for different peers.
2025-01-06 09:02:05 -05:00
glozow
62a9ff1870 [refactor] change type of unique_parents to Txid 2025-01-06 09:02:05 -05:00
Fabian Jahr
42d5d53363
interfaces: Add helper function for wallet on pruning 2025-01-05 17:28:19 +01:00
Ryan Ofsky
5bbbc0d0ee
Merge bitcoin/bitcoin#31325: Make m_tip_block std::optional
81cea5d4ee Ensure m_tip_block is never ZERO (Sjors Provoost)
e058544d0e Make m_tip_block an std::optional (Sjors Provoost)

Pull request description:

  Suggested in https://github.com/bitcoin/bitcoin/pull/31297#discussion_r1844244309

ACKs for top commit:
  fjahr:
    re-ACK 81cea5d4ee
  tdb3:
    code review re ACK 81cea5d4ee
  l0rinc:
    ACK 81cea5d4ee

Tree-SHA512: 31a75ba29e3d567bab32e4e7925a419d9d7a4d2d85ed1c1012116d8d22adc14d31d5b4ce5f6c499c994188dcd26a01cced05be74f94c892fc90ae17a6783a472
2024-12-19 10:05:31 -05:00
Ryan Ofsky
c1252b14d7
Merge bitcoin/bitcoin#31520: #31318 followups
4f06ae05ed refactor: fix typo in node/types.h (Sjors Provoost)
366fbf152c test: drop extraneous bracket in mining util (Sjors Provoost)

Pull request description:

  #31318 followups

  Drops an extraneous bracket and fixes a typo.

ACKs for top commit:
  maflcko:
    lgtm ACK 4f06ae05ed
  vasild:
    ACK 4f06ae05ed
  ryanofsky:
    Code review ACK 4f06ae05ed

Tree-SHA512: e168bab124f1e62f6666a21349ee4ac8ac11aadeb04813513e89f6c8f8479a67bcf3490460fd49c8d7f9b7264a32e7ea95ac727dfe4597a59b934017ec9fe44e
2024-12-18 15:41:28 -05:00
Ryan Ofsky
fa0c473d4c
Merge bitcoin/bitcoin#31196: Prune mining interface
c991cea1a0 Remove processNewBlock() from mining interface (Sjors Provoost)
9a47852d88 Remove getTransactionsUpdated() from mining interface (Sjors Provoost)
bfc4e029d4 Remove testBlockValidity() from mining interface (Sjors Provoost)

Pull request description:

  There are three methods in the mining interface that can be dropped. The Template Provider doesn't need them and other application should probably not use them either.

  1. `processNewBlock()` was added in 7b4d3249ce, but became unnecessary with the introduction of interfaces::BlockTemplate::submitSolution in 7b4d3249ce.

  Dropping it was suggested in https://github.com/bitcoin/bitcoin/pull/30200#issuecomment-2404460342

  2. `getTransactionsUpdated()`: this is used in the implementation of #31003 `waitFeesChanged`. It's not very useful generically because the mempool updates very frequently.

  3. `testBlockValidity()`: it might be useful for mining application to have a way to check the validity of a block template they modified, but the Stratum v2 Template Provider doesn't do that, and this method is a bit brittle (e.g. the block needs to build on the tip).

ACKs for top commit:
  TheCharlatan:
    Re-ACK c991cea1a0
  ryanofsky:
    Code review ACK c991cea1a0. Since last review, just rebased to avoid conflicts in surrounding code, and edited a commit message
  tdb3:
    code review ACK c991cea1a0

Tree-SHA512: 2138e54f920b26e01c068b24498c6a210c5c4358138dce0702ab58185d9ae148a18f04c97ac9f043646d40f8031618d80a718a176b1ce4779c237de6fb9c4a67
2024-12-18 14:44:14 -05:00
Ryan Ofsky
ea53568a06
Merge bitcoin/bitcoin#31393: refactor: Move GuessVerificationProgress into ChainstateManager
facb4d010c refactor: Move GuessVerificationProgress into ChainstateManager (MarcoFalke)

Pull request description:

  Currently the function is standalone, which means any passed-in data like `TxData` or the block pointer needs to be taken from the `ChainstateManager` and passed in. This is currently verbose and may become even more verbose if the function is reworked in the future. As the function can not be called without a `ChainstateManager` in production code anyway, make it a member function on the class.

ACKs for top commit:
  ryanofsky:
    Code review ACK facb4d010c. Nice cleanup, that should make this code less awkward to work with
  TheCharlatan:
    ACK facb4d010c
  danielabrozzoni:
    reACK facb4d010c

Tree-SHA512: b17977e12cd7c6e308c47e6a1aa920acecd4442696e46d1f30bd7c201e9898ca2d581ff0bf2cc9f7334e146c1b0c50925adb849c8c17f65dcdf6877be1c5f776
2024-12-18 13:58:10 -05:00
Sjors Provoost
4f06ae05ed
refactor: fix typo in node/types.h 2024-12-18 14:55:35 +07:00