Bitcoin Core integration/staging tree
Go to file
Ryan Ofsky 2f53f2273d
Merge bitcoin/bitcoin#29975: blockstorage: Separate reindexing from saving new blocks
e41667b720 blockstorage: Don't move cursor backwards in UpdateBlockInfo (Ryan Ofsky)
17103637c6 blockstorage: Rename FindBlockPos and have it return a FlatFilePos (Martin Zumsande)
d9e477c4dc validation, blockstorage: Separate code paths for reindex and saving new blocks (Martin Zumsande)
064859bbad blockstorage: split up FindBlockPos function (Martin Zumsande)
fdae638e83 doc: Improve doc for functions involved in saving blocks to disk (Martin Zumsande)
0d114e3cb2 blockstorage: Add Assume for fKnown / snapshot chainstate (Martin Zumsande)

Pull request description:

  `SaveBlockToDisk` / `FindBlockPos` are used for two purposes, depending on whether they are called during reindexing (`dbp` set,  `fKnown = true`) or in the "normal" case when adding new blocks (`dbp == nullptr`,  `fKnown = false`).
  The actual tasks are quite different
  - In normal mode, preparations for saving a new block are made, which is then saved: find the correct position on disk (maybe skipping to a new blk file), check for available disk space, update the blockfile info db, save the block.
  - during reindex, most of this is not necessary (the block is already on disk after all), only the blockfile info needs to rebuilt because reindex wiped the leveldb it's saved in.

  Using one function with many conditional statements for this leads to code that is hard to read / understand and bug-prone:
  - many code paths in `FindBlockPos` are conditional on `fKnown` or `!fKnown`
  - It's not really clear what actually needs to be done during reindex (we don't need to "save a block to disk" or "find a block pos" as the function names suggest)
  - logic that should be applied to only one of the two modes is sometimes applied to both (see first commit, or #27039)

  #24858 and #27039 were recent bugs directly related to the differences between reindexing and normal mode, and in both cases the simple fix took a long time to be reviewed and merged.

  This PR proposes to clean this code up by splitting out the reindex logic into a separate function (`UpdateBlockInfo`) which will be called directly from validation. As a result, `SaveBlockToDisk` and `FindBlockPos` only need to cover the non-reindex logic.

ACKs for top commit:
  paplorinc:
    ACK e41667b720
  TheCharlatan:
    Re-ACK e41667b720
  ryanofsky:
    Code review ACK e41667b720. Just improvements to comments since last review.

Tree-SHA512: a14ff9a0facf6b1e3c1cd724a2d19a79a25d4b48de64398fdd172671532a472bc10a20cbb64ac3a3e55814dcc877d0597a3e1699cabc4f9d9a86b439b6eaba20
2024-05-16 11:16:08 -04:00
.github ci: Roll test-each-commit Ubuntu 2024-05-15 09:53:04 +02:00
.tx qt: Bump Transifex slug for 27.x 2024-02-07 09:24:32 +00:00
build_msvc Merge bitcoin/bitcoin#29494: build: Assume HAVE_CONFIG_H, Add IWYU pragma keep to bitcoin-config.h includes 2024-05-07 14:14:03 -04:00
build-aux/m4 Add lint check for bitcoin-config.h include IWYU pragma 2024-05-01 08:33:43 +02:00
ci ci: Exclude feature_init for now in valgrind task 2024-05-07 08:53:18 +02:00
contrib Merge bitcoin/bitcoin#30074: contrib: use ENV flags in get_arch 2024-05-15 09:02:32 +08:00
depends depends: set RANLIB for CMake 2024-05-13 20:01:45 +08:00
doc Merge bitcoin/bitcoin#27101: Support JSON-RPC 2.0 when requested by client 2024-05-16 10:18:04 -04:00
share contrib: rpcauth.py - Add new option (-j/--json) to output text in json format 2024-04-25 08:32:28 -05:00
src Merge bitcoin/bitcoin#29975: blockstorage: Separate reindexing from saving new blocks 2024-05-16 11:16:08 -04:00
test Merge bitcoin/bitcoin#27101: Support JSON-RPC 2.0 when requested by client 2024-05-16 10:18:04 -04:00
.cirrus.yml ci: Skip git install if it is already installed 2024-02-16 16:06:45 +01:00
.editorconfig ci: Drop AppVeyor CI integration 2021-09-07 06:12:53 +03:00
.gitattributes Separate protocol versioning from clientversion 2014-10-29 00:24:40 -04:00
.gitignore Merge bitcoin/bitcoin#29733: build, macos: Drop unused osx_volname target 2024-04-02 14:57:22 +01:00
.python-version Bump .python-version from 3.9.17 to 3.9.18 2023-10-24 18:51:24 +02:00
.style.yapf Update .style.yapf 2023-06-01 23:35:10 +05:30
autogen.sh build: make sure we can overwrite config.{guess,sub} 2023-06-13 14:58:43 +02:00
configure.ac build: swap otool for (llvm-)objdump 2024-05-08 16:36:41 +08:00
CONTRIBUTING.md Merge bitcoin/bitcoin#29645: doc: update release-process.md 2024-04-30 20:17:04 -04:00
COPYING doc: upgrade Bitcoin Core license to 2024 2024-01-10 16:29:01 -06:00
INSTALL.md doc: Added hyperlink for doc/build 2021-09-09 19:53:12 +05:30
Makefile.am Merge bitcoin/bitcoin#29739: build: swap cctools otool for llvm-objdump 2024-05-11 18:34:42 +08:00
README.md doc: Explain Bitcoin Core in README.md 2022-05-10 07:49:09 +02:00
SECURITY.md Update security.md contact for achow101 2023-12-14 18:14:54 -05:00

Bitcoin Core integration/staging tree

https://bitcoincore.org

For an immediately usable, binary version of the Bitcoin Core software, see https://bitcoincore.org/en/download/.

What is Bitcoin Core?

Bitcoin Core connects to the Bitcoin peer-to-peer network to download and fully validate blocks and transactions. It also includes a wallet and graphical user interface, which can be optionally built.

Further information about Bitcoin Core is available in the doc folder.

License

Bitcoin Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is regularly built (see doc/build-*.md for instructions) and tested, but it is not guaranteed to be completely stable. Tags are created regularly from release branches to indicate new official, stable release versions of Bitcoin Core.

The https://github.com/bitcoin-core/gui repository is used exclusively for the development of the GUI. Its master branch is identical in all monotree repositories. Release branches and tags do not exist, so please do not fork that repository unless it is for development reasons.

The contribution workflow is described in CONTRIBUTING.md and useful hints for developers can be found in doc/developer-notes.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The CI (Continuous Integration) systems make sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Bitcoin Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.