Commit Graph

41465 Commits

Author SHA1 Message Date
Pieter Wuille
ddc184d999 random: get rid of GetRand by inlining 2024-07-01 12:39:53 -04:00
Pieter Wuille
e2d1f84858 random: make GetRand() support entire range (incl. max)
The existing code uses GetRand(nMax), with a default value for nMax, where nMax is the
range of values (not the maximum!) that the output is allowed to take. This will always
miss the last possible value (e.g. GetRand<uint32_t>() will never return 0xffffffff).

Fix this, by moving the functionality largely in RandomMixin, and also adding a
separate RandomMixin::rand function, which returns a value in the entire (non-negative)
range of an integer.
2024-07-01 10:26:46 -04:00
Pieter Wuille
810cdf6b4e tests: overhaul deterministic test randomness
The existing code provides two randomness mechanisms for test purposes:
- g_insecure_rand_ctx (with its wrappers InsecureRand*), which during tests is
  initialized using either zeros (SeedRand::ZEROS), or using environment-provided
  randomness (SeedRand::SEED).
- g_mock_deterministic_tests, which controls some (but not all) of the normal
  randomness output if set, but then makes it extremely predictable (identical
  output repeatedly).

Replace this with a single mechanism, which retains the SeedRand modes to control
all randomness. There is a new internal deterministic PRNG inside the random
module, which is used in GetRandBytes() when in test mode, and which is also used
to initialize g_insecure_rand_ctx. This means that during tests, all random numbers
are made deterministic. There is one exception, GetStrongRandBytes(), which even
in test mode still uses the normal PRNG state.

This probably opens the door to removing a lot of the ad-hoc "deterministic" mode
functions littered through the codebase (by simply running relevant tests in
SeedRand::ZEROS mode), but this isn't done yet.
2024-07-01 10:26:46 -04:00
Pieter Wuille
6cfdc5b104 random: convert XoRoShiRo128PlusPlus into full RNG
Convert XoRoShiRo128PlusPlus into a full RandomMixin-based RNG class,
providing all utility functionality that FastRandomContext has. In doing so,
it is renamed to InsecureRandomContext, highlighting its non-cryptographic
nature.

To do this, a fillrand fallback is added to RandomMixin (where it is used by
InsecureRandomContext), but FastRandomContext still uses its own fillrand.
2024-07-01 10:26:46 -04:00
Pieter Wuille
8cc2f45065 random: move XoRoShiRo128PlusPlus into random module
This is preparation for making it more generally accessible.
2024-07-01 10:26:46 -04:00
Pieter Wuille
8f5ac0d0b6 xoroshiro128plusplus: drop comment about nonexisting copy() 2024-07-01 10:26:46 -04:00
Pieter Wuille
8924f5120f random: modernize XoRoShiRo128PlusPlus a bit
Make use of C++20 functions in XoRoShiRo128PlusPlus.
2024-07-01 10:26:46 -04:00
Pieter Wuille
ddb7d26cfd random: add RandomMixin::randbits with compile-known bits
In many cases, it is known at compile time how many bits are requested from
randbits. Provide a variant of randbits that accepts this number as a template,
to make sure the compiler can make use of this knowledge. This is used immediately
in rand32() and randbool(), and a few further call sites.
2024-07-01 10:26:46 -04:00
Pieter Wuille
21ce9d8658 random: Improve RandomMixin::randbits
The previous randbits code would, when requesting more randomness than available
in its random bits buffer, discard the remaining entropy and generate new.

Benchmarks show that it's usually better to first consume the existing randomness
and only then generate new ones. This adds some complexity to randbits, but it
doesn't weigh up against the reduced need to generate more randomness.
2024-07-01 10:26:46 -04:00
Pieter Wuille
9b14d3d2da random: refactor: move rand* utilities to RandomMixin
Rather than make all the useful types of randomness be exclusive to
FastRandomContext, move it to a separate RandomMixin class where it can be reused by
other RNGs.

A Curiously Recurring Template Pattern (CRTP) is used for this, to provide the ability
for individual RNG classes to override one or more randomness functions, without
needing the runtime-cost of virtual classes.

Specifically, RNGs are expected to only provide fillrand and rand64, while all others
are derived from those:
- randbits
- randrange
- randbytes
- rand32
- rand256
- randbool
- rand_uniform_delay
- rand_uniform_duration
- min(), max(), operator()(), to comply with C++ URBG concept.
2024-07-01 10:26:46 -04:00
Pieter Wuille
40dd86fc3b random: use BasicByte concept in randbytes 2024-07-01 10:26:46 -04:00
Pieter Wuille
27cefc7fd6 random: add a few noexcepts to FastRandomContext 2024-07-01 10:26:46 -04:00
Pieter Wuille
b3b382dde2 random: move rand256() and randbytes() to .h file 2024-07-01 10:26:46 -04:00
Pieter Wuille
493a2e024e random: write rand256() in function of fillrand() 2024-07-01 10:26:46 -04:00
glozow
0bd2bd1efb
Merge bitcoin/bitcoin#30237: test: Add Compact Block Encoding test ReceiveWithExtraTransactions covering non-empty extra_txn
55eea003af test: Make blockencodings_tests deterministic (AngusP)
4c99301220 test: Add ReceiveWithExtraTransactions Compact Block receive test. (AngusP)
4621e7cc8f test: refactor: Rename extra_txn to const empty_extra_txn as it is empty in all test cases (AngusP)

Pull request description:

  This test uses the `extra_txn` (`vExtraTxnForCompact`) vector of optional orphan/conflicted/etc. transactions to provide transactions to a PartiallyDownloadedBlock that are not otherwise present in the mempool, and check that they are used.

  This also covers a former nullptr deref bug that was fixed in #29752 (bf031a517c) where the `extra_txn` vec/circular-buffer was null-initialized and not yet filled when dereferenced in `PartiallyDownloadedBlock::InitData`.

ACKs for top commit:
  marcofleon:
    Code review ACK 55eea003af. I ran the `blockencodings` unit test and no issues with the new test case.
  dergoegge:
    Code review ACK 55eea003af
  glozow:
    ACK 55eea003af

Tree-SHA512: d7909c212bb069e1f6184b26390a5000dcc5f2b18e49b86cceccb9f1ec4f874dd43bc9bc92abd4207c71dd78112ba58400042c230c42e93afe55ba51b943262c
2024-07-01 14:11:52 +01:00
merge-script
4c573e5718
Merge bitcoin/bitcoin#30306: fuzz: Improve stability for txorphan and mini_miner harnesses
e009bf681c Don't use iterator addresses in IteratorComparator (dergoegge)

Pull request description:

  See #29018.

  Stability for `txorphan` is now >90%. `mini_miner` needs further investigation, stability still low (although slightly improved by this PR) at ~62%.

ACKs for top commit:
  marcofleon:
    Tested ACK e009bf681c. Using afl++, stability for `txorphan` went from 82% to ~94% and for `mini_miner` it went from 84% to 97%. I ran them both using the corpora in qa-assets.
  glozow:
    utACK e009bf681c

Tree-SHA512: 6d0a20fd7ceedca8e702d8adde5fca500d8b0187147aee8d43b4e9eb5176dcacf60180f42a7158f037d18dbb27e479b6c069a0f3c912226505cbff5aa073a415
2024-07-01 12:11:27 +01:00
merge-script
c3b446a494
Merge bitcoin/bitcoin#30273: fuzz: FuzzedSock::Recv() don't lose bytes from MSG_PEEK read
4d81b4de33 fuzz: FuzzedSock::Recv() don't lose bytes from MSG_PEEK read (Vasil Dimov)
b51d75ea97 fuzz: simplify FuzzedSock::m_peek_data (Vasil Dimov)

Pull request description:

  Problem:

  If `FuzzedSock::Recv(N, MSG_PEEK)` is called then `N` bytes would be
  retrieved from the fuzz provider, saved in `m_peek_data` and returned
  to the caller (ok).

  If after this `FuzzedSock::Recv(M, 0)` is called where `M < N`
  then the first `M` bytes from `m_peek_data` would be returned
  to the caller (ok), but the remaining `N - M` bytes in `m_peek_data`
  would be discarded/lost (not ok). They must be returned by a subsequent
  `Recv()`.

  To resolve this, only remove the head `N` bytes from `m_peek_data`.

  ---

  This is a followup to https://github.com/bitcoin/bitcoin/pull/30211, more specifically:

  https://github.com/bitcoin/bitcoin/pull/30211#discussion_r1633199919
  https://github.com/bitcoin/bitcoin/pull/30211#discussion_r1633216366

ACKs for top commit:
  marcofleon:
    ACK 4d81b4de33. Tested this with the I2P fuzz target and there's no loss in coverage. I think overall this is an improvement in the robustness of `Recv` in `FuzzedSock`.
  dergoegge:
    Code review ACK 4d81b4de33
  brunoerg:
    utACK 4d81b4de33

Tree-SHA512: 73b5cb396784652447874998850e45899e8cba49dcd2cc96b2d1f63be78e48201ab88a76cf1c3cb880abac57af07f2c65d673a1021ee1a577d0496c3a4b0c5dd
2024-07-01 11:58:58 +01:00
merge-script
2f813154ef
Merge bitcoin/bitcoin#30358: scripted-diff: Log parameter interaction not thrice
fa1bc7c88b scripted-diff: Log parameter interaction not thrice (MarcoFalke)
fafb7875e1 doc: Fix outdated dev comment about logging (MarcoFalke)

Pull request description:

  Seems a bit overkill to log the words "parameter interaction" thrice, when at least once is enough. So do that.

  Before:

  ```
  2024-06-28T15:30:57Z [init.cpp:745] [InitParameterInteraction] InitParameterInteraction: parameter interaction: -connect or -maxconnections=0 set -> setting -dnsseed=0
  ```

  After:

  ```
  2024-06-28T15:47:27Z [init.cpp:745] [InitParameterInteraction] parameter interaction: -connect or -maxconnections=0 set -> setting -dnsseed=0

ACKs for top commit:
  paplorinc:
    ACK fa1bc7c88b
  fjahr:
    utACK fa1bc7c88b
  TheCharlatan:
    Nice, ACK fa1bc7c88b
  hodlinator:
    utACK fa1bc7c88b

Tree-SHA512: 83cd92e20dffa38737d4fd31764481284383e12671d9e4b33cfa496743c95c10921a113b1da2caafeb44fca3759a28a8e230df5e30c29fb55d5854ff1531382c
2024-07-01 11:56:23 +01:00
merge-script
b3c22e0c72
Merge bitcoin/bitcoin#30362: test: p2p: check that connecting to ourself leads to disconnect
5d2fb14baf test: p2p: check that connecting to ourself leads to disconnect (Sebastian Falbesoner)

Pull request description:

  This small PR adds test coverage for the scenario of connecting to ourself, leading to an immediate disconnect:
  2f6dca4d1c/src/net_processing.cpp (L3729-L3735)

  This logic has been first introduced by Satoshi in October 2009, together with a couple of other changes and a version bump to "v0.1.6 BETA" (see commit cc0b4c3b62).

ACKs for top commit:
  kevkevinpal:
    tACK [5d2fb14](5d2fb14baf)
  maflcko:
    ACK 5d2fb14baf
  fjahr:
    tACK 5d2fb14baf
  tdb3:
    ACK 5d2fb14baf

Tree-SHA512: 30fb8c82cef94701affeca386ecd59daa32231635fa770fe225feb69fdab2ffedbfa157edd563f65099ec209f2dafffc1154f7f9292c2ea68bbd114750904875
2024-07-01 10:32:52 +01:00
Sebastian Falbesoner
5d2fb14baf test: p2p: check that connecting to ourself leads to disconnect
The "connect to ourself" detection logic has been first introduced
by Satoshi in October 2009, together with a couple of other changes
and a version bump to "v0.1.6 BETA" (see commit
cc0b4c3b62).
2024-06-29 00:30:14 +02:00
MarcoFalke
fa1bc7c88b
scripted-diff: Log parameter interaction not thrice
-BEGIN VERIFY SCRIPT-
 sed -i 's/LogPrintf("%s: \(parameter interaction: .*\)", __func__/LogInfo("\1"/g' ./src/init.cpp
-END VERIFY SCRIPT-
2024-06-28 17:46:00 +02:00
MarcoFalke
fafb7875e1
doc: Fix outdated dev comment about logging 2024-06-28 17:37:58 +02:00
Ryan Ofsky
2f6dca4d1c
Merge bitcoin/bitcoin#30335: Mining interface followups, reduce cs_main locking, test rpc bug fix
a74b0f93ef Have testBlockValidity hold cs_main instead of caller (Sjors Provoost)
f6dc6db44d refactor: use CHECK_NONFATAL to avoid single-use symbol (Sjors Provoost)
5fb2b70489 Drop unneeded lock from createNewBlock (Sjors Provoost)
75ce7637ad refactor: testBlockValidity make out argument last (Sjors Provoost)
83a9bef0e2 Add missing include for mining interface (Sjors Provoost)

Pull request description:

  Followups from #30200

  Fixes:
  - `std::unique_ptr` needs `#include <memory>` (noticed while working on #30332, which has fewer includes than its parent PR that I originally tested with)
  - Drop lock from createNewBlock that was spuriously added
  - Have testBlockValidity hold cs_main instead of caller (also fixes a race condition in test-only code)

  Refactor:
  - Use CHECK_NONFATAL to avoid single-use symbol (refactor)
  - move output argument `state` to the end of `testBlockValidity`, see https://github.com/bitcoin/bitcoin/pull/30200#discussion_r1647987176

ACKs for top commit:
  AngusP:
    Code Review ACK a74b0f93ef
  itornaza:
    Tested ACK a74b0f93ef
  ryanofsky:
    Code review ACK a74b0f93ef. Just new error string is added since last review, and a commit message was updated

Tree-SHA512: 805e133bb59303fcee107d6f02b3e2761396c290efb731a85e6a29ae56b4b1b9cd28ada9629e979704dcfd98cf35034e7e6b618e29923049eb1eca2f65630e41
2024-06-27 18:16:27 -04:00
Ryan Ofsky
d38dbaad98
Merge bitcoin/bitcoin#28167: init: Add option for rpccookie permissions (replace 26088)
73f0a6cbd0 doc: detail -rpccookieperms option (willcl-ark)
d2afa2690c test: add rpccookieperms test (willcl-ark)
f467aede78 init: add option for rpccookie permissions (willcl-ark)
7df03f1a92 util: add perm string helper functions (willcl-ark)

Pull request description:

  This PR picks up #26088 by aureleoules which adds a bitcoind launch option `-rpccookieperms` to set the file permissions of the cookie generated by bitcoin core.

  Example usage to make the generated cookie group-readable: `./src/bitcoind -rpccookieperms=group`.

  Accepted values for `-rpccookieperms` are `[owner|group|all]`. We let `fs::perms` handle platform-specific permissions changes.

ACKs for top commit:
  achow101:
    ACK 73f0a6cbd0
  ryanofsky:
    Code review ACK 73f0a6cbd0. Main change since last review is no longer throwing a skip exception in the rpc test on windows, so other checks can run after it, and overall test result is passing, not skipped. Also were clarifying renames and documentation improvements.
  tdb3:
    cr ACK 73f0a6cbd0

Tree-SHA512: e800d59a44aca10e1c58ca69bf3fdde9f6ccf5eab4b7b962645af6d6bc0cfa3a357701e409c8c60d8d7744fcd33a91e77ada11790aa88cd7811ef60fab86ab11
2024-06-27 17:35:08 -04:00
Ava Chow
f0745d028e
Merge bitcoin/bitcoin#30050: refactor, wallet: get serialized size of CRecipients directly
a9c7300135 move-only: refactor CreateTransactionInternal (josibake)
adc6ab25bb wallet: use CRecipient instead of CTxOut (josibake)

Pull request description:

  Broken out from #28201

  ---

  In order to estimate fees properly, we need to know what the final serialized transaction size will be. This PR refactors `CreateTransactionInternal` to:

  * Get the serialized size directly from the `CRecipient`: this sets us up in a future PR to calculate the serialized size of silent payment `CTxDestinations` (see 797e21c8c1)
  * Use the new `GetSerializeSizeForRecipient` to move the serialize size calculation to *before* coin selection and the output creation to *after* coin selection: this also sets us up for silent payments sending in a future PR in that silent payments outputs cannot be created until after the inputs to the transaction have been selected

  Aside from the silent payments use case, I think this structure logically makes more sense. As a reminder, move-only commits are best reviewed with something like `git diff -w --color-moved=dimmed-zebra`

ACKs for top commit:
  S3RK:
    reACK a9c7300135
  achow101:
    ACK a9c7300135
  rkrux:
    tACK [a9c7300](a9c7300135)

Tree-SHA512: 412e1764b98f7428c8530c3a68f55e32063d6b66ab2ff613e1c7e12d49b049807cb60055cfe7f7e8ffe7ac7f0f9931427cbfd3efe7d4f97a5a0f6d1bf1aaac58
2024-06-27 13:59:46 -04:00
willcl-ark
73f0a6cbd0
doc: detail -rpccookieperms option
Co-authored-by: tdb3 <106488469+tdb3@users.noreply.github.com>
2024-06-27 15:08:27 +01:00
willcl-ark
d2afa2690c
test: add rpccookieperms test
Tests various perms on non-Windows OSes
2024-06-27 15:08:23 +01:00
willcl-ark
f467aede78
init: add option for rpccookie permissions
Add a bitcoind launch option `-rpccookieperms` to configure the file
permissions of the cookie on Unix systems.
2024-06-27 15:08:19 +01:00
willcl-ark
7df03f1a92
util: add perm string helper functions
PermsToSymbolicString will convert from fs::perms to string type
'rwxrwxrwx'.

InterpretPermString will convert from a user-supplied "perm string" such
as 'owner', 'group' or 'all, into appropriate fs::perms.
2024-06-27 14:55:10 +01:00
Sjors Provoost
a74b0f93ef
Have testBlockValidity hold cs_main instead of caller
The goal of interfaces is to eventually run in their own process,
so we can't use EXCLUSIVE_LOCKS_REQUIRED in their declaration.

However TestBlockValidaty will crash (in its call to ConnectBlock)
if the tip changes from under the proposed block.

Have the testBlockValidity implementation  hold the lock instead,
and non-fatally check for this condition.
2024-06-27 08:58:25 +02:00
Sjors Provoost
f6dc6db44d
refactor: use CHECK_NONFATAL to avoid single-use symbol 2024-06-27 08:58:24 +02:00
Sjors Provoost
5fb2b70489
Drop unneeded lock from createNewBlock
This was added in 4bf2e361da, but
BlockAssembler::CreateNewBlock already locks cs_main internally.
2024-06-27 08:56:20 +02:00
Ava Chow
b27afb7fb7
Merge bitcoin/bitcoin#29833: i2p: fix and improve logs
7d3662fbe3 i2p: fix log when an interruption happens during `Accept` (brunoerg)
3d3a83fab2 i2p: log errors properly according to their severity (brunoerg)

Pull request description:

  This PR improves and fixes i2p logs (joint work with vasild).

  - It replaces `LogPrint` to `LogPrintLevel` so we can log according to the severity.
  - Fix log when interruption happens during `Accept`. Before this PR, when an interruption happens, it just logs "Error accepting:", no reason is logged as it does for other situations. This PR changes it to log "Accept interrupted".
  - Log errors according to the severity. Stuff like creating SAM session, destroying SAM session, etc... are logged as 'debug'.

ACKs for top commit:
  achow101:
    ACK 7d3662fbe3
  marcofleon:
    ACK 7d3662fbe3.
  vasild:
    ACK 7d3662fbe3

Tree-SHA512: 1c3d92108dbc22833f37a78e18b4efd723433d10f28166d17c74eab884cd97e908b4e0a0908fd16288df895eb2eb480f781de37b2ec6a6d414abfb71e0c86fe2
2024-06-26 15:28:26 -04:00
Ava Chow
9ac4f69ec2
Merge bitcoin/bitcoin#30334: Update libsecp256k1 subtree to latest master
1408944d2e Squashed 'src/secp256k1/' changes from 06bff6dec8..4af241b320 (fanquake)

Pull request description:

  Updates the libsecp256k1 subtree to f473c959f0. This includes a number of CMake related changes, including one that prevents CMake from segfaulting when we were configuring the subtree. A number of these changes have come from the review/discussion in https://github.com/hebasto/bitcoin/pull/192:

  * https://github.com/bitcoin-core/secp256k1/pull/1529
  * https://github.com/bitcoin-core/secp256k1/pull/1532
  * https://github.com/bitcoin-core/secp256k1/pull/1535
  * https://github.com/bitcoin-core/secp256k1/pull/1543
  * https://github.com/bitcoin-core/secp256k1/pull/1545
  * https://github.com/bitcoin-core/secp256k1/pull/1546

  Also includes:

  * https://github.com/bitcoin-core/secp256k1/pull/1488
  * https://github.com/bitcoin-core/secp256k1/pull/1517
  * https://github.com/bitcoin-core/secp256k1/pull/1533
  * https://github.com/bitcoin-core/secp256k1/pull/1548
  * https://github.com/bitcoin-core/secp256k1/pull/1550

ACKs for top commit:
  achow101:
    ACK cc58e958f3
  TheCharlatan:
    ACK cc58e958f3
  hebasto:
    re-ACK cc58e958f3.
  real-or-random:
    utACK cc58e958f3

Tree-SHA512: 41409bc7f65bd17a9feb5c0455e2de2d291a25e4ce14e4a01fe25fcf9d45c64ddf55f274c17d1c86a63ab6b4870997ab79c65ec2795e5b3b49502823770c500f
2024-06-26 12:37:28 -04:00
Ava Chow
1d00601b9b
Merge bitcoin/bitcoin#30309: wallet: notify when preset + automatic inputs exceed max weight
72b226882f wallet: notify when preset + automatic inputs exceed max weight (furszy)

Pull request description:

  Small change. Found it while finishing my review on #29523. This does not interfere with it.

  Basically, we are erroring out early when the automatic coin selection process exceeds the maximum weight, but we are not doing so when the user-preselected inputs combined with the wallet-selected inputs exceed the maximum weight.
  This change avoids signing all inputs before erroring out and introduces test coverage for `fundrawtransaction`.

ACKs for top commit:
  achow101:
    ACK 72b226882f
  tdb3:
    re ACK for 72b226882f
  rkrux:
    tACK [72b2268](72b226882f)
  ismaelsadeeq:
    utACK 72b226882f

Tree-SHA512: d77be19231023383a9c79a5d66b642dcbc6ebfc31a363e0b9f063c44898720a7859ec211cdbc0914ac7a3bfdf15e52fb8fc20d97f171431f70492c0f159dbc36
2024-06-26 12:16:16 -04:00
merge-script
d3d2bbf576
Merge bitcoin/bitcoin#30327: build: Drop redundant sys/sysctl.h header check
c0b5ea5901 build: Drop redundant `sys/sysctl.h` header check (Hennadii Stepanov)

Pull request description:

  The `AC_CHECK_HEADERS` macro defines `HAVE_SYS_SYSCTL_H` if the `sys/sysctl.h` header is found. However, in the source code, this header is guarded by `HAVE_SYSCTL` and `HAVE_SYSCTL_ARND` macros, which have their own checks. Since `HAVE_SYS_SYSCTL_H` is not used, we can skip the `AC_CHECK_HEADERS(... sys/sysctl.h ...)` check.

ACKs for top commit:
  laanwj:
    ACK c0b5ea5901
  fanquake:
    ACK c0b5ea5901 - we could got the other way, and add nested #defs, but that doesn't seem worthwhile.

Tree-SHA512: 73bc4bbfc5c457cd2c38e40f8e57d2a70c06ef661d76d4148d683d262be45b9405b8cda1958ac611c312ca7d9e2f9624cf2cac1b61f1008af0856875c62f0eac
2024-06-26 15:25:46 +01:00
merge-script
0c57a798b5
Merge bitcoin/bitcoin#29987: guix: build with glibc 2.31
b5fc6d46a3 guix: use glibc 2.31 (fanquake)

Pull request description:

  Set minimum required glibc to 2.31.
  The glibc 2.31 branch is still maintained: https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.31/master.

  Remove the stack-protector check from test-security-check, as the test
  no-longer fails, and given the control we have of the end, the actual
  security-check test seems sufficient (this might also be applied to some
  of the other checks).

  Drops runtime support for Ubuntu Bionic 18.04 and RHEL-8 from the release binaries.

ACKs for top commit:
  TheCharlatan:
    ACK b5fc6d46a3

Tree-SHA512: ba7e727240fa0ebebfb8b749024c71cbfdec37c33b39627866d78f9318ccdc687fd5103a63ee0e98cf809d9954dde56b1b305691c33d1de275ed0519f716c921
2024-06-26 15:21:44 +01:00
merge-script
3f0ee7655b
Merge bitcoin/bitcoin#30007: chainparams: Add achow101 DNS seeder
2721d64989 chainparams: Add achow101 DNS seeder (Ava Chow)

Pull request description:

  I wrote a [DNS seeder](https://github.com/achow101/dnsseedrs) and have been running it for the past 2 months now. I believe it is ready/good enough to be used as an additional DNS seeder for all of our supported public networks.

ACKs for top commit:
  laanwj:
    ACK 2721d64989
  1440000bytes:
    ~~reACK 2721d64989~~
  mzumsande:
    ACK 2721d64989
  willcl-ark:
    reACK 2721d64989

Tree-SHA512: 857a6cf7dd33962f0008a89db4d6b57d3c6aa622704cdcca6ab710babeead3a2970d9a6fa190949c7bbf7cb7d006e814d6314be3d8c8180eed29013c7c1ac7e1
2024-06-26 11:43:58 +01:00
merge-script
b4b9854394
Merge bitcoin/bitcoin#30321: rest: don't copy data when sending binary response
1556d21599 rest: don't copy data when sending binary response (Roman Zeyde)

Pull request description:

  Also, change `HTTPRequest::WriteReply` to accept `std::span`.

ACKs for top commit:
  laanwj:
    re-ACK 1556d21599
  stickies-v:
    ACK 1556d21599

Tree-SHA512: 3e563d8072f0e1b90b00f85adb140d4e5fef169b6882a837b08d1e8391b64c21bea3c4256c4e2a624ac1fb3d374f12a1cc16dc59b2155ec857728162d1daaceb
2024-06-26 11:42:06 +01:00
Sjors Provoost
75ce7637ad
refactor: testBlockValidity make out argument last 2024-06-26 12:24:48 +02:00
Sjors Provoost
83a9bef0e2
Add missing include for mining interface
Needed for std::unique_ptr
2024-06-26 12:24:47 +02:00
merge-script
2cd7c6bd93
Merge bitcoin/bitcoin#30147: contrib: Fixup verify-binaries OS platform parsing
3ab2520190 contrib: Fixup verify-binaries OS platform parsing (Ben Westgate)

Pull request description:

  Closes #30145.

  This PR solves two major issues in the `parse_version_string` function of verify-binaries:
  1. `-aarch64` binaries cannot be specifically downloaded. The -platform string gets interpreted as a release candidate that doesn't exist due to containing sub-string "rc".
  2. Specifying a platform with a "-" in the name causes the parser to ignore both "-platform" AND "-rcN" and download the potentially wrong (non-rc) version for every platform. This also prevented specifying just one platform binary the user wished to download.

  It also updates the accompanying `test.py` to cover problem two and adds two examples that were formerly broken to `README.md` to show what is now possible. Including the most useful case of downloading only 1 specific platform's binary.

  This improves the Bitcoin verify-binaries tools user experience by not:
  1. Failing to download for inexplicable reasons,
  2. Downloading more files than what the user told it to, or in the worst case
  3. Downloading only the wrong files.

  * A test was added to cover the command `verify-binaries/verify.py pub 22.0-x86_64-linux-gnu.tar.gz` which checks that _bitcoin-22.0-x86_64-linux-gnu.tar.gz_ downloads successfully AND ONLY _bitcoin-22.0-x86_64-linux-gnu.tar.gz_ downloads.
  * The steps to reproduce each bug are in the referenced issue #30145. Explanation of the potential issue as well as reasoning for the way the bug was fixed are in my commit descriptions.
  * This delivers the promised feature of "only download the binaries for a certain platform", by allowing strings with '-' to be accepted, allowing for single file downloads for any specific platform which was not always possible before.
  * Removes 6 lines of code from the offending `parse_version_string` function, while fixing the bugs/errors, and extending the functionality to be practical for users with slow connections.
  * Makes the error message more helpful when no file matches the provided platform string, now prints "Did you mean: `closest-match`" to help correct typos.

  Thanks for reading my PR. I look forward to getting this helpful tool in its best shape yet.

  Log of this branch passing the new test.py:
  ```
  python3 test.py
  ✓ 'Nonexistent version should fail' passed
  ✓ 'Malformed version should fail' passed
  ✓ '--min-good-sigs 20 should fail' passed
  - testing verification (22.0-x86_64-linux-gnu.tar.gz)
  ✓ '22.0-x86_64-linux-gnu.tar.gz should succeed' passed
  - testing verification (22.0)
  ✓ '22.0 should succeed' passed
  ```

  Log of master failing the new test.py:
  ```
  python3 test.py
  ✓ 'Nonexistent version should fail' passed
  ✓ 'Malformed version should fail' passed
  ✓ '--min-good-sigs 20 should fail' passed
  - testing verification (22.0-x86_64-linux-gnu.tar.gz)
  ✓ '22.0-x86_64-linux-gnu.tar.gz should succeed' passed
  Traceback (most recent call last):
    File "/home/ben/Documents/GitHub/bitcoin/contrib/verify-binaries/test.py", line 74, in <module>
      main()
    File "/home/ben/Documents/GitHub/bitcoin/contrib/verify-binaries/test.py", line 27, in main
      assert len(v) == 1
             ^^^^^^^^^^^
  AssertionError
  ```

ACKs for top commit:
  stickies-v:
    re-ACK 3ab2520190
  willcl-ark:
    re-ACK 3ab2520190

Tree-SHA512: 6093228bb876cd0ac84d1cd2630074e17a3f09c4b23325b9419d859a5721a802f928844575233b135df52b882287dd18d6fadf4419d88ec0a2cdf82db315329e
2024-06-26 10:28:44 +01:00
merge-script
be4df59696
Merge bitcoin/bitcoin#30312: contrib: add R(UN)PATH check to ELF symbol-check
4289dd02cc contrib: add R(UN)PATH check to ELF symbol-check (fanquake)

Pull request description:

  Our binaries shouldn't contain any rpaths, or runpaths, so check that at release time.

  Guix build (aarch64):
  ```bash
  14f1b54936f71aaf8fedb987e1c2f5642c34ac35f4856cdbd7bf7b4a9f42507c  guix-build-4289dd02cce6/output/aarch64-linux-gnu/SHA256SUMS.part
  b120503ac4a37c160aa1bdc662348a2a662cb9b3d0477daa177e92afacab6a27  guix-build-4289dd02cce6/output/aarch64-linux-gnu/bitcoin-4289dd02cce6-aarch64-linux-gnu-debug.tar.gz
  6416e3678aa13301ba2327e65dcd83afefd15d21c96c1b574cf616a65466a260  guix-build-4289dd02cce6/output/aarch64-linux-gnu/bitcoin-4289dd02cce6-aarch64-linux-gnu.tar.gz
  b7f11fa4abc034b32c208a9d642a0cafc0de693d49c8f3b28d1cf2d318af6826  guix-build-4289dd02cce6/output/arm-linux-gnueabihf/SHA256SUMS.part
  7cf6269025b10bc5ad65cd98baba68b4b65d3caabcd4d62e7af892bbb16c253f  guix-build-4289dd02cce6/output/arm-linux-gnueabihf/bitcoin-4289dd02cce6-arm-linux-gnueabihf-debug.tar.gz
  c136665417cc1f9254f4541c1161ea6d24d2325e4baa2dd4a306f67a1edf9c20  guix-build-4289dd02cce6/output/arm-linux-gnueabihf/bitcoin-4289dd02cce6-arm-linux-gnueabihf.tar.gz
  37dd0359e073cbb38dcd6e17953d31e414ce27516c31b18ef6aa1bc9530dbed2  guix-build-4289dd02cce6/output/arm64-apple-darwin/SHA256SUMS.part
  3d01f5db06e1b1cbebc5a5a4277ad3da8911afe35e111d4650e2beee6038cfdb  guix-build-4289dd02cce6/output/arm64-apple-darwin/bitcoin-4289dd02cce6-arm64-apple-darwin-unsigned.tar.gz
  9d8ad7cbdc1e154b33c138de21e7a0ffc8a8857b522db9272c70d5c10732460c  guix-build-4289dd02cce6/output/arm64-apple-darwin/bitcoin-4289dd02cce6-arm64-apple-darwin-unsigned.zip
  23f14eeaeac2e881e41543a796dfa65f132e89153281f9fbcb8019d08fff7eb2  guix-build-4289dd02cce6/output/arm64-apple-darwin/bitcoin-4289dd02cce6-arm64-apple-darwin.tar.gz
  b62d9e202e5414ebefe129e733beaee76b4f9b05a7dd20807fbbdbe24f27f692  guix-build-4289dd02cce6/output/dist-archive/bitcoin-4289dd02cce6.tar.gz
  1301b33745b9016bbb7f038ee2004f630244deaa6575b6c75391cda2d8d3455d  guix-build-4289dd02cce6/output/powerpc64-linux-gnu/SHA256SUMS.part
  54e988c3d3fc204d4b232dcf24df8a80733c46e3adf9ea0c81a1637609fc1131  guix-build-4289dd02cce6/output/powerpc64-linux-gnu/bitcoin-4289dd02cce6-powerpc64-linux-gnu-debug.tar.gz
  6acb3348d67d3e96a6f3a6e782ae088532b75d4b38cbfe249e114e07eafefdb6  guix-build-4289dd02cce6/output/powerpc64-linux-gnu/bitcoin-4289dd02cce6-powerpc64-linux-gnu.tar.gz
  cc30cfe72eefa498f347d825c57a19501d0c24330d2b4bcdfa432cf7a17880e8  guix-build-4289dd02cce6/output/riscv64-linux-gnu/SHA256SUMS.part
  a56144f8a816c6b62970c29d6c805fe7d26c4d1171e96379e1e6a53c7cb504f6  guix-build-4289dd02cce6/output/riscv64-linux-gnu/bitcoin-4289dd02cce6-riscv64-linux-gnu-debug.tar.gz
  7d67eac59846d478b5ecefa1396d3214c2f9b394ec4de4c099b2b2304750e13e  guix-build-4289dd02cce6/output/riscv64-linux-gnu/bitcoin-4289dd02cce6-riscv64-linux-gnu.tar.gz
  2d1b1ec32f566ad37b7fb6e2d8b150389c0d02013fc99b12e9e93cb938cfdbc7  guix-build-4289dd02cce6/output/x86_64-apple-darwin/SHA256SUMS.part
  1bb7a360974cc839499f82cf00dc6e59c3584a3094382086217eacb97b6da931  guix-build-4289dd02cce6/output/x86_64-apple-darwin/bitcoin-4289dd02cce6-x86_64-apple-darwin-unsigned.tar.gz
  821a1587e285a8514a9f2ed9bec7d422e91d5f1db3cbb9e4c5ce4467298e692f  guix-build-4289dd02cce6/output/x86_64-apple-darwin/bitcoin-4289dd02cce6-x86_64-apple-darwin-unsigned.zip
  15956787628f5014adc53fe98fc995c56424c8e6c1e698283f65099751a61dbd  guix-build-4289dd02cce6/output/x86_64-apple-darwin/bitcoin-4289dd02cce6-x86_64-apple-darwin.tar.gz
  76e8a8275f27a5d2b4eb29c0440c5dfbecb0d9348362ff7defe989820d5ec80b  guix-build-4289dd02cce6/output/x86_64-linux-gnu/SHA256SUMS.part
  3c8b19f1a130ebb12a1ca4103fa283e71d909f09da5cef8047db1e7e7dfea162  guix-build-4289dd02cce6/output/x86_64-linux-gnu/bitcoin-4289dd02cce6-x86_64-linux-gnu-debug.tar.gz
  87b3735cda595556e8adfd4de5c9cc966e9996cdb76b8f9dfa8c978d37397d40  guix-build-4289dd02cce6/output/x86_64-linux-gnu/bitcoin-4289dd02cce6-x86_64-linux-gnu.tar.gz
  fc535f76e6323fb0feeeee790865d0d2c8ea9d2c85d72a620acfb83546fb7628  guix-build-4289dd02cce6/output/x86_64-w64-mingw32/SHA256SUMS.part
  9fd5d3b4e91fa8c4d137c9f8d2089c81f4c4165bb6ab9a404b163f7b363587fd  guix-build-4289dd02cce6/output/x86_64-w64-mingw32/bitcoin-4289dd02cce6-win64-debug.zip
  b4658cdc94e70cd7b810a44770684b742d96d7d40d720cc54f209784e0ad4ad6  guix-build-4289dd02cce6/output/x86_64-w64-mingw32/bitcoin-4289dd02cce6-win64-setup-unsigned.exe
  e4557d1f8c1b0eb5014ad56d2a17a0a618c3ba7e6d073628e633382e1d4191aa  guix-build-4289dd02cce6/output/x86_64-w64-mingw32/bitcoin-4289dd02cce6-win64-unsigned.tar.gz
  3f9a104633f6962b0b8efb64a7d6be453f56617561e7f5b49a84ae191942ea5d  guix-build-4289dd02cce6/output/x86_64-w64-mingw32/bitcoin-4289dd02cce6-win64.zip
  ```

ACKs for top commit:
  theuni:
    utACK 4289dd02cc
  hebasto:
    ACK 4289dd02cc.

Tree-SHA512: f2e1f6525f699e0d069c70f5e7f1a7e8f8758dcdaff778d49e671bc106a9592f9a7b25c4dbeeaf92b654646fd3b902ad45c8547fd1b28e3287933e8c1497333d
2024-06-26 10:27:44 +01:00
Roman Zeyde
1556d21599
rest: don't copy data when sending binary response
Also, change `HTTPRequest::WriteReply` to accept `std::span`.
2024-06-26 06:47:30 +03:00
Ben Westgate
3ab2520190 contrib: Fixup verify-binaries OS platform parsing
Parse platform strings with "-" or '.' correctly such as "linux-gnu" or
"x86_64-linux-gnu.tar.gz" to download the matching files or file. String
partition() is used to tolerate more dashes. Update `VERSION_EXAMPLE`
with a new string parsed correctly now. Fix "-aarch64" interpreted as a
release candidate due to sub-string "rc", causing all downloads to fail.
Now "rc" must immediately follow first "-" to indicate an [-rc] string.
Local variables `version_rc`, `version_os` renamed to `rc`, `platform`.
If "-rcN" is specified, `platform` is reassigned to remove the '-rcN'.

Changes are useful to only download one bitcoin core binary on slow
connections. Making `verify.py pub` more intuitive, robust, and
versatile. Closes #30145

When user types a platform string not found in any filename lets help
and say the platform closest to what they typed in a `f"No files
matched the platform specified. Did you mean: {closest_match}"` log.
Improves UX when unaware how we name our files.
Uses the difflib Python built-in which was already imported elsewhere.

Update test.py to test single file verification
verify-binaries/verify.py can accept an entire filename filter for its
"-platform" parameter now so let us test that it succeeds and downloads
and verifies only one file. `verify.py pub 22.0-x86_64-linux-gnu.tar.gz`
should get and verify only the requested binary. It is placed before the
existing <version> wide verification as it is a faster test and possibly
easier to break.

Update doc with examples now possible after bugfix
Add example to show release candidates now work with "-platform" strings
containing "-" and string provided can be from the middle of filename:
`./contrib/verify-binaries/verify.py --json pub 23.0-rc5-linux-gnu`
Change example 5 to not match example 3.
New examples to show platform can now be provided specifically enough to
download only a single binary down to its file extension:
`./contrib/verify-binaries/verify.py pub 25.2-x86_64-linux`
`./contrib/verify-binaries/verify.py pub 24.1-rc1-darwin`
`./contrib/verify-binaries/verify.py pub 27.0-win64-setup.exe`
This is the most common use if not verifying all files so users see it
as the first example for "only download the binaries for a certain
architecture and/or platform". Downloading one file is intuitively what
most will think this meant and this change delivers on that expectation.

Co-authored-by: stickies-v
2024-06-25 11:32:56 -05:00
fanquake
cc58e958f3
Update secp256k1 subtree to latest master 2024-06-25 15:01:00 +01:00
fanquake
1408944d2e Squashed 'src/secp256k1/' changes from 06bff6dec8..4af241b320
4af241b320 Merge bitcoin-core/secp256k1#1535: build: Replace hardcoded "auto" value with default one
f473c959f0 Merge bitcoin-core/secp256k1#1543: cmake: Do not modify build types when integrating by downstream project
d403eea484 Merge bitcoin-core/secp256k1#1546: cmake: Rename `SECP256K1_LATE_CFLAGS` and switch to Bitcoin Core's approach
d7ae25ce6f Merge bitcoin-core/secp256k1#1550: fix: typos in secp256k1.c
0e2fadb20c fix: typos in secp256k1.c
69b2192ad4 Merge bitcoin-core/secp256k1#1545: cmake: Do not set `CTEST_TEST_TARGET_ALIAS`
5dd637f3cf Merge bitcoin-core/secp256k1#1548: README: mention ellswift module
7454a53736 README: mention ellswift module
4706be2cd0 cmake: Reimplement `SECP256K1_APPEND_CFLAGS` using Bitcoin Core approach
c2764dbb99 cmake: Rename `SECP256K1_LATE_CFLAGS` to `SECP256K1_APPEND_CFLAGS`
f87a3589f4 cmake: Do not set `CTEST_TEST_TARGET_ALIAS`
158f9e5eae cmake: Do not modify build types when integrating by downstream project
35c0fdc86b Merge bitcoin-core/secp256k1#1529: cmake: Fix cache issue when integrating by downstream project
4392f0f717 Merge bitcoin-core/secp256k1#1533: tests: refactor: tidy up util functions (#1491)
bedffd53d8 Merge bitcoin-core/secp256k1#1488: ci: Add native macOS arm64 job
4b8d5eeacf Merge bitcoin-core/secp256k1#1532: cmake: Disable eager MSan in ctime_tests
f55703ba49 autotools: Delete unneeded compiler test
396e885886 autotools: Align MSan checking code with CMake's implementation
abde59f52d cmake: Report more compiler details in summary
7abf979a43 cmake: Disable `ctime_tests` if build with `-fsanitize=memory`
4d9645bee0 cmake: Remove "AUTO" value of `SECP256K1_ECMULT_GEN_KB` option
a06805ee74 cmake: Remove "AUTO" value of `SECP256K1_ECMULT_WINDOW_SIZE` option
1791f6fce4 Merge bitcoin-core/secp256k1#1517: autotools: Disable eager MSan in ctime_tests
26b94ee92a autotools: Remove "auto" value of `--with-ecmult-gen-kb` option
122dbaeb37 autotools: Remove "auto" value of `--with-ecmult-window` option
e73f6f8fd9 tests: refactor: drop `secp256k1_` prefix from testrand.h functions
0ee7453a99 tests: refactor: add `testutil_` prefix to testutil.h functions
0c6bc76dcd tests: refactor: move `random_` helpers from tests.c to testutil.h
0fef8479be tests: refactor: rename `random_field_element_magnitude` -> `random_fe_magnitude`
59db007f0f tests: refactor: rename `random_group_element_...` -> `random_ge_...`
ebfb82ee2f ci: Add job with -fsanitize-memory-param-retval
e1bef0961c configure: Move "experimental" warning to bottom
55e5d975db autotools: Disable eager MSan in ctime_tests
ec4c002faa cmake: Simplify `PROJECT_IS_TOP_LEVEL` emulation
cae9a7ad14 cmake: Do not set emulated PROJECT_IS_TOP_LEVEL as cache variable
218f0cc93b ci: Add native macOS arm64 job

git-subtree-dir: src/secp256k1
git-subtree-split: 4af241b32099067464e015fa66daac5096206dea
2024-06-25 15:01:00 +01:00
Ryan Ofsky
323b0acfcb
Merge bitcoin/bitcoin#30200: Introduce Mining interface
a9716c53f0 rpc: call IsInitialBlockDownload via miner interface (Sjors Provoost)
dda0b0834f rpc: minize getTipHash() calls in gbt (Sjors Provoost)
7b4d3249ce rpc: call processNewBlock via miner interface (Sjors Provoost)
9e228351e7 rpc: getTransactionsUpdated via miner interface (Sjors Provoost)
64ebb0f971 Always pass options to BlockAssembler constructor (Sjors Provoost)
4bf2e361da rpc: call CreateNewBlock via miner interface (Sjors Provoost)
404b01c436 rpc: getblocktemplate getTipHash() via Miner interface (Sjors Provoost)
d8a3496b5a rpc: call TestBlockValidity via miner interface (Sjors Provoost)
8ecb681678 Introduce Mining interface (Sjors Provoost)

Pull request description:

  Introduce a `Mining` interface for the `getblocktemplate`, `generateblock` and other mining RPCs to use now, and for Stratum v2 to use later.

  Suggested here: https://github.com/bitcoin/bitcoin/pull/29346#issuecomment-2108528652

  The selection of methods added to the interface is mostly based on what the Template Provider in #29432 uses. It could be expanded further so that `rpc/mining.cpp` no longer needs `EnsureMemPool` and `EnsureChainman`.

  This PR should be a pure refactor.

ACKs for top commit:
  tdb3:
    re ACK a9716c53f0
  itornaza:
    Code review and std-tests ACK a9716c53f0
  ryanofsky:
    Code review ACK a9716c53f0 with one minor suggestion in case you update. Only changes since last review were other small changes to the interface.

Tree-SHA512: cf97f87d6e9ed89da3835a0730da3b24a7b14c8605ea221149103a5915e79598cf082a95f2bc88e33f1c450e3d4aad88aed1163a29195acca88bcace055af724
2024-06-24 19:29:48 -04:00
merge-script
a57da5e014
Merge bitcoin/bitcoin#30308: QA: Expect PACKAGE_NAME rather than constant "Bitcoin Core"
197b5404b0 QA: Expect PACKAGE_NAME rather than constant "Bitcoin Core" (Luke Dashjr)

Pull request description:

  Followup to #29144

ACKs for top commit:
  kevkevinpal:
    ACK [197b540](197b5404b0)
  tdb3:
    ACK 197b5404b0

Tree-SHA512: 6a2c7f7da56effa7e3eba1d103b1b4442d74a21f2ba588564cddd6d61a46c3721bf0942d4ac947ecbbbfe476501ab7b03a8414d7d0840ce9106b056811583010
2024-06-24 15:17:27 +01:00
merge-script
aef5ac7f2c
Merge bitcoin/bitcoin#29876: build: add -Wundef
e3dc64f499 build: add -Wundef (fanquake)
82b43955f7 refactor: use #ifdef HAVE_SOCKADDR_UN (fanquake)
40cd7585a0 randomenv: use ifdef over if (fanquake)
7839503b30 zmq: use #ifdef ENABLE_ZMQ (fanquake)
79e197b175 build: Suppress warnings from boost and capnproto in multiprocess code (Ryan Ofsky)

Pull request description:

  Turn on `-Wundef`.

  [> Warn if an undefined identifier is evaluated in an #if directive. Such identifiers are replaced with zero.](https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wundef).

  Note that this is still beneficial with CMake, and may even be nice to have enabled prior, to catch any change in behaviour.

  If we end up with this enabled, it should probably be enough to fix #16419.

ACKs for top commit:
  hebasto:
    ACK e3dc64f499, I have reviewed the code and it looks OK.

Tree-SHA512: 73436ead07f3a09ba0d30f7105df50d9b2ec8452f11e866bc1c7ebc10c005772ee77fedaa125f444175663c04dfc472f98c2699c63711da356089b66a8cc3e0a
2024-06-24 15:15:34 +01:00