Commit Graph

39924 Commits

Author SHA1 Message Date
fanquake
e3b68b3b83
Merge bitcoin/bitcoin#28875: build: Pass sanitize flags to instrument libsecp256k1 code
cbea49c0d3 build: Pass sanitize flags to instrument `libsecp256k1` code (Hennadii Stepanov)

Pull request description:

  This PR is a revived https://github.com/bitcoin/bitcoin/pull/27991 with an addressed [comment](https://github.com/bitcoin/bitcoin/pull/27991#discussion_r1252148488).

  Fixes https://github.com/bitcoin/bitcoin/issues/27990.

  Might be tested as follows:
  ```
  $ ./autogen.sh && ./configure --enable-fuzz --with-sanitizers=fuzzer CC=clang-13 CXX=clang++-13
  $ make clean > /dev/null && make
  $ objdump --disassemble=secp256k1_xonly_pubkey_serialize src/test/fuzz/fuzz | grep __sanitizer_cov
   1953bd0:e8 bb c6 05 ff       call   9b0290 <__sanitizer_cov_trace_const_cmp8>
   1953d32:e8 69 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
   1953d58:e8 43 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
   1953d82:e8 19 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
  ```

ACKs for top commit:
  fanquake:
    ACK cbea49c0d3
  dergoegge:
    reACK cbea49c0d3

Tree-SHA512: 801994e75b711d20eaf0d675f378da07d693f4a7de026efd93860f5f1deabed855a83eca3561725263e4fe605fcc5f91eb73c021ec91c831864e6deb575e3885
2024-01-26 11:31:34 +00:00
Ava Chow
717103bcce
Merge bitcoin/bitcoin#29315: refactor: Compile unreachable walletdb code
fa3373d3ad refactor: Compile unreachable code (MarcoFalke)

Pull request description:

  When unreachable code isn't compiled, compile failures are not detected.

  Fix this by leaving it unreachable, but compiling it.

  Fixes https://github.com/bitcoin/bitcoin/pull/28999#discussion_r1465010916

ACKs for top commit:
  achow101:
    ACK fa3373d3ad
  ryanofsky:
    Code review ACK fa3373d3ad. This looks good, and should prevent code in the else blocks from accidentally breaking.

Tree-SHA512: 3a3764915dfc935bf5d7a48f1ca151dcbac340c1cbdce8236b24ae9b4f04d6ee9771ed058ca60bcbca6e19d13671de3517f828a8f7ab6444c7cc4e3538d1ba4e
2024-01-25 17:16:09 -05:00
Ava Chow
36720994a4
Merge bitcoin/bitcoin#20827: During IBD, prune as much as possible until we get close to where we will eventually keep blocks
d298ff8b62 During IBD, prune as much as possible until we get close to where we will eventually keep blocks (Luke Dashjr)

Pull request description:

  This should reduce pruning flushes even more, speeding up IBD with pruning on systems that have a sufficient dbcache.

  Assumes 1 MB per block between tip and best header chain. Simply adds this to the buffer pruning is trying to leave available, which results in pruning almost everything up until we get close to where we need to be keeping blocks.

ACKs for top commit:
  andrewtoth:
    ACK d298ff8b62
  fjahr:
    utACK d298ff8b62
  achow101:
    ACK d298ff8b62

Tree-SHA512: 2a482376bfb177e2ba7c2f0bb0b58b02efdb38b34755a18d1fc3e869df5959c85b6f1009e1386fa8b89c4f90d520383e36bd3e21dec221042315134efb1a455b
2024-01-25 15:20:17 -05:00
fanquake
ac923e70e7
Merge bitcoin/bitcoin#29287: depends: Do not override CFLAGS when building SQLite with DEBUG=1
5fb8f0f80f depends: Do not override CFLAGS when building SQLite with DEBUG=1 (Hennadii Stepanov)
2b0dd88f1c depends: Ensure definitions are passed when building SQLite with DEBUG=1 (Hennadii Stepanov)

Pull request description:

  The `--enable-debug` configure option for the SQLite package does two things:
  ```autoconf
  #-----------------------------------------------------------------------
  #   --enable-debug
  #
  AC_ARG_ENABLE(debug, [AS_HELP_STRING(
    [--enable-debug], [build with debugging features enabled [default=no]])],
    [], [])
  AC_MSG_CHECKING([Build type])
  if test x"$enable_debug" = "xyes"; then
    BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_DEBUG -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE"
    CFLAGS="-g -O0"
    AC_MSG_RESULT([debug])
  else
    AC_MSG_RESULT([release])
  fi
  #-----------------------------------------------------------------------
  ```

  It adds three preprocessor definitions and overrides `CFLAGS` with `"-g -O0"`. The latter breaks the user's ability to provide sanitizer and LTO flags.

  This PR might be especially useful for OSS-Fuzz where `DEBUG=1` has been used since https://github.com/google/oss-fuzz/pull/10503.

  Also it makes a workaround for building SQLite for 32-bit unneeded. For details, please refer to https://github.com/hebasto/oss-fuzz/tree/240120-sqlite.

  Changes in https://github.com/bitcoin/bitcoin/pull/29282 might not be strictly required now. However, I consider them an improvement.

ACKs for top commit:
  fanquake:
    ACK 5fb8f0f80f - downstream is also green, so i'll fixup the PR there.

Tree-SHA512: 8593d8a0237ebb270d5da763fb65ed642ab8ed0d44e57704a34154621f49e3d5c58b462cc0070251fa1ba556c58a3c7d3620530d6839dc6dc9e0887010330eca
2024-01-25 15:46:56 +00:00
MarcoFalke
fa3373d3ad
refactor: Compile unreachable code
When unreachable code isn't compiled, compile failures are not detected.

Fix this by leaving it unreachable, but compiling it.

Fixes https://github.com/bitcoin/bitcoin/pull/28999#discussion_r1465010916

Can be reviewed with --ignore-all-space
2024-01-25 16:25:55 +01:00
fanquake
7699a1aab8
Merge bitcoin/bitcoin#29313: ci: Update cache action
ec25e74542 ci: Update cache action (Hennadii Stepanov)

Pull request description:

  This PR fixes deprecation [warnings](https://github.com/bitcoin/bitcoin/actions/runs/7652979339) for Node.js 16 actions in the GHA CI:
  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/ea7b0708-8b2f-446f-a16d-ecc2c8a1da45)

  See:
  - https://github.com/marketplace/actions/cache
  - https://github.com/actions/cache/releases/tag/v4.0.0

Top commit has no ACKs.

Tree-SHA512: 48503abab5d188d6fac2a1ead62512c217a831f611c4dce0e05666d72fac4db26f947cbe9a42fda0307cbdcb9aa0bd4b4d7a15ac2c14c757f92ba2916da0020b
2024-01-25 14:23:41 +00:00
Hennadii Stepanov
5fb8f0f80f
depends: Do not override CFLAGS when building SQLite with DEBUG=1
The `--enable-debug` configure option for the SQLite package does two
things. It adds three preprocessor definitions and overrides CFLAGS with
"-g -O0". The latter breaks the user's ability to provide sanitizer and
LTO flags.
2024-01-25 12:25:27 +00:00
Hennadii Stepanov
2b0dd88f1c
depends: Ensure definitions are passed when building SQLite with DEBUG=1
The SQLite build system overrides the `CFLAGS` when is configured with
the `--enable-debug` option.
2024-01-25 12:23:49 +00:00
Hennadii Stepanov
ec25e74542
ci: Update cache action
This change fixes deprecation warnings for Node.js 16 actions in the GHA
CI.

See:
- https://github.com/marketplace/actions/cache
- https://github.com/actions/cache/releases/tag/v4.0.0
2024-01-25 11:55:57 +00:00
fanquake
4ad83ef09b
Merge bitcoin/bitcoin#29205: build: always set -g -O2 in CORE_CXXFLAGS
00c1e2aa44 build: fix optimisation flags used for --coverage (fanquake)
1dc2c9b385 ci: cleanup C*FLAG usage in Valgrind jobs (fanquake)
6cc2a38c13 build: add sanitizer flags to configure output (fanquake)
08cd5aca18 build: always set -g -O2 in CORE_CXXFLAGS (fanquake)

Pull request description:

  Rather than trying to sporadically rely on / override Autoconf default behaviour. Just always override (if unset), and always set the flags we want (which are the same as the Autoconf defaults).

  Removes the need for duplicate code to clear (if not overridden) `CXXFLAGS`.

  Fixes cases of "missing" `-O2`. i.e this PR when running a Valgrind CI job with changes here:
  ```bash
  CXXFLAGS        =  -g -O2  -fdebug-prefix-map=$(abs_top_srcdir)=.  -Wstack-protector -fstack-protector-all -mbranch-protection=bti   -Werror  -fsanitize=fuzzer  -gdwarf-4
  ```

  Fixes configure output to reflect actual compilation flag ordering, so it's useful.

  Note that if we do still end up with a duplicate "-g -O2" when compiling, that has no effect, and I don't really thinks it's something worth trying to optimize.

ACKs for top commit:
  TheCharlatan:
    lgtm ACK 00c1e2aa44
  hebasto:
    ACK 00c1e2aa44, I have reviewed the code and it looks OK. Also tested `ci/test/00_setup_env_native_valgrind.sh`.
  theuni:
    ACK 00c1e2aa44

Tree-SHA512: cf6c7acf813ba10b198561e83eb72e9b2532a39cb1767c452d031e82921dcd42a47b129735b24c4e36131fd0c8fe7457f7cae870c1e011cdfdd430bdc4d4912b
2024-01-25 10:12:56 +00:00
Ava Chow
207220ce8b
Merge bitcoin/bitcoin#29302: wallet: clarify replaced_by_txid and replaces_txid in help output
ff54314d4a wallet: clarify replaced_by_txid and replaces_txid in help output (marco)

Pull request description:

  Resolves issue #27781

ACKs for top commit:
  achow101:
    ACK ff54314d4a
  ryanofsky:
    Code review ACK ff54314d4a. Seems like a helpful clarification

Tree-SHA512: b13a0e24505dfaee083467ac6f357b96460b5d1841dc29c4df4a503c290d379cef3d50fcc76f33bbc95741f484dd9d2461b0c2e8bdebf57a8a72edfbeece2a79
2024-01-24 13:04:27 -05:00
fanquake
ea4ddd8652
Merge bitcoin/bitcoin#29304: fuzz: Exit and log stderr for parse_test_list errors
9d09c873a5 fuzz: Exit and log stderr for parse_test_list errors (dergoegge)

Pull request description:

  We should log all errors that occur when attempting to print the harness list in the fuzz test runner.

ACKs for top commit:
  maflcko:
    lgtm ACK 9d09c873a5

Tree-SHA512: 50471b732c8cbe287dacba14487e7c8a5826f146432d93aa3bb55d063a8ba158d01641d6cb1360241dd4cd54ef5e045b0412f9cc34d06c181134921d1f1ceced
2024-01-24 15:14:16 +00:00
dergoegge
9d09c873a5 fuzz: Exit and log stderr for parse_test_list errors 2024-01-24 11:42:30 +00:00
marco
ff54314d4a wallet: clarify replaced_by_txid and replaces_txid in help output 2024-01-23 17:34:16 -07:00
Ava Chow
e69796c79c
Merge bitcoin/bitcoin#28560: wallet, rpc: FundTransaction refactor
18ad1b9142 refactor: pass CRecipient to FundTransaction (josibake)
5ad19668db refactor: simplify `CreateRecipients` (josibake)
47353a608d refactor: remove out param from `ParseRecipients` (josibake)
f7384b921c refactor: move parsing to new function (josibake)
6f569ac903 refactor: move normalization to new function (josibake)
435fe5cd96 test: add tests for fundrawtx and sendmany rpcs (josibake)

Pull request description:

  ## Motivation

  The primary motivation for this PR is to enable `FundTransaction` to take a vector of `CRecipient` objects to allow passing BIP352 silent payment addresses to RPCs that use `FundTransaction` (e.g. `send`, `walletcreatefundedpsbt`). To do that, SFFO logic needs to be moved out of `FundTransaction` so the `CRecipient` objects with the correct SFFO information can be created and then passed to `FundTransaction`.

  As a secondary motivation, this PR moves the SFFO stuff closer to the caller, making the code cleaner and easier to understand. This is done by having a single function which parses RPC inputs for SFFO and consistently using the `set<int>` method for communicating SFFO.

  I'm also not convinced we need to pass a full `CMutableTx` object to `FundTransaction`, but I'm leaving that for a follow-up PR/discussion, as its not a blocker for silent payments.

ACKs for top commit:
  S3RK:
    reACK 18ad1b9142
  josibake:
    > According to my `range-diff` nothing changed. reACK [18ad1b9](18ad1b9142)
  achow101:
    ACK 18ad1b9142

Tree-SHA512: d61f017cf7d98489ef216475b68693fd77e7b53a26a6477dcd73e7e5ceff5036b2d21476e377839e710bb73644759d42c4f9f4b14ed96b3e56ed87b07aa6d1a7
2024-01-23 16:40:58 -05:00
Ava Chow
2f218c664b
Merge bitcoin/bitcoin#28921: multiprocess: Add basic type conversion hooks
6acec6b9ff multiprocess: Add type conversion code for UniValue types (Ryan Ofsky)
0cc74fce72 multiprocess: Add type conversion code for serializable types (Ryan Ofsky)
4aaee23921 test: add ipc test to test multiprocess type conversion code (Ryan Ofsky)

Pull request description:

  Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly.

  The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  achow101:
    ACK 6acec6b9ff
  dergoegge:
    reACK 6acec6b9ff

Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
2024-01-23 16:22:29 -05:00
Ava Chow
874c8bdb9e
Merge bitcoin/bitcoin#29144: init: handle empty settings file gracefully
e9014042a6 settings: add auto-generated warning msg for editing the file manually (furszy)
966f5de99a init: improve corrupted/empty settings file error msg (furszy)

Pull request description:

  Small and simple issue reported [here](https://community.umbrel.com/t/bitcoin-docker-container-keeps-restarting/2144).

  Improving a confusing situation reported by users who did not understand why a
  settings parsing error occurred when the file was empty and did not know how to solve it.

  Empty setting file could be due (1) corruption or (2) an user manually cleaning up the file content.
  In both scenarios, the 'Unable to parse settings file' error does not help the user move forward.

ACKs for top commit:
  achow101:
    ACK e9014042a6
  hebasto:
    re-ACK e9014042a6.
  ryanofsky:
    Code review ACK e9014042a6. Just whitespace formatting changes and shortening a test string literal since last review
  shaavan:
    Code review ACK e9014042a6

Tree-SHA512: 2910654c6b9e9112de391eedb8e46980280f822fa3059724dd278db7436804dd27fae628d2003f2c6ac1599b07ac5c589af016be693486e949f558515e662bec
2024-01-23 15:14:03 -05:00
Ava Chow
6f732ffc3c
Merge bitcoin/bitcoin#28774: wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it
32a9f13cb8 wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it (Vasil Dimov)

Pull request description:

  `CWallet::GetEncryptionKey()` would return a reference to the internal
  `CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.

  Returning a copy would be a shorter solution, but could have security
  implications of the master key remaining somewhere in the memory even
  after `CWallet::Lock()` (the current calls to
  `CWallet::GetEncryptionKey()` are safe, but that is not future proof).

  So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
  change the `GetEncryptionKey()` method to provide the encryption
  key to a given callback:
  `m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`

  This silences the following (clang 18):

  ```
  wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
   3520 |     return vMasterKey;
        |            ^
  ```

  ---
  _Previously this PR modified both ArgsManager and wallet code. But the ArgsManager commit 856c88776f was merged in https://github.com/bitcoin/bitcoin/pull/29040 so now this only affects wallet code. The previous PR description was:_

  Avoid this unsafe pattern from `ArgsManager` and `CWallet`:

  ```cpp
  class A
  {
      Mutex mutex;
      Foo member GUARDED_BY(mutex);
      const Foo& Get()
      {
          LOCK(mutex);
          return member;
      } // callers of `Get()` will have access to `member` without owning the mutex.
  ```

ACKs for top commit:
  achow101:
    ACK 32a9f13cb8
  ryanofsky:
    Code review ACK 32a9f13cb8. This seems like a potentially real race condition, and the fix here is pretty simple.
  furszy:
    ACK 32a9f13c

Tree-SHA512: 133da84691642afc1a73cf14ad004a7266cb4be1a6a3ec634d131dca5dbcdef52522c1d5eb04f5b6c4e06e1fc3e6ac57315f8fe1e207b464ca025c2b4edefdc1
2024-01-23 15:05:23 -05:00
Ava Chow
7cb7759b25
Merge bitcoin/bitcoin#29272: wallet: fix coin selection tracing to return -1 when no change pos
d55fdb1a49 Move TRACEx parameters to seperate lines (Richard Myers)
2d58629ee6 wallet: fix coin selection tracing to return -1 when no change pos (Richard Myers)

Pull request description:

  This is a bugfix for from when [optional was introduced](758501b713)  for `change_pos` in the wallet. When optional `change_pos` is unset, we should return -1 and not 0.

  I added two new checks to the `test/functional/interface_usdt_coinselection.py` which adds coverage for the situations when `normal_create_tx_internal` and `aps_create_tx_internal` events occur with no change.

  You can reproduce this bug using the coin-selection-simulation scripts as described in [issue #16](https://github.com/achow101/coin-selection-simulation/issues/16). You can also run the `interface_usdt_coinselection.py` test  without the changes to `wallet/spend.cpp`.

ACKs for top commit:
  0xB10C:
    ACK d55fdb1a49
  achow101:
    ACK d55fdb1a49
  murchandamus:
    ACK d55fdb1a49

Tree-SHA512: 6efac3b756bdf51debbcb759dc3c4b7a4304626bc047b70025cec02f3a04937ace7712e9558ac71e560fd136005a98c518ac5bb4b90c3282d776beccd0de9749
2024-01-23 14:33:43 -05:00
fanquake
f1ab078ed7
Merge bitcoin/bitcoin#29276: depends: Update libmultiprocess library to fix C++20 macos build error
b8105b3ed7 depends: Update libmultiprocess library to fix C++20 macos build error (Ryan Ofsky)

Pull request description:

  Fixes #29248

  The std::result_of type was removed in c++20, but was being referenced in some old, unused code in the library. The issue was fixed in:

  - https://github.com/chaincodelabs/libmultiprocess/pull/91

  This update also includes other recent libmultiprocess changes to improve C++20 support and fix build issues:

  - https://github.com/chaincodelabs/libmultiprocess/pull/89
  - https://github.com/chaincodelabs/libmultiprocess/pull/90
  - https://github.com/chaincodelabs/libmultiprocess/pull/93

ACKs for top commit:
  fanquake:
    ACK b8105b3ed7.

Tree-SHA512: 2ca64b5fc27be752baba38df4b4faf62152e18c70ead6e0e063f1cb0c25dd5d924dec7ebfd7f8bbd651ae50eb35e8d8b591a9847c36f22558b5f5effccf56536
2024-01-23 17:06:57 +00:00
fanquake
8c9dceb962
Merge bitcoin/bitcoin#29291: Add test for negative transaction version w/ CSV to tx_valid.json
97181decf5 Add test for negative transaction version w/ CSV to tx_valid.json (Chris Stewart)

Pull request description:

  This PR adds a static test vector corresponding to the bug found in various implementations of the bitcoin protocol discovered by dergoegge

  For more information see:

  https://delvingbitcoin.org/t/disclosure-btcd-consensus-bugs-due-to-usage-of-signed-transaction-version/455

ACKs for top commit:
  darosior:
    ACK 97181decf5
  dergoegge:
    ACK 97181decf5

Tree-SHA512: 92bbcd3cd10a569757b4de91e1b2bcfebc2b75ddb0160be36d8e512a6fa4623cced1aba93bd1cc044962cd2b10e1d184ef109ccdfe3cfcf85cf4b9585d80d115
2024-01-23 16:53:37 +00:00
Ryan Ofsky
b8105b3ed7 depends: Update libmultiprocess library to fix C++20 macos build error
Fixes #29248

The std::result_of type was removed in c++20, but was being referenced in some
old, unused code in the library. The issue was fixed in:

https://github.com/chaincodelabs/libmultiprocess/pull/91 util: Drop Bind, BindTuple, ComposeFn, GetFn, and ThrowFn helpers

This update also includes other recent libmultiprocess changes to improve C++20
support and fix build issues:

https://github.com/chaincodelabs/libmultiprocess/pull/89 pkgconfig: Drop -std=c++17 compile flag
https://github.com/chaincodelabs/libmultiprocess/pull/90 pkgconfig: Use @CMAKE_INSTALL_LIBDIR@ variable
https://github.com/chaincodelabs/libmultiprocess/pull/93 Fix support for vector<bool> serialization with libc++
2024-01-22 11:47:13 -05:00
furszy
e9014042a6
settings: add auto-generated warning msg for editing the file manually
Hopefully, refraining users from modifying the file unless they are
certain about the potential consequences.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
furszy
966f5de99a
init: improve corrupted/empty settings file error msg
The preceding "Unable to parse settings file" message lacked
the necessary detail and guidance for users on what steps to
take next in order to resolve the startup error.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
glozow
651fb034d8
Merge bitcoin/bitcoin#29260: refactor: remove CTxMemPool::queryHashes()
282b12ddb0 refactor: remove CTxMemPool::queryHashes() (stickies-v)

Pull request description:

  `CTxMemPool::queryHashes()` is only used in `MempoolToJSON()`, where it can just as easily be replaced with the more general `CTxMemPool::entryAll()`. No behaviour change, just cleans up the code.

ACKs for top commit:
  dergoegge:
    Code review ACK 282b12ddb0
  TheCharlatan:
    ACK 282b12ddb0
  glozow:
    ACK 282b12ddb0. Looks like there's no conflicts.

Tree-SHA512: 16160dec8e1f2457fa0f62dc96d2d2efd92c4bab810ecdb0e08918b8e85a667702c8e41421eeb4ea6abe92a5956a2a39a7a6368514973b78be0d22de2ad299b2
2024-01-22 10:03:57 +00:00
Richard Myers
d55fdb1a49
Move TRACEx parameters to seperate lines 2024-01-20 14:58:17 +01:00
Richard Myers
2d58629ee6
wallet: fix coin selection tracing to return -1 when no change pos 2024-01-20 14:56:41 +01:00
josibake
18ad1b9142
refactor: pass CRecipient to FundTransaction
Instead turning tx.vout into a vector of `CRecipient`, make `FundTransaction`
take a `CRecipient` vector directly. This allows us to remove SFFO logic from
the wrapper RPC `FundTransaction` since the `CRecipient` objects have already
been created with the correct SFFO values. This also allows us to remove
SFFO from both `FundTransaction` function signatures.

This sets us up in a future PR to be able to use these RPCs with BIP352
static payment codes.
2024-01-19 15:04:56 +01:00
josibake
5ad19668db
refactor: simplify CreateRecipients
Move validation logic out of `CreateRecipients` and instead take the
already validated outputs from `ParseOutputs` as an input.

Move SFFO parsing out of `CreateRecipients` into a new function,
`InterpretSubtractFeeFromOutputsInstructions`. This takes the SFFO instructions
from `sendmany` and `sendtoaddress` and turns them into a set of integers.
In a later commit, we will also move the SFFO parsing logic from
`FundTransaction` into this function.

Worth noting: a user can pass duplicate addresses and addresses that dont exist
in the transaction outputs as SFFO args to `sendmany` and `sendtoaddress`
without triggering a warning. This behavior is preserved in to keep this commit
strictly a refactor.
2024-01-19 15:04:56 +01:00
josibake
47353a608d
refactor: remove out param from ParseRecipients
Have `ParseRecipients` return a vector of `CRecipients` and rename to `CreateRecipients`.
2024-01-19 15:04:56 +01:00
josibake
f7384b921c
refactor: move parsing to new function
Move the parsing and validation out of `AddOutputs` into its own function,
`ParseOutputs`. This allows us to re-use this logic in `ParseRecipients` in a
later commit, where the code is currently duplicated.

The new `ParseOutputs` function returns a CTxDestination,CAmount tuples.
This allows the caller to then translate the validated outputs into
either CRecipients or CTxOuts.
2024-01-19 15:04:56 +01:00
josibake
6f569ac903
refactor: move normalization to new function
Move the univalue formatting logic out of AddOutputs and into its own function,
`NormalizeOutputs`. This allows us to re-use this logic in later commits.
2024-01-19 15:04:56 +01:00
josibake
435fe5cd96
test: add tests for fundrawtx and sendmany rpcs
If the serialized transaction passed to `fundrawtransaction` contains
duplicates, they will be deserialized and added to the transaction. Add
a test to ensure this behavior is not changed during the refactor.

A user can pass any number of duplicated and unrelated addresses as an
SFFO argument to `sendmany` and the RPC will not throw an error (note,
all the rest of the RPCs which take SFFO as an argument will error if
the user passes duplicates or specifies outputs not present in the
transaction). Add a test to ensure this behavior is not changed during
the refactor.
2024-01-19 15:04:56 +01:00
fanquake
03752444cd
Merge bitcoin/bitcoin#29249: depends: add NM output to gen_id
6ec2813cd8 depends: add NM output to gen_id (fanquake)

Pull request description:

  `NM` is part of the current toolset, and can be set by the user. Include it in `gen_id`.

ACKs for top commit:
  TheCharlatan:
    Re-ACK 6ec2813cd8

Tree-SHA512: 2ada61e03783f9eb441f285ef5da50557ad729cb52ce2d2c4b2c38103dab29920a26262d4545fd2ac7fbf1cedc4902cd2359833544fbc0debf829c12a63e9769
2024-01-19 13:17:16 +00:00
Hennadii Stepanov
cbea49c0d3
build: Pass sanitize flags to instrument libsecp256k1 code
Also a new UBSan suppression has been added.
2024-01-19 10:08:41 +00:00
stickies-v
282b12ddb0
refactor: remove CTxMemPool::queryHashes()
Its only usage can easily be replaced with CTxMemPool::entryAll()
2024-01-18 21:54:56 +00:00
Ava Chow
5f3a0574c4
Merge bitcoin/bitcoin#29262: rpc: Fix race in loadtxoutset
5555d8db33 test: Use blocks_path where possible (MarcoFalke)
fa9108941f rpc: Fix race in loadtxoutset (MarcoFalke)

Pull request description:

  The tip may have advanced, also if it did not, there is no reason to
  have two variables point to the same block.

  Fixes https://github.com/bitcoin/bitcoin/pull/27596#discussion_r1344694600

ACKs for top commit:
  achow101:
    ACK 5555d8db33
  pablomartin4btc:
    ACK 5555d8db33
  BrandonOdiwuor:
    Code Review ACK 5555d8db33

Tree-SHA512: 23a82924a915b61bb1adab8ad20ec8914139c8ee647817af34ca27ee310a2e45833d8b285503e0feebe63e4667193d6d98cfcbbc1509bf40712225e04dd19e8b
2024-01-18 13:17:35 -05:00
Ava Chow
ac3901ebd0
Merge bitcoin/bitcoin#29228: test: Remove all-lint.py script
fa2b95cf3f test: Remove all-lint.py script (MarcoFalke)
fadb06c361 doc: move-only lint docs to one place (MarcoFalke)

Pull request description:

  Seems confusing to have a test runner that calls another runner (`all-lint.py`), which calls a subset of the lint tests.

  Fix that by just calling this subset of lint tests in the test runner directly, and remove the now unused `all-lint.py`.

  To run all lint checks locally, refer to the documentation: https://github.com/bitcoin/bitcoin/blob/master/test/lint/README.md#running-locally

ACKs for top commit:
  kevkevinpal:
    ACK [fa2b95c](fa2b95cf3f)
  achow101:
    ACK fa2b95cf3f
  TheCharlatan:
    ACK fa2b95cf3f
  pablomartin4btc:
    tACK fa2b95cf3f
  brunoerg:
    utACK fa2b95cf3f

Tree-SHA512: 43fac9acb4e9a6744d695dd49c7202e19ab4bf480f4cccff768647d0157a065f40e6ad70b9f6a65ba42048cc5fa9834365aa8e7aa0ed64c09e0cd4eb8dccb831
2024-01-18 13:02:15 -05:00
Vasil Dimov
32a9f13cb8
wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it
`CWallet::GetEncryptionKey()` would return a reference to the internal
`CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.

Returning a copy would be a shorter solution, but could have security
implications of the master key remaining somewhere in the memory even
after `CWallet::Lock()` (the current calls to
`CWallet::GetEncryptionKey()` are safe, but that is not future proof).

So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
change the `GetEncryptionKey()` method to provide the encryption
key to a given callback:
`m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`

This silences the following (clang 18):

```
wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
 3520 |     return vMasterKey;
      |            ^
```
2024-01-18 18:12:59 +01:00
fanquake
03c5b0064d
Merge bitcoin/bitcoin#29085: refactor: C++20: Use std::rotl
6044628543 crypto, hash: replace custom rotl32 with std::rotl (Fabian Jahr)

Pull request description:

  While exploring some C++20 changes and checking against our code I found this potential improvement:

  1. We can replace our custom implementation of `rotl32` in crypto/chacha20 with `std::rotl` from the [new `bit` header](https://en.cppreference.com/w/cpp/header/bit).

ACKs for top commit:
  fanquake:
    ACK 6044628543

Tree-SHA512: db55b366f20fca2ef62e5f10a838f8a709d531678c35c1dba20898754029c788a2fd47995208ed6d187cf814109a7ca397bc2c301504500aee79da04c95d6895
2024-01-18 09:40:44 +00:00
fanquake
3d52cedb49
Merge bitcoin/bitcoin#29251: contrib: Update clang-format-diff
52149b7a2c contrib: Fix clang-format-diff.py lint errors (TheCharlatan)
008e81e025 contrib: Latest clang-format-diff.py script (TheCharlatan)

Pull request description:

  This was taken from 900bb318b5/clang/tools/clang-format/clang-format-diff.py and is useful for systems where clang tools are shipped with a version suffix.

ACKs for top commit:
  maflcko:
    lgtm ACK 52149b7a2c  🌱

Tree-SHA512: cad720b283650e36c5b3ec597278112899ae6413a42c79b6296c58242000f32ae6ab7ed732a3a0f30f1f2586d7e3c0d1ef856a4821e28272bae6f428ed9497e9
2024-01-17 16:08:10 +00:00
fanquake
514268170b
Merge bitcoin/bitcoin#29133: refactor: Allow std::span construction from CKey
fa96d93711 refactor: Allow std::span construction from CKey (MarcoFalke)
999962d68d Add missing XOnlyPubKey::data() to get mutable data (MarcoFalke)

Pull request description:

  Is is possible to construct a `Span` from a reference to a `CKey`. However, the same is not possible with `std::span`.

  Fix that.

ACKs for top commit:
  shaavan:
    ReACK fa96d93711
  willcl-ark:
    ACK fa96d93711

Tree-SHA512: 44fccdce5f32bc16b44f3b1bd32e86d9eabfd09bca6abe79f2d6db0cb0b5e4aaeaff710f023cb21ccde9315d2007d55f1b43f29416e81bceeeabe3948f673d3a
2024-01-17 16:00:32 +00:00
MarcoFalke
5555d8db33
test: Use blocks_path where possible 2024-01-17 16:48:47 +01:00
MarcoFalke
fa9108941f
rpc: Fix race in loadtxoutset
The tip may have advanced, also if it did not, there is no reason to
have two variables point to the same block.
2024-01-17 16:48:42 +01:00
fanquake
c818607ed5
Merge bitcoin/bitcoin#29233: build: depends move macOS C(XX) FLAGS out of C & CXX
cbc9bf11fe build: move -mlinker-version to *FLAGS (fanquake)
42b2283765 depends: deduplicate use of mmacosx-version-min in macOS build (fanquake)

Pull request description:

  Move some C/CXX FLAGS out of C/CXX. The remaining flags are host/SDK related, and will need some more thought.
  This is more correct in any case, and simplifies future changes.
  Related to #21778.

ACKs for top commit:
  theuni:
    utACK cbc9bf11fe
  TheCharlatan:
    ACK cbc9bf11fe

Tree-SHA512: 373216c1de32375faddc161ecc09b14fed0e0994cbd5ed62c862c38a5aea80c7f1740f32f36b8a60ac1caf749309750d22164c50b89916f41838d6453296ac4a
2024-01-17 10:46:16 +00:00
TheCharlatan
52149b7a2c
contrib: Fix clang-format-diff.py lint errors
We assume to be using python3, so don't check for it. This removes a
type error on the line `from io import BytesIO as StringIO`.

Specify the encoding as "utf8" when opening a file.
2024-01-17 11:10:15 +01:00
Ava Chow
8106b268cd
Merge bitcoin/bitcoin#29239: rpc: Make v2transport default for addnode RPC when enabled
3ba815b42d Make v2transport default for addnode RPC when enabled (Pieter Wuille)

Pull request description:

  Since #29058, several types of manually configured connections will attempt v2 connections when `-v2transport` is enabled, except for the `addnode` RPC, as that one has an explicit argument to enable or disable.

  Make the default for that RPC match the `-v2transport` setting so the behavior matches that of other manual connections from a user perspective.

ACKs for top commit:
  achow101:
    ACK 3ba815b42d
  kristapsk:
    ACK 3ba815b42d
  theStack:
    Code-review ACK 3ba815b42d

Tree-SHA512: 31ef48cf1e533abb17866020378c004df929e626074dc98b3229fb60a66de58435e95c8fda8d1b463e1208aa39d1f42d239818e7e58595a3738089920598befc
2024-01-16 16:50:03 -05:00
Ava Chow
a3fb1f80ac
Merge bitcoin/bitcoin#28791: snapshots: don't core dump when running -checkblockindex after loadtxoutset
cdc6ac4126 snapshots: don't core dump when running -checkblockindex after `loadtxoutset` (Mark Friedenbach)

Pull request description:

  Transaction counts aren't known for block history loaded from a snapshot. If you start with `-checkblockindex` after loading a snapshot, the bitcoin daemon will core dump. The test suite does not check for this because all the snapshots have no non-coinbase transactions (all blocks prior to the snapshot are assumed to have `nTx = 1`).

  Recommend for backport to 26.x

ACKs for top commit:
  fjahr:
    utACK cdc6ac4126
  achow101:
    ACK cdc6ac4126
  pablomartin4btc:
    tACK cdc6ac4126

Tree-SHA512: f7488a85cc29056e2ac443ce8f34aea4dfde6ba246efce82235d6a4dca2dca4344f07b93c93424b4addcb83e4cb2ae49a3ebb37d89840d42d2aeea35904cab04
2024-01-16 15:02:53 -05:00
Ava Chow
5711da6588
Merge bitcoin/bitcoin#29213: doc, test: test and explain service flag handling
74ebd4d135 doc, test: Test and explain service flag handling (Martin Zumsande)

Pull request description:

  Service flags received from the peer-to-peer network are handled differently, depending on how we receive them.
  If received directly from an outbound peer the flags belong to, they replace existing flags.
  If received via gossip relay (so that anyone could send them), new flags are added, but existing ones but cannot be overwritten.

  Document that and add test coverage for it.

ACKs for top commit:
  achow101:
    ACK 74ebd4d135
  furszy:
    ACK 74ebd4d135
  brunoerg:
    utACK 74ebd4d135

Tree-SHA512: 604adc3304b8e3cb1a10dfd017025c10b029bebd3ef533f96bcb5856fee5d4396a9aed4949908b8e7ef267ad21320d1814dd80f88426330c5c9c2c529c497591
2024-01-16 13:35:45 -05:00
Ava Chow
27d935f58b
Merge bitcoin/bitcoin#29179: test: wallet rescan with reorged parent + IsFromMe child in mempool
df30247705 [test] import descriptor wallet with reorged parent + IsFromMe child in mempool (glozow)
c3d02be536 [test] rescan legacy wallet with reorged parent + IsFromMe child in mempool (Gloria Zhao)

Pull request description:

  Originally motivated by #29019, which reverts back to having `requestMempoolTransactions` emit `transactionAddedToMempool` in `mapTx` default order instead of `GetSortedDepthAndScore` order.

  It's important that these notifications happen in topological order, otherwise the wallet rescan may miss transactions that belong to it. Notably, checking whether a transaction `IsFromMe` requires knowing its inputs, which may be from a mempool parent.

  When using `mapTx` order, a parent may come later than its child if it was added from a block disconnected in a reorg.

  This PR adds a test for this case.

ACKs for top commit:
  achow101:
    ACK df30247705
  furszy:
    Code review ACK df30247705, nits can be disregarded.

Tree-SHA512: 2f1d9ef92313228adbbef94e634e5f7a9ec6e6a2c88e16aa343bdc95ffc9b9f9c82a569b412c9a3841db9d789e52f9283e8b9385731668d59355903e26e58a5d
2024-01-16 12:49:09 -05:00