Commit Graph

36706 Commits

Author SHA1 Message Date
Andrew Chow
53b07b2b47 verify-commits: Move trusted-keys valid sig check into verify-commits itself
Instead of having gpg.sh check against the trusted keys for a valid
signature, do it inside of verify-commits itself.

This also allows us to use the same trusted-keys throughout the
verify-commits.py check rather than it possibly being modified during
the clean merge check.
2023-02-16 12:46:40 -05:00
fanquake
75f0e0b607
Merge bitcoin/bitcoin#26773: doc: FreeBSD build doc updates to reflect removal of install_db4.sh
c572eae989 update the freebsd build doc to reflect recent changes to DB4 install process (Murray Nesbitt)

Pull request description:

  This PR introduces documentation changes needed to keep up with #26834.

ACKs for top commit:
  fanquake:
    ACK c572eae989 - have not tested, but looks ok.

Tree-SHA512: 42a79e7b45834916b1b738db524b51b9ff4fde8348ba66fc331ff6603532dd9fce73ea392eef97d31112326c6d60ec2c5c7c29e66aab33aaf846aab8aea1d1aa
2023-02-16 14:50:00 +00:00
merge-script
437dfe1c26
Merge bitcoin/bitcoin#26714: test: add coverage for unparsable -maxuploadtarget
7a83aa0982 test: add coverage for unparsable `-maxuploadtarget` (brunoerg)

Pull request description:

  This PR adds test coverage for the following error:
  7386da7a0b/src/init.cpp (L1096-L1099)

Top commit has no ACKs.

Tree-SHA512: c115b2b4d2d0eb2316bf9fafd7e0046aa18c9650062779b3a82d6145d188765bff5317f4ca5f79607732fde6d83e1f67756ac20a12c98d060ee68d8acc20c76e
2023-02-16 15:17:54 +01:00
merge-script
98042a0456
Merge bitcoin/bitcoin#27107: doc: remove mention of "proper signing key"
304ae6dc8e doc: remove mention of "proper signing key" (fanquake)

Pull request description:

  This key is no-longer in use: https://lists.linuxfoundation.org/pipermail/bitcoin-core-dev/2023-February/000115.html
  > Please remove it from verification pipelines.

ACKs for top commit:
  hebasto:
    ACK 304ae6dc8e

Tree-SHA512: 3dfd221a48f69ac56b4568db06b5d5b5d6a60b7d027a26157912219a2073589a0a3934cb30e11a161d48db55d3a637338f96617e3f3b92cb9e60e0d1d1dd372a
2023-02-16 15:10:41 +01:00
merge-script
3a68e194f5
Merge bitcoin/bitcoin#26586: test: previous releases: add v24.0.1
741908afc1 test: previous releases: add v24.0.1 (Sebastian Falbesoner)

Pull request description:

  The same procedure as every release (see dba1231672 [v23.0] and d8b705f1ca [v22.0]), only a little simpler now: thanks to #25650, the previous release fetch script defaults to downloading/building the necessary tags, i.e. we don't need to extend the tag list in the CI scripts and test/README.md anymore.

ACKs for top commit:
  Sjors:
    tACK 741908afc1

Tree-SHA512: a5426e989bd0bba42aa13e7d4cf60f792bf36bd9a6cdb6ef5799f7574d9a8a20979244627bbd0c6219630367e7fd73bac9e677814bc50233f64592ad035e713e
2023-02-16 15:02:02 +01:00
fanquake
3995c88e43
Merge bitcoin/bitcoin#25898: util: remove WSL 1 workaround in fs
5669afb80e fs: drop old WSL1 hack. (sinetek)

Pull request description:

  Following discussion, the WSL1 patch will be removed, as WSL1 is no longer being developed by Microsoft. Instead, please upgrade to a mainstream WSL2 version. More information can be found on [the official website](https://docs.microsoft.com/en-us/windows/wsl/).

ACKs for top commit:
  1440000bytes:
    ACK 5669afb80e
  fanquake:
    ACK 5669afb80e - seems ok as-is.

Tree-SHA512: 256c13985f6dd3453caf39c7ef1c951dbdfa8457a18cd05e4624db36d8ed8a4f809bb78a7b3c82c72997e9ed3823d5566a5c2d0812d2501aba2e54bc5e6eec79
2023-02-16 12:33:26 +00:00
fanquake
304ae6dc8e
doc: remove mention of "proper signing key"
This key is no-longer in use:
https://lists.linuxfoundation.org/pipermail/bitcoin-core-dev/2023-February/000115.html
2023-02-16 10:27:19 +00:00
fanquake
fb82d91a9c
Merge bitcoin/bitcoin#24149: Signing support for Miniscript Descriptors
6c7a17a8e0 psbt: support externally provided preimages for Miniscript satisfaction (Antoine Poinsot)
840a396029 qa: add a "smart" Miniscript fuzz target (Antoine Poinsot)
17e3547241 qa: add a fuzz target generating random nodes from a binary encoding (Antoine Poinsot)
611e12502a qa: functional test Miniscript signing with key and timelocks (Antoine Poinsot)
d57b7f2021 refactor: make descriptors in Miniscript functional test more readable (Antoine Poinsot)
0a8fc9e200 wallet: check solvability using descriptor in AvailableCoins (Antoine Poinsot)
560e62b1e2 script/sign: signing support for Miniscripts with hash preimage challenges (Antoine Poinsot)
a2f81b6a8f script/sign: signing support for Miniscript with timelocks (Antoine Poinsot)
61c6d1a844 script/sign: basic signing support for Miniscript descriptors (Antoine Poinsot)
4242c1c521 Align 'e' property of or_d and andor with website spec (Pieter Wuille)
f5deb41780 Various additional explanations of the satisfaction logic from Pieter (Pieter Wuille)
22c5b00345 miniscript: satisfaction support (Antoine Poinsot)

Pull request description:

  This makes the Miniscript descriptors solvable.

  Note this introduces signing support for much more complex scripts than the wallet was previously able to solve, and the whole tooling isn't provided for a complete Miniscript integration in the wallet. Particularly, the PSBT<->Miniscript integration isn't entirely covered in this PR.

ACKs for top commit:
  achow101:
    ACK 6c7a17a8e0
  sipa:
    utACK 6c7a17a8e0 (to the extent that it's not my own code).

Tree-SHA512: a71ec002aaf66bd429012caa338fc58384067bcd2f453a46e21d381ed1bacc8e57afb9db57c0fb4bf40de43b30808815e9ebc0ae1fbd9e61df0e7b91a17771cc
2023-02-16 10:01:33 +00:00
fanquake
cbf511b3f4
Merge bitcoin/bitcoin#27054: Remove laanwj from trusted-keys
aafa5e945c Remove laanwj from trusted-keys (laanwj)

Pull request description:

  allow-revsig-commits list generated using:

      git log --format="%H %ce" --merges 577bd51a4b8de066466a445192c1c653872657e2..master | grep laanwj | cut -c -40 >> allow-revsig-commits

ACKs for top commit:
  Sjors:
    tACK aafa5e945c 😢
  achow101:
    ACK aafa5e945c
  fanquake:
    ACK aafa5e945c

Tree-SHA512: 5e38ac8101f948030f9577480bfba14674351a7d697d7f6985966d98a0200fa110cee13fb331a1ff0c05874d92d9d03402c540f063155e7eea093accb5f4590e
2023-02-16 09:42:11 +00:00
fanquake
424fc603d8
Merge bitcoin/bitcoin#26997: psbt: s/transcation/transaction/
906631450d s/transcation/transaction/ (Greg Sanders)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK 906631450d - looks like other comments are being addressed elsewhere.

Tree-SHA512: c835a14db2e0cf5e0317c95c8c7441df1f7c6cb14be7809fd947e07ea9d23f1f171f111429aabd0509b7f17601bc742041316b18e1135e547a966961f2c65038
2023-02-16 09:39:50 +00:00
Andrew Chow
54742532ce
Merge bitcoin-core/gui#711: refactor: Disable unused special members functions in UnlockContext
9fa43b5af6 refactor: Disable unused special members functions in `UnlockContext` (Hennadii Stepanov)

Pull request description:

  Also `UnlockContext::valid` and `UnlockContext::relock` are `const` now.

ACKs for top commit:
  achow101:
    ACK 9fa43b5af6
  john-moffett:
    ACK 9fa43b5af6
  furszy:
    ACK 9fa43b5a

Tree-SHA512: 6d9fa8208676b9bd5d85b73cb2d3136e7f28ef59e68ee34915ec598458868e302a80b9ef1384c0bf7a4c42f936830c3add9662ca0bae73860a55a25cc374b699
2023-02-15 18:41:44 -05:00
fanquake
5ecd14a31c
Merge bitcoin/bitcoin#26844: Net: Pass MSG_MORE flag when sending non-final network messages (round 2)
691eaf8873 Pass MSG_MORE flag when sending non-final network messages (Matt Whitlock)

Pull request description:

  **N.B.:** This is my second attempt at introducing this optimization. #12519 (2018) was closed in deference to switching to doing gathering socket writes using `sendmsg(2)`, which I agree would have superior performance due to fewer syscalls, but that work was apparently abandoned in late 2018. Ever since, Bitcoin Core has continued writing tons of runt packets to the wire. Can we proceed with my halfway solution for now?

  ----

  Since Nagle's algorithm is disabled, each and every call to `send(2)` can potentially generate a separate TCP segment on the wire. This is especially inefficient when sending the tiny header preceding each message payload.

  Linux implements a `MSG_MORE` flag that tells the kernel not to push the passed data immediately to the connected peer but rather to collect it in the socket's internal transmit buffer where it can be combined with data from successive calls to `send(2)`. Where available, specify this flag when calling `send(2)` in `CConnman::SocketSendData(CNode &)` if the data buffer being sent is not the last one in `node.vSendMsg`.

ACKs for top commit:
  sipa:
    ACK 691eaf8873
  vasild:
    ACK 691eaf8873

Tree-SHA512: 9a7f46bc12edbf78d488f05d1c46760110a24c95af74b627d2604fcd198fa3f511c5956bac36d0034e88c632d432f7d394147e667a11b027af0a30f70a546d70
2023-02-15 16:10:46 +00:00
merge-script
a65d2259f1
Merge bitcoin/bitcoin#27035: test: simplify and speedup mempool_updatefromblock.py by using MiniWallet
dee8549be3 test: simplify and speedup mempool_updatefromblock.py by using MiniWallet (Sebastian Falbesoner)

Pull request description:

  This PR simplifies the functional test mempool_updatefromblock.py by using MiniWallet in order to avoid manual low-level tx creation (signing, outputs selection, fee calculation). Most of the tedious work is done by the method `MiniWallet.send_self_transfer_multi` (calling `create_self_transfer_multi` internally) which supports spending a given set of UTXOs and creating a certain number of outputs.

  As a nice side-effect, the test's performance increases significantly (~3.5x on my system):

  ```
  master
      1m56.80s real     1m50.10s user     0m06.36s system

  PR
      0m32.34s real     0m30.26s user     0m01.41s system
  ```

  The arguments `start_input_txid` and `end_address` have been removed from the `transaction_graph_test` method, as they are currently unused and I don't see them being needed for future tests.

ACKs for top commit:
  brunoerg:
    crACK dee8549be3
  MarcoFalke:
    lgtm ACK dee8549be3 🚏

Tree-SHA512: 9f6da634bdc8c272f9a2af1cddaa364ee371d4e95554463a066249eecebb668d8c6cb123ec8a5404c41b3291010c0c8806a8a01dd227733cec03e73aa93b0103
2023-02-15 16:26:00 +01:00
fanquake
1e0198b6c1
Merge bitcoin/bitcoin#26153: Reduce wasted pseudorandom bytes in ChaCha20 + various improvements
511aa4f1c7 Add unit test for ChaCha20's new caching (Pieter Wuille)
fb243d25f7 Improve test vectors for ChaCha20 (Pieter Wuille)
93aee8bbda Inline ChaCha20 32-byte specific constants (Pieter Wuille)
62ec713961 Only support 32-byte keys in ChaCha20{,Aligned} (Pieter Wuille)
f21994a02e Use ChaCha20Aligned in MuHash3072 code (Pieter Wuille)
5d16f75763 Use ChaCha20 caching in FastRandomContext (Pieter Wuille)
38eaece67b Add fuzz test for testing that ChaCha20 works as a stream (Pieter Wuille)
5f05b27841 Add xoroshiro128++ PRNG (Martin Leitner-Ankerl)
12ff72476a Make unrestricted ChaCha20 cipher not waste keystream bytes (Pieter Wuille)
6babf40213 Rename ChaCha20::Seek -> Seek64 to clarify multiple of 64 (Pieter Wuille)
e37bcaa0a6 Split ChaCha20 into aligned/unaligned variants (Pieter Wuille)

Pull request description:

  This is an alternative to #25354 (by my benchmarking, somewhat faster), subsumes #25712, and adds additional test vectors.

  It separates the multiple-of-64-bytes-only "core" logic (which becomes simpler) from a layer around which performs caching/slicing to support arbitrary byte amounts. Both have their uses (in particular, the MuHash3072 code can benefit from multiple-of-64-bytes assumptions), plus the separation results in more readable code. Also, since FastRandomContext effectively had its own (more naive) caching on top of ChaCha20, that can be dropped in favor of ChaCha20's new built-in caching.

  I thought about rebasing #25712 on top of this, but the changes before are fairly extensive, so redid it instead.

ACKs for top commit:
  ajtowns:
    ut reACK 511aa4f1c7
  dhruv:
    tACK crACK 511aa4f1c7

Tree-SHA512: 3aa80971322a93e780c75a8d35bd39da3a9ea570fbae4491eaf0c45242f5f670a24a592c50ad870d5fd09b9f88ec06e274e8aa3cefd9561d623c63f7198cf2c7
2023-02-15 14:58:47 +00:00
fanquake
2b0cd7679f
Merge bitcoin/bitcoin#27076: verify-commits: Bump trusted git root to after most recent laanwj merge
6ada37d44c verify-commits: Bump trusted git root to after most recent laanwj merge (Andrew Chow)

Pull request description:

  To prepare for the removal of laanwj's key from trusted key (#27054), the trusted git root needs to be newer than the most recent merge commit signed by his key.

  This can be tested by removing the laanwj's key from trusted keys (e.g. by merging with #27054) and running `verify-commits.py` with `--clean-merge 0`: `./contrib/verify-commits/verify-commits.py --clean-merge 0 HEAD~`. (`--clean-merge 0` disables the clean merge check which will checkout some commits, which results in the `trusted-keys` used in checking of subsequent commits to be different than expected).

ACKs for top commit:
  fanquake:
    ACK 6ada37d44c
  hebasto:
    ACK 6ada37d44c, I've verified the history of laanwj's merge commits.

Tree-SHA512: 55cafeddd54aa2b62d7b7cd41c542f4fd974b322a0405de546600d88658575714ebc893b087eb31f28c205559a7b213f88d9038de431271fca00be866610df74
2023-02-15 12:30:38 +00:00
Hennadii Stepanov
e43ff4eab2
Merge bitcoin-core/gui#603: Add settings.json prune-prev, proxy-prev, onion-prev settings
9d3127b11e Add settings.json prune-prev, proxy-prev, onion-prev settings (Ryan Ofsky)

Pull request description:

  With #602, if proxy and pruning settings are disabled in the GUI and the GUI is restarted, proxy and prune values are not stored anywhere. So if these settings are enabled in the future, default values will be shown, not previous values.

  This PR stores previous values so they will preserved across restarts. I'm not sure I like this behavior because showing default values seems simpler and safer to me. Previous values may just have been set temporarily and may have never actually worked, and it adds some code complexity to store them.

  This PR is one way of resolving #596. Other solutions are possible and could be implemented as alternatives.

ACKs for top commit:
  hebasto:
    ACK 9d3127b11e, tested on Ubuntu 22.04.
  vasild:
    ACK 9d3127b11e
  jarolrod:
    tACK 9d3127b11e

Tree-SHA512: 1778d1819443490c880cfd5c1711d9c5ac75ea3ee8440e2f0ced81d293247163a78ae8aba6027215110aec6533bd7dc6472aeead6796bfbd51bf2354e28f24a9
2023-02-15 12:21:31 +00:00
Murray Nesbitt
c572eae989 update the freebsd build doc to reflect recent changes to DB4 install process 2023-02-15 01:12:00 -08:00
merge-script
68e484afbb
Merge bitcoin/bitcoin#26584: cli: include local ("unroutable") peers in -netinfo table
77192c9598 cli: include local ("unreachable") peers in -netinfo table (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/26579

  The `-netinfo` dashboard did not list peers that were connected via "unroutable" networks. This included local peers including local-network peers. Personally, I run one bitcoind instance on my network that is used by other services like Wasabi Wallet and LND running on other machines.

  This PR adds an "npr" (not publicly routable) column to the table of networks (ipv4, ipv6, onion, etc) so that every connection to the node is listed, and the totals are accurate as they relate to max inbound and max outbound limits.

  Example connecting in regtest mode to one local and one remote peer:

  ```
  Bitcoin Core client v24.99.0-151ce099ea8f-dirty regtest - server 70016/Satoshi:24.99.0/

  <->   type   net  mping   ping send recv  txn  blk  hb addrp addrl  age id address         version
   in          npr      0      0   90   90                              1  1 127.0.0.1:59180 70016/Satoshi:24.99.0/
  out manual  ipv4     63     63   84   84         3                    3  0 143.244.175.41  70016/Satoshi:24.0.1/
                       ms     ms  sec  sec  min  min                  min

           ipv4    ipv6     npr   total   block  manual
  in          0       0       1       1
  out         1       0       0       1       0       1
  total       1       0       1       2

  Local addresses: n/a

  ```

ACKs for top commit:
  jonatack:
    Re-tested ACK 77192c9598

Tree-SHA512: 78aa68bcff0dbaadb5f0604bf023fe8fd921313bd8276d12581f7655c089466a48765f9e123cb31d7f1d294d5ca45fdefdf8aa220466ff738f32414f41099c06
2023-02-15 09:18:57 +01:00
Andrew Chow
576e16e702
Merge bitcoin/bitcoin#26184: test: p2p: check that headers message with invalid proof-of-work disconnects peer
772671245d test: p2p: check that headers message with invalid proof-of-work disconnects peer (Sebastian Falbesoner)

Pull request description:

  One of the earliest anti-DoS checks done after receiving and deserializing a `headers` message from a peer is verifying whether the proof-of-work is valid (called in method `PeerManagerImpl::ProcessHeadersMessage`):
  f227e153e8/src/net_processing.cpp (L2752-L2762)
  The called method `PeerManagerImpl::CheckHeadersPoW` calls `Misbehaving` with a score of 100, i.e. leading to an immediate disconnect of the peer:
  f227e153e8/src/net_processing.cpp (L2368-L2372)

  This PR adds a simple test for both the misbehaving log and the resulting disconnect. For creating a block header with invalid proof-of-work, we first create one that is accepted by the node (the difficulty field `nBits` is copied from the genesis block) and based on that the nonce is modified until we have block header hash prefix that is too high to fulfill even the minimum difficulty.

ACKs for top commit:
  Sjors:
    ACK 772671245d
  achow101:
    ACK 772671245d
  brunoerg:
    crACK 772671245d
  furszy:
    Code review ACK 77267124 with a non-blocking speedup.

Tree-SHA512: 680aa7939158d1dc672b90aa6554ba2b3a92584b6d3bcb0227776035858429feb8bc66eed18b47de0fe56df7d9b3ddaee231aaeaa360136603b9ad4b19e6ac11
2023-02-14 18:45:35 -05:00
Hennadii Stepanov
9fa43b5af6
refactor: Disable unused special members functions in UnlockContext 2023-02-14 17:55:57 +00:00
fanquake
fb2f093479
Merge bitcoin/bitcoin#27097: descriptors: fix docstring (param [in] vs [out])
588fad868d descriptors: fix docstring (param [in] vs [out]) (SomberNight)

Pull request description:

  As in title, these docstrings look incorrect.

ACKs for top commit:
  john-moffett:
    ACK 588fad868d

Tree-SHA512: 1ab343a1b1fc57a7d6bd8363b84db9d96e8ea11a4cec85bcf79885c9df53da889fe2fb10b1fa92d824ddf0dee800c07353f46f1fea9887d2ad518bed0afebe3d
2023-02-14 17:02:29 +00:00
fanquake
af49d86dd7
Merge bitcoin/bitcoin#27093: test: Fix intermittent sync issue in wallet_pruning
fa9ec7b0fe test: Fix intermittent sync issue in wallet_pruning (MarcoFalke)

Pull request description:

  The `sync_fun=self.no_op` has no motivation or rationale, and seems to be causing issues.

  Fix that by removing it.

  Actually fixes https://github.com/bitcoin/bitcoin/issues/27065, see https://github.com/bitcoin/bitcoin/pull/27066#issuecomment-1428249997

ACKs for top commit:
  fanquake:
    ACK fa9ec7b0fe

Tree-SHA512: 3c67da6705d6698fcabb29de169a2b4723f74705c979380d1fddce5fe9595b4595445fd7d9790a6b2a89f10ce8ec3c64ce45248f58fd920b72b7b6fba8afb09f
2023-02-14 16:52:18 +00:00
SomberNight
588fad868d
descriptors: fix docstring (param [in] vs [out])
As in title, these docstrings look incorrect.
2023-02-14 14:28:08 +00:00
fanquake
d6ef44cccb
Merge bitcoin/bitcoin#27081: Modernize rpcauth.py
e4e17907b6 Modernize rpcauth.py and its tests (Pieter Wuille)

Pull request description:

  Use Python3 constructions, and f-strings.

ACKs for top commit:
  jamesob:
    Github ACK e4e17907b6

Tree-SHA512: 005573d967e04400fec727f45739f138879be703e692745c0a639272d37d221d230f388de23f2615cb954bb47179fb46e53da0410ae9f0865319b91bb2dc01f4
2023-02-14 11:26:17 +00:00
Pieter Wuille
e4e17907b6 Modernize rpcauth.py and its tests 2023-02-13 17:11:15 -05:00
Andrew Chow
2c1fe27bf3
Merge bitcoin/bitcoin#27080: Wallet: Zero out wallet master key upon locking so it doesn't persist in memory
3a11adc700 Zero out wallet master key upon lock (John Moffett)

Pull request description:

  When an encrypted wallet is locked (for instance via the RPC `walletlock`), the documentation indicates that the key is removed from memory:

  b92d609fb2/src/wallet/rpc/encrypt.cpp (L157-L158)

  However, the vector (a `std::vector<unsigned char, secure_allocator<unsigned char>>`) is merely _cleared_. As it is a member variable, it also stays in scope as long as the wallet is loaded, preventing the secure allocator from deallocating. This allows the key to persist indefinitely in memory. I confirmed this behavior on my macOS machine by using an open-source third party memory inspector ("Bit Slicer"). I was able to find my wallet's master key in Bit Slicer after unlocking and re-locking my encrypted wallet. I then confirmed the key data was at the address in LLDB.

  This PR manually fills the bytes with zeroes before calling `clear()` by using our `memory_cleanse` function, which is designed to prevent the compiler from optimizing it away. I confirmed that it does remove the data from memory on my machine upon locking.

  Note: An alternative approach could be to call `vMasterKey.shrink_to_fit()` after the `clear()`, which would trigger the secure allocator's deallocation. However, `shrink_to_fit()` is not _guaranteed_ to actually change the vector's capacity, so I think it's unwise to rely on it.

  ## Edit: A little more clarity on why this is an improvement.

  Since `mlock`ed memory is guaranteed not to be swapped to disk and our threat model doesn't consider a super-user monitoring the memory in realtime, why is this an improvement? Most importantly, consider hibernation. Even `mlock`ed memory may get written to disk. From the `mlock` [manpage](https://man7.org/linux/man-pages/man2/mlock.2.html):

  > (But be aware that the suspend mode on laptops and some desktop computers will save a copy of the system's RAM to disk, regardless of memory locks.)

  As far as I can tell, this is true of [Windows](https://web.archive.org/web/20190127110059/https://blogs.msdn.microsoft.com/oldnewthing/20140207-00/?p=1833#:~:text=%5BThere%20does%20not%20appear%20to%20be%20any%20guarantee%20that%20the%20memory%20won%27t%20be%20written%20to%20disk%20while%20locked.%20As%20you%20noted%2C%20the%20machine%20may%20be%20hibernated%2C%20or%20it%20may%20be%20running%20in%20a%20VM%20that%20gets%20snapshotted.%20%2DRaymond%5D) and macOS as well.

  Therefore, a user with a strong OS password and a strong wallet passphrase could still have their keys stolen if a thief takes their (hibernated) machine and reads the permanent storage.

ACKs for top commit:
  S3RK:
    Code review ACK 3a11adc700
  achow101:
    ACK 3a11adc700

Tree-SHA512: c4e3dab452ad051da74855a13aa711892c9b34c43cc43a45a3b1688ab044e75d715b42843c229219761913b4861abccbcc8d5cb6ac54957d74f6e357f04e8730
2023-02-13 15:18:16 -05:00
MarcoFalke
fa9ec7b0fe
test: Fix intermittent sync issue in wallet_pruning 2023-02-13 17:32:42 +01:00
fanquake
1ad0711d7c
Merge bitcoin/bitcoin#27016: mapport: require miniupnpc API version 17 or later
b3b673f704 mapport: require miniupnpc API version 17 or later (fanquake)

Pull request description:

  Version 17 is currently the latest version, see: https://github.com/miniupnp/miniupnp/blob/master/miniupnpc/apiversions.txt, and has been available since the release of 2.1. 2.1 or newer is readily available across all distros, see https://repology.org/project/miniupnpc/versions, so drop support for the older API versions.

  Split out of #22644.

ACKs for top commit:
  hebasto:
    ACK b3b673f704, tested on Ubuntu 20.04 w/ and w/o [`libminiupnpc-dev`](https://packages.ubuntu.com/focal/libminiupnpc-dev) package.
  TheCharlatan:
    ACK b3b673f704

Tree-SHA512: f53b36b82462c4ea83d9b83413dca8097885d1620f7ca0a53a79d6b3d3cf37c7773828b23f4278ccfcc3b14fcb0faffa35f60191b519b04570f3d2783d0303e2
2023-02-13 16:25:09 +00:00
Antoine Poinsot
6c7a17a8e0
psbt: support externally provided preimages for Miniscript satisfaction
Co-Authored-By: Andrew Chow <github@achow101.com>
2023-02-13 15:39:25 +01:00
merge-script
8126551d54
Merge bitcoin/bitcoin#27011: Add simulation-based CCoinsViewCache fuzzer
561848aaf2 Exercise non-DIRTY spent coins in caches in fuzz test (Pieter Wuille)
59e6828bb5 Add deterministic mode to CCoinsViewCache (Pieter Wuille)
b0ff310840 Add CCoinsViewCache::SanityCheck() and use it in fuzz test (Pieter Wuille)
3c9cea1340 Add simulation-based CCoinsViewCache fuzzer (Pieter Wuille)

Pull request description:

  The fuzzer goes through a sequence of operations that get applied to both a real stack of `CCoinsViewCache` objects, and to simulation data, comparing the two at the end.

ACKs for top commit:
  jamesob:
    re-ACK 561848aaf2
  dergoegge:
    Code review ACK 561848aaf2

Tree-SHA512: 68634f251fdb39436b128ecba093f651bff12ac11508dc9885253e57fd21efd44edf3b22b0f821c228175ec507df7d46c7f9f5404fc1eb8187fdbd136a5d5ee2
2023-02-13 15:31:50 +01:00
merge-script
141115a060
Merge bitcoin/bitcoin#27033: ci: Cache stuff in volumes, not host folders
fa8e92c022 doc: Update ci docs (721217.xyz)
5fffff54e9 ci: Cache stuff in volumes, not host folders (MarcoFalke)

Pull request description:

  Storing cached stuff in host system folders may lead to unexpected issues when the ci-built stuff is used for a non-ci build or a ci task leaks into another ci task.

ACKs for top commit:
  john-moffett:
    ACK fa8e92c022

Tree-SHA512: 8b0c9019452fbe507a272c1037c3dce3c178c21f85ab1096ed3372ad9d4b3c7aa27d89e5bf80c9a6260ea652e0268be0cbe61d6a4fcb3add569fa38076d32287
2023-02-13 11:53:50 +01:00
merge-script
a6316590d5
Merge bitcoin/bitcoin#26970: test: fix immediate tx relay in wallet_groups.py
ab4efad51b test: fix immediate tx relay in wallet_groups.py (Sebastian Falbesoner)

Pull request description:

  In the functional test wallet_groups.py we whitelist peers on all nodes (`-whitelist=noban@127.0.0.1`) to enable immediate tx relay for fast mempool synchronization. However, considering that this setting only applies to inbound peers and the default test topology looks like this:
  ```
      node0 <--- node1 <---- node2 <--- ... <-- nodeN
  ```

  txs propagate fast only from lower- to higher-numbered nodes (i.e. "left to right" in the above diagram) and take long from higher- to lower-numbered nodes ("right to left") since in the latter direction we only have outbound peers, where the trickle relay is still active. As a consequence, if a tx is submitted from any node other than node0, the mempool synchronization can take quite long.

  This PR fixes this by simply adding another connection from node0 to the last node, leading to a ~2-3x speedup (5 runs measured via `time ./test/functional/wallet_groups.py` are shown):

  ```
  master:
      0m53.31s real     0m08.22s user     0m05.60s system
      0m32.85s real     0m07.44s user     0m04.08s system
      0m46.40s real     0m09.18s user     0m04.23s system
      0m46.96s real     0m11.10s user     0m05.74s system
      0m57.23s real     0m10.53s user     0m05.59s system

  PR:
      0m19.64s real     0m09.58s user     0m05.50s system
      0m18.05s real     0m07.77s user     0m04.03s system
      0m18.99s real     0m07.90s user     0m04.25s system
      0m17.49s real     0m07.56s user     0m03.92s system
      0m18.11s real     0m07.74s user     0m03.88s system
  ```
  Note that in most tests this is not a problem since txs very often originate from node0.

ACKs for top commit:
  brunoerg:
    utACK ab4efad51b

Tree-SHA512: 12675357e6eb5a18383f2bfe719a184c0790863b37a98749d8e757dd5dc3a36212e16a81f0a192340c11b793eda00db359c7011f46f7c27e3a093af4f5b62147
2023-02-13 11:51:03 +01:00
Antoine Poinsot
840a396029
qa: add a "smart" Miniscript fuzz target
At the expense of more complexity, this target generates a valid
Miniscript node at every iteration.

This target will at first run populate a list of recipe (a map from
desired type to possible ways of creating such type) and curate it
(remove the unavailable or redundant recipes).
Then, at each iteration it will pick a type, choose a manner to create a
node of such type from the available recipes, and then
pseudo-recursively do the same for the type constraints of the picked
recipe.

For instance, if it is instructed based on the fuzzer output to create a
Miniscript node of type 'Bd', it could choose to create an 'or_i(subA, subB)'
nodes with type constraints 'B' for subA and 'Bd' for subB. It then
consults the recipes for creating subA and subB, etc...

Here is the list of all the existing recipes, by type constraint:

B: 0()
B: 1()
B: older()
B: after()
B: sha256()
B: hash256()
B: ripemd160()
B: hash160()
B: c:(K)
B: d:(Vz)
B: j:(Bn)
B: n:(B)
B: and_v(V,B)
B: and_b(B,W)
B: or_b(Bd,Wd)
B: or_d(Bdu,B)
B: or_i(B,B)
B: andor(Bdu,B,B)
B: thresh(Bdu)
B: thresh(Bdu,Wdu)
B: thresh(Bdu,Wdu,Wdu)
B: multi()

V: v:(B)
V: and_v(V,V)
V: or_c(Bdu,V)
V: or_i(V,V)
V: andor(Bdu,V,V)

K: pk_k()
K: pk_h()
K: and_v(V,K)
K: or_i(K,K)
K: andor(Bdu,K,K)

W: a:(B)
W: s:(Bo)

Bz: 0()
Bz: 1()
Bz: older()
Bz: after()
Bz: n:(Bz)
Bz: and_v(Vz,Bz)
Bz: or_d(Bzdu,Bz)
Bz: andor(Bzdu,Bz,Bz)
Bz: thresh(Bzdu)

Vz: v:(Bz)
Vz: and_v(Vz,Vz)
Vz: or_c(Bzdu,Vz)
Vz: andor(Bzdu,Vz,Vz)

Bo: sha256()
Bo: hash256()
Bo: ripemd160()
Bo: hash160()
Bo: c:(Ko)
Bo: d:(Vz)
Bo: j:(Bon)
Bo: n:(Bo)
Bo: and_v(Vz,Bo)
Bo: and_v(Vo,Bz)
Bo: or_d(Bodu,Bz)
Bo: or_i(Bz,Bz)
Bo: andor(Bzdu,Bo,Bo)
Bo: andor(Bodu,Bz,Bz)
Bo: thresh(Bodu)

Vo: v:(Bo)
Vo: and_v(Vz,Vo)
Vo: and_v(Vo,Vz)
Vo: or_c(Bodu,Vz)
Vo: or_i(Vz,Vz)
Vo: andor(Bzdu,Vo,Vo)
Vo: andor(Bodu,Vz,Vz)

Ko: pk_k()
Ko: and_v(Vz,Ko)
Ko: andor(Bzdu,Ko,Ko)

Bn: sha256()
Bn: hash256()
Bn: ripemd160()
Bn: hash160()
Bn: c:(Kn)
Bn: d:(Vz)
Bn: j:(Bn)
Bn: n:(Bn)
Bn: and_v(Vz,Bn)
Bn: and_v(Vn,B)
Bn: and_b(Bn,W)
Bn: multi()

Vn: v:(Bn)
Vn: and_v(Vz,Vn)
Vn: and_v(Vn,V)

Kn: pk_k()
Kn: pk_h()
Kn: and_v(Vz,Kn)
Kn: and_v(Vn,K)

Bon: sha256()
Bon: hash256()
Bon: ripemd160()
Bon: hash160()
Bon: c:(Kon)
Bon: d:(Vz)
Bon: j:(Bon)
Bon: n:(Bon)
Bon: and_v(Vz,Bon)
Bon: and_v(Von,Bz)

Von: v:(Bon)
Von: and_v(Vz,Von)
Von: and_v(Von,Vz)

Kon: pk_k()
Kon: and_v(Vz,Kon)

Bd: 0()
Bd: sha256()
Bd: hash256()
Bd: ripemd160()
Bd: hash160()
Bd: c:(Kd)
Bd: d:(Vz)
Bd: j:(Bn)
Bd: n:(Bd)
Bd: and_b(Bd,Wd)
Bd: or_b(Bd,Wd)
Bd: or_d(Bdu,Bd)
Bd: or_i(B,Bd)
Bd: or_i(Bd,B)
Bd: andor(Bdu,B,Bd)
Bd: thresh(Bdu)
Bd: thresh(Bdu,Wdu)
Bd: thresh(Bdu,Wdu,Wdu)
Bd: multi()

Kd: pk_k()
Kd: pk_h()
Kd: or_i(K,Kd)
Kd: or_i(Kd,K)
Kd: andor(Bdu,K,Kd)

Wd: a:(Bd)
Wd: s:(Bod)

Bzd: 0()
Bzd: n:(Bzd)
Bzd: or_d(Bzdu,Bzd)
Bzd: andor(Bzdu,Bz,Bzd)
Bzd: thresh(Bzdu)

Bod: sha256()
Bod: hash256()
Bod: ripemd160()
Bod: hash160()
Bod: c:(Kod)
Bod: d:(Vz)
Bod: j:(Bon)
Bod: n:(Bod)
Bod: or_d(Bodu,Bzd)
Bod: or_i(Bz,Bzd)
Bod: or_i(Bzd,Bz)
Bod: andor(Bzdu,Bo,Bod)
Bod: andor(Bodu,Bz,Bzd)
Bod: thresh(Bodu)

Kod: pk_k()
Kod: andor(Bzdu,Ko,Kod)

Bu: 0()
Bu: 1()
Bu: sha256()
Bu: hash256()
Bu: ripemd160()
Bu: hash160()
Bu: c:(K)
Bu: d:(Vz)
Bu: j:(Bnu)
Bu: n:(B)
Bu: and_v(V,Bu)
Bu: and_b(B,W)
Bu: or_b(Bd,Wd)
Bu: or_d(Bdu,Bu)
Bu: or_i(Bu,Bu)
Bu: andor(Bdu,Bu,Bu)
Bu: thresh(Bdu)
Bu: thresh(Bdu,Wdu)
Bu: thresh(Bdu,Wdu,Wdu)
Bu: multi()

Bzu: 0()
Bzu: 1()
Bzu: n:(Bz)
Bzu: and_v(Vz,Bzu)
Bzu: or_d(Bzdu,Bzu)
Bzu: andor(Bzdu,Bzu,Bzu)
Bzu: thresh(Bzdu)

Bou: sha256()
Bou: hash256()
Bou: ripemd160()
Bou: hash160()
Bou: c:(Ko)
Bou: d:(Vz)
Bou: j:(Bonu)
Bou: n:(Bo)
Bou: and_v(Vz,Bou)
Bou: and_v(Vo,Bzu)
Bou: or_d(Bodu,Bzu)
Bou: or_i(Bzu,Bzu)
Bou: andor(Bzdu,Bou,Bou)
Bou: andor(Bodu,Bzu,Bzu)
Bou: thresh(Bodu)

Bnu: sha256()
Bnu: hash256()
Bnu: ripemd160()
Bnu: hash160()
Bnu: c:(Kn)
Bnu: d:(Vz)
Bnu: j:(Bnu)
Bnu: n:(Bn)
Bnu: and_v(Vz,Bnu)
Bnu: and_v(Vn,Bu)
Bnu: and_b(Bn,W)
Bnu: multi()

Bonu: sha256()
Bonu: hash256()
Bonu: ripemd160()
Bonu: hash160()
Bonu: c:(Kon)
Bonu: d:(Vz)
Bonu: j:(Bonu)
Bonu: n:(Bon)
Bonu: and_v(Vz,Bonu)
Bonu: and_v(Von,Bzu)

Bdu: 0()
Bdu: sha256()
Bdu: hash256()
Bdu: ripemd160()
Bdu: hash160()
Bdu: c:(Kd)
Bdu: d:(Vz)
Bdu: j:(Bnu)
Bdu: n:(Bd)
Bdu: and_b(Bd,Wd)
Bdu: or_b(Bd,Wd)
Bdu: or_d(Bdu,Bdu)
Bdu: or_i(Bu,Bdu)
Bdu: or_i(Bdu,Bu)
Bdu: andor(Bdu,Bu,Bdu)
Bdu: thresh(Bdu)
Bdu: thresh(Bdu,Wdu)
Bdu: thresh(Bdu,Wdu,Wdu)
Bdu: multi()

Wdu: a:(Bdu)
Wdu: s:(Bodu)

Bzdu: 0()
Bzdu: n:(Bzd)
Bzdu: or_d(Bzdu,Bzdu)
Bzdu: andor(Bzdu,Bzu,Bzdu)
Bzdu: thresh(Bzdu)

Bodu: sha256()
Bodu: hash256()
Bodu: ripemd160()
Bodu: hash160()
Bodu: c:(Kod)
Bodu: d:(Vz)
Bodu: j:(Bonu)
Bodu: n:(Bod)
Bodu: or_d(Bodu,Bzdu)
Bodu: or_i(Bzu,Bzdu)
Bodu: or_i(Bzdu,Bzu)
Bodu: andor(Bzdu,Bou,Bodu)
Bodu: andor(Bodu,Bzu,Bzdu)
Bodu: thresh(Bodu)

Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2023-02-11 16:51:18 +01:00
Antoine Poinsot
17e3547241
qa: add a fuzz target generating random nodes from a binary encoding
This is a "dumb" way of randomly generating a Miniscript node from
fuzzer input. It defines a strict binary encoding and will always generate
a node defined from the encoding without "helping" to create valid nodes.
It will cut through as soon as it encounters an invalid fragment so
hopefully the fuzzer can tend to learn the encoding and generate valid
nodes with a higher probability.

On a valid generated node a number of invariants are checked, especially
around the satisfactions and testing them against the Script
interpreter.

The node generation and testing is modular in order to later introduce
other ways to generate nodes from fuzzer inputs with minimal code.

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-02-11 16:51:17 +01:00
Antoine Poinsot
611e12502a
qa: functional test Miniscript signing with key and timelocks
We'll need a better integration of the hash preimages PSBT fields to
satisfy Miniscript with such challenges from the RPC.

Thanks to Greg Sanders for his examples and suggestions to improve this
test.
2023-02-11 14:12:13 +01:00
Antoine Poinsot
d57b7f2021
refactor: make descriptors in Miniscript functional test more readable
We'll add more of them in the next commit, let's keep it bearable.
2023-02-11 14:12:13 +01:00
Antoine Poinsot
0a8fc9e200
wallet: check solvability using descriptor in AvailableCoins
This is a workaround for Miniscript descriptors containing hash
challenges. For those we can't mock the signature creator without making
OP_EQUAL mockable in the interpreter, so CalculateMaximumInputSize will
always return -1 and outputs for these descriptors would appear
unsolvable while they actually are.
2023-02-11 14:12:12 +01:00
Antoine Poinsot
560e62b1e2
script/sign: signing support for Miniscripts with hash preimage challenges
Preimages must be externally provided (typically, via a PSBT).
2023-02-11 14:12:12 +01:00
Antoine Poinsot
a2f81b6a8f
script/sign: signing support for Miniscript with timelocks 2023-02-11 14:12:11 +01:00
Antoine Poinsot
61c6d1a844
script/sign: basic signing support for Miniscript descriptors
Try to solve a script using the Miniscript satisfier if the legacy
solver fails under P2WSH context. Only solve public key and public key
hash challenges for now.

We don't entirely replace the raw solver and especially rule out trying to
solve CHECKMULTISIG-based multisigs with the Miniscript satisfier since
some features, such as the transaction input combiner, rely on the
specific behaviour of the former.
2023-02-11 14:12:10 +01:00
Pieter Wuille
4242c1c521
Align 'e' property of or_d and andor with website spec 2023-02-11 14:12:10 +01:00
Pieter Wuille
f5deb41780
Various additional explanations of the satisfaction logic from Pieter
Cherry-picked and squashed from
https://github.com/sipa/bitcoin/commits/202302_miniscript_improve.

- Explain thresh() and multi() satisfaction algorithms
- Comment on and_v dissatisfaction
- Mark overcomplete thresh() dissats as malleable and explain
- Add comment on unnecessity of Malleable() in and_b dissat
2023-02-11 14:12:09 +01:00
Antoine Poinsot
22c5b00345
miniscript: satisfaction support
This introduces the logic to "sign for" a Miniscript.

Co-Authored-By: Pieter Wuille <pieter.wuille@gmail.com>
2023-02-11 14:12:09 +01:00
John Moffett
3a11adc700 Zero out wallet master key upon lock
When an encrypted wallet is locked (for instance via the
RPC `walletlock`), the docs indicate that the key is
removed from memory. However, the vector (with a secure
allocator) is merely cleared. This allows the key to persist
indefinitely in memory. Instead, manually fill the bytes with
zeroes before clearing.
2023-02-10 20:21:23 -05:00
Andrew Chow
6ada37d44c verify-commits: Bump trusted git root to after most recent laanwj merge
To prepare for the removal of laanwj's key from trusted key, the trusted
git root needs to be newer than the most recent merge commit signed by
his key.
2023-02-10 11:36:06 -05:00
721217.xyz
fa8e92c022
doc: Update ci docs 2023-02-10 17:05:39 +01:00
merge-script
b92d609fb2
Merge bitcoin/bitcoin#27072: doc: Remove unused REVIEWERS file
fa8e3aa60d doc: Remove unused REVIEWERS file (721217.xyz)

Pull request description:

  Unused for way more than two months after https://github.com/bitcoin/bitcoin/pull/25613#issuecomment-1200113115

  See also bb5ebadeaa

ACKs for top commit:
  sipa:
    ACK fa8e3aa60d
  john-moffett:
    ACK fa8e3aa60d

Tree-SHA512: def650d34f04abd18e37cf19b5df34463157a99fbb537c4c98092c8312eb2a572ccde13ec5374f3b9e7cb90cef220cfe07f4522ffbfeb9407b20b3ada1d7706c
2023-02-10 16:57:53 +01:00
merge-script
e0d8378f2d
Merge bitcoin/bitcoin#27069: net: add Ensure{any}Banman
2d955ff006 net: add `Ensure{any}Banman` (brunoerg)

Pull request description:

  This PR adds `Ensure{any}Banman` functions to avoid code repetition and make it cleaner. Same approach as done with argsman, chainman, connman and others.

ACKs for top commit:
  davidgumberg:
    ACK [2d955ff](2d955ff006)

Tree-SHA512: 0beb7125312168a3df130c1793a1412ab423ef0f46023bfe2a121630c79df7e55d3d143fcf053bd09e2d96e9385a7a04594635da3e5c6be0c5d3a9cafbe3b631
2023-02-10 15:10:21 +01:00
721217.xyz
fa8e3aa60d
doc: Remove unused REVIEWERS file 2023-02-10 14:11:31 +01:00
merge-script
4f841cbb81
Merge bitcoin/bitcoin#27070: ci: Fix fingerprint_script for depends subdir caches
d66efa30cd ci: Fix `fingerprint_script` for `depends` subdir caches (Hennadii Stepanov)

Pull request description:

  https://github.com/bitcoin/bitcoin/pull/26977 made current `git rev-list -1 HEAD ./depends` [not working](https://github.com/bitcoin/bitcoin/pull/26977#issuecomment-1424614490).

  This PR fixes this issue with an idea from https://github.com/bitcoin/bitcoin/pull/26977#issuecomment-1424636503.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK d66efa30cd

Tree-SHA512: e6dbb1a80439bf8c044e50afc44b8b08b0903d20da5ef9dcbb265f2d64b6810ea5f4cb8abb44e00b96673082f1dbdd350627627d9d28f362bdb662c92ef257d8
2023-02-10 09:39:11 +01:00