Commit Graph

36892 Commits

Author SHA1 Message Date
Andrew Chow
86bacd75e7
Merge bitcoin/bitcoin#26742: http: Track active requests and wait for last to finish - 2nd attempt
60978c8080 test: Reduce extended timeout on abortnode test (Fabian Jahr)
660bdbf785 http: Release server before waiting for event base loop exit (João Barbosa)
8c6d007c80 http: Track active requests and wait for last to finish (João Barbosa)

Pull request description:

  This revives #19420. Since promag is not so active at the moment, I can support this to finally get it merged.

  The PR is rebased and comments by jonatack have been addressed.

  Once this is merged, I will also reopen #19434.

ACKs for top commit:
  achow101:
    ACK 60978c8080
  stickies-v:
    re-ACK [60978c8](60978c8080)
  hebasto:
    ACK 60978c8080

Tree-SHA512: eef0fe1081e9331b95cfafc71d82f2398abd1d3439dac5b2fa5c6d9c0a3f63ef19adde1c38c88d3b4e7fb41ce7c097943f1815c10e33d165918ccbdec512fe1c
2023-03-06 19:35:59 -05:00
Andrew Chow
4ea3a8b71d
Merge bitcoin/bitcoin#25806: wallet: group outputs only once, decouple it from Coin Selection
6a302d40df wallet: single output groups filtering and grouping process (furszy)
bd91ed1cb2 wallet: unify outputs grouping process (furszy)
55962001da test: coinselector_tests refactor, use CoinsResult instead of plain std::vector (furszy)
34f54a0a3a wallet: decouple outputs grouping process from each ChooseSelectionResult (furszy)
461f0821a2 refactor: make OutputGroup::m_outputs field a vector of shared_ptr (furszy)
d8e749bb84 test: wallet, add coverage for outputs grouping process (furszy)
06ec8f9928 wallet: make OutputGroup "positive_only" filter explicit (furszy)

Pull request description:

  The idea originates from https://github.com/bitcoin/bitcoin/pull/24845#issuecomment-1130310321.

  Note:
  For clarity, it's recommended to start reviewing from the end result to understand the structure of the flow.

  #### GroupOutputs function rationale:
  If "Avoid Partial Spends" is enabled, the function gathers outputs with the same script together inside a container. So Coin Selection can treats them as if them were just one possible input and either select them all or not select them.

  #### How the Inputs Fetch + Selection process roughly works:

  ```
  1. Fetch user’s manually selected inputs.
  2. Fetch wallet available coins (walks through the entire wallet txes map) and insert them into a set of vectors (each vector store outputs from a single type).
  3. Coin Selection Process:
     Call `AttemptSelection` 8 times. Each of them expands the coin eligibility filter (accepting a larger subset of coins in the calculation) until it founds a solutions or completely fails if no solutions gets founds after the 8 rounds.

     Each `AttemptSelection` call performs the following actions:
       - For each output type supported by the wallet (P2SH, P2PK, P2WPKH, P2WSH and a combination of all of them):
         Call ‘ChooseSelectionResult’ providing the respective, filtered by type, coins vector. Which:
             I. Groups the outputs vector twice (one for positive only and a second one who includes the negative ones as well).
                - GroupOutputs walks-through the entire inputted coins vector one time at least, + more if we are avoiding partial spends, to generate a vector of OutputGroups.
             II. Then performs every coin selection algorithm using the recently created vector of OutputGroup: (1) BnB, (2) knapsack and (3) SRD.
             III. Then returns the best solution out of them.
  ```

  We perform the general operation of gathering outputs, with the same script, into a single container inside:
  Each coins selection attempt (8 times —> each coin eligibility filter), for each of the outputs vector who were filtered by type (plus another one joining all the outputs as well if needed), twice (one for the positive only outputs effective value and a second one for all of them).

  So, in the worst case scenario where no solution is found after the 8 Coin Selection attempts, the `GroupOutputs` function is called 80 times (8 * 5 * 2).

  #### Improvements:

  This proposal streamlines the process so that the output groups, filtered by coin eligibility and type, are created in a single loop outside of the Coin Selection Process.

  The new process is as follows:

  ```
  1. Fetch user’s manually selected inputs.
  2. Fetch wallet available coins.
  3. Group outputs by each coin eligibility filter and each different output type found.
  4. Coin Selection Process:
     Call AttemptSelection 8 times. Each of them expands the coin eligibility filter (accepting different output groups) until it founds a solutions or completely fails if no solutions gets founds after the 8 rounds.

     Each ‘AttemptSelection’ call performs the following actions:
        - For each output type supported by the wallet (P2SH, P2PK, P2WPKH, P2WSH and all of them):
            A. Call ‘ChooseSelectionResult’ providing the respective, filtered by type, output group. Which:
               I. Performs every coin selection algorithm using the provided vector of OutputGroup: (1) BnB, (2) knapsack and (3) SRD.
               II. Then returns the best solution out of them.
  ```

  Extra Note:
  The next steps after this PR will be to:
  1) Merge `AvailableCoins` and `GroupOutputs` processes.
  2) Skip entire coin selection rounds if no new coins are added into the subsequent round.
  3) Remove global feerates from the OutputGroup class.
  4) Remove secondary "grouped" tx creation from `CreateTransactionInternal` by running Coin Selection results over the aps grouped outputs vs non-aps ones.

ACKs for top commit:
  S3RK:
    ReACK 6a302d4
  achow101:
    ACK 6a302d40df
  theStack:
    re-ACK 6a302d40df 🥥

Tree-SHA512: dff849063be328e7d9c358ec80239a6db2cd6131963b511b83699b95b337d3106263507eaba0119eaac63e6ac21c6c42d187ae23d79d9220b90c323d44b01d24
2023-03-06 18:51:34 -05:00
Andrew Chow
5e1aab2334
Merge bitcoin/bitcoin#27155: doc: Expand scantxoutset help text to cover tr() and miniscript
e4ede64fe8 Expand scantxoutset help text to cover tr() and miniscript (Greg Sanders)

Pull request description:

ACKs for top commit:
  achow101:
    ACK e4ede64fe8
  darosior:
    webACK e4ede64fe8

Tree-SHA512: 6b5d9e7fccc8242f4534861c1b438ec40fb03fbf5968c5a4af5ddbced73df6d666812053c2e12a1e0a34057f6f4fe11987c8c59d8cdfa48ca7ab7d2720b51ef9
2023-03-06 11:15:16 -05:00
Andrew Chow
dddc936d83
Merge bitcoin/bitcoin#25491: wallet: use Mutex for g_sqlite_mutex instead of GlobalMutex
4163093d63 wallet: use Mutex for g_sqlite_mutex instead of GlobalMutex (Vasil Dimov)

Pull request description:

  Using `Mutex` provides stronger guarantee than `GlobalMutex` wrt Clang's
  thread safety analysis. Thus it is better to reduce the usage of
  `GlobalMutex` in favor of `Mutex`.

  Using `Mutex` for `g_sqlite_mutex` is ok because its usage is limited in
  `wallet/sqlite.cpp` and it does not require propagating the negative
  annotations to not relevant code.

ACKs for top commit:
  achow101:
    ACK 4163093d63
  hebasto:
    re-ACK 4163093d63
  TheCharlatan:
    ACK 4163093d63

Tree-SHA512: 4913bcb8437ecf0e6b6cb781d02a6d24ffb4bf3e2e1899fa60785eab41c4c65dbdd9600bcb696290c873661b873ad61e5a4c4f205b7e66fdef2ae17c676cd12f
2023-03-06 10:50:10 -05:00
Greg Sanders
e4ede64fe8 Expand scantxoutset help text to cover tr() and miniscript 2023-03-06 10:49:43 -05:00
glozow
2a0c05defd
Merge bitcoin/bitcoin#27209: ci: Remove unused EXPECTED_TESTS_DURATION_IN_SECONDS env var
3fffff50f6 ci: Remove unused EXPECTED_TESTS_DURATION_IN_SECONDS env var (MarcoFalke)

Pull request description:

  Remove long unused travis leftover

ACKs for top commit:
  fanquake:
    ACK 3fffff50f6
  dergoegge:
    ACK 3fffff50f6

Tree-SHA512: 34b85a18c0d34f54bbc6ff5c2454307318e4747145e11f52f08baddefef9e1b80d8f6c85efcad97e575380162fd193f2aa837e99b156ed7e3e95370b635ddf4e
2023-03-06 14:00:36 +00:00
furszy
6a302d40df
wallet: single output groups filtering and grouping process
Optimizes coin selection by performing the "group outputs"
procedure only once, outside the "attempt selection" process.

Avoiding the repeated execution of the 'GroupOutputs' operation
that occurs on each coin eligibility filters (up to 8 of them);
then for every coin vector type plus one for all the coins together.

This also let us not perform coin selection over coin eligibility
filtered groups that don't add new elements.
(because, if the previous round failed, and the subsequent one has
the same coins, then this new round will fail again).
2023-03-06 09:45:40 -03:00
furszy
bd91ed1cb2
wallet: unify outputs grouping process
The 'GroupOutputs()' function performs the same
calculations for only-positive and mixed groups,
the only difference is that when we look for
only-positive groups, we discard negative utxos.

So, instead of wasting resources calling GroupOutputs()
for positive-only first, then call it again to include
the negative ones in the result, we can execute
GroupOutputs() only once, including in the response
both group types (positive-only and mixed).
2023-03-06 09:45:40 -03:00
furszy
55962001da
test: coinselector_tests refactor, use CoinsResult instead of plain std::vector
No functional changes. Only cosmetic changes to simplify the follow-up commit.
2023-03-06 09:45:40 -03:00
furszy
34f54a0a3a
wallet: decouple outputs grouping process from each ChooseSelectionResult
Another step towards the single OutputGroups calculation goal
2023-03-06 09:45:40 -03:00
furszy
461f0821a2
refactor: make OutputGroup::m_outputs field a vector of shared_ptr
Initial steps towards sharing COutput instances across all possible
OutputGroups (instead of copying them time after time).
2023-03-06 09:45:40 -03:00
furszy
d8e749bb84
test: wallet, add coverage for outputs grouping process
The following scenarios are covered:

1) 10 UTXO with the same script:
   partial spends is enabled --> outputs must not be grouped.

2) 10 UTXO with the same script:
   partial spends disabled --> outputs must be grouped.

3) 20 UTXO, 10 one from scriptA + 10 from scriptB:
   a) if partial spends is enabled --> outputs must not be grouped.
   b) if partial spends is not enabled --> 2 output groups expected (one per script).

3) Try to add a negative output (value - fee < 0):
   a) if "positive_only" is enabled --> negative output must be skipped.
   b) if "positive_only" is disabled --> negative output must be added.

4) Try to add a non-eligible UTXO (due not fulfilling the min depth target for
 "not mine" UTXOs) --> it must not be added to any group

5) Try to add a non-eligible UTXO (due not fulfilling the min depth target for
 "mine" UTXOs) --> it must not be added to any group

6) Surpass the 'OUTPUT_GROUP_MAX_ENTRIES' size and verify that a second partial
group gets created.
2023-03-06 09:45:40 -03:00
fanquake
40c6c85c05
Merge bitcoin/bitcoin#27192: util: add missing include and fix function signature
8847ce44e0 util: add missing include and fix function signature (Cory Fields)

Pull request description:

  ping hebasto

  Discovered while testing pre-compiled header support with CMake: https://github.com/theuni/bitcoin/commits/cmake-pch-poc. Compilation of that branch fails without this fix and succeeds with it.

  Similar to the fix in #27144.

  The problem of having a default argument in the definition was masked by the missing include. Using PCH forces that include, so we end up with the compiler error we should've been getting all along.

ACKs for top commit:
  fanquake:
    ACK 8847ce44e0

Tree-SHA512: 5eb9a6691ee37cbc5033a48aedcbf5c93af3b234614ae14c3fcc858f1ee505f630ad68f8bbb69ffa280080c8d0f91451362cb3819290b741ce906b2b3224a622
2023-03-04 08:17:37 +01:00
fanquake
236cd231d0
Merge bitcoin/bitcoin#27197: Fix typos in comments to make linter happy
987f1bb41c Fixed a couple of typos in comments to make linter happy (hernanmarino)

Pull request description:

  While working on a different PR, I stumbled upon a couple of typos being reported by the linter and fixed them.

ACKs for top commit:
  kristapsk:
    utACK 987f1bb41c

Tree-SHA512: bf53e150bb7c6b59a9dc270bdf748bf8fb0430f35d1d0a68ef70e9345d322470c4a0b5cd7f2c3cfc81ce087bb6624d820d5f4501f58aaa5806f358ddd4e0b551
2023-03-04 08:10:46 +01:00
Cory Fields
8847ce44e0 util: add missing include and fix function signature 2023-03-03 22:19:00 +00:00
hernanmarino
987f1bb41c Fixed a couple of typos in comments to make linter happy 2023-03-03 19:06:02 -03:00
furszy
06ec8f9928
wallet: make OutputGroup "positive_only" filter explicit
And not hide it inside the `OutputGroup::Insert` method.
This method does not return anything if insertion fails.

We can know before calling `Insert` whether the coin
will be accepted or not.
2023-03-03 18:18:03 -03:00
fanquake
3b88c85025
Merge bitcoin/bitcoin#26612: refactor: RPC: pass named argument value as string_view
545ff924ab refactor: use string_view for RPC named argument values (stickies-v)
7727603e44 refactor: reduce unnecessary complexity in ParseNonRFCJSONValue (stickies-v)
1d02e59901 test: add cases to JSON parsing (stickies-v)

Pull request description:

  Inspired by MarcoFalke's [comment](https://github.com/bitcoin/bitcoin/pull/26506#discussion_r1036149426). Main purpose of this PR is to minimize copying (potentially large) RPC named arguments when calling `.substr()` by using `std::string_view` instead of `std::string`. Furthermore, cleans up the code by removing unnecessary complexity in `ParseNonRFCJSONValue()` (done first to avoid refactoring required to concatenate `string` and `string_view`), updates some naming and adds a few test cases. Should not introduce any behaviour change.

  ## Questions
  - ~Was there actually any merit to `ParseNonRFCJSONValue()` surrounding the value with brackets and then parsing it as an array? I don't see it, and the new approach doesn't fail any tests. Still a bit suspicious about it though.~
    - Cleared up by https://github.com/bitcoin/bitcoin/pull/26506#pullrequestreview-1211984059
    - If there are no objections to 7727603e44, I think we should follow up with a PR to rename `ParseNonRFCJSONValue()` to a local `Parse()` helper function (that throws if invalid), remove it from `client.h` and merge the test coverage we currently have on `ParseNonRFCJSONValue()` with the coverage we have on `UniValue::read()`.

ACKs for top commit:
  ryanofsky:
    Code review ACK 545ff924ab
  MarcoFalke:
    review ACK 545ff924ab 📻

Tree-SHA512: b1c89fb010ac9c3054b023cac1acbba2a539a09cf39a7baffbd7f7571ee268d5a6d98701c7ac10d68a814526e8fd0fe96ac1d1fb072f272033e415b753f64a5c
2023-03-03 15:23:43 +01:00
fanquake
a12b27a2a6
Merge bitcoin/bitcoin#27183: doc: Update Transifex links and slug format in Release Process
9c371e50a2 doc: Update Transifex links and slug format in Release Process (Hennadii Stepanov)

Pull request description:

  Reflected the recent changes in Transifex's [workflow](https://github.com/bitcoin/bitcoin/pull/26321) and on its website.

ACKs for top commit:
  jarolrod:
    ACK 9c371e50a2

Tree-SHA512: d11f2fa197bba2e3198ca255d80edd7178e8074dc84b03262054d01fa33c8160ca492a1358e9da1e7d32cec21a298593322d10e4cb045ef99ccec2cbe901cb63
2023-03-02 22:02:58 +01:00
Andrew Chow
74981aa02d
Merge bitcoin/bitcoin#27172: guix: switch to some minimal versions of packages in our manifest
2c9eb4afe1 guix: use cmake-minimal over cmake (fanquake)
1475515312 guix: use coreutils-minimal over coreutils (fanquake)
4445621415 guix: use bash-minimal over bash (fanquake)

Pull request description:

  Minimal versions of the same packages, that should still be sufficient for our use:

  > (define-public bash-minimal
    ;; A stripped-down Bash for non-interactive use.

  > (define-public coreutils-minimal
    ;; Coreutils without its optional dependencies.

  > ;;; This minimal variant of CMake does not include the documentation.  It is
  ;;; used by the cmake-build-system.
  (define-public cmake-minimal

ACKs for top commit:
  TheCharlatan:
    ACK 2c9eb4afe1
  Sjors:
    tACK 2c9eb4afe1
  achow101:
    ACK 2c9eb4afe1
  hebasto:
    ACK 2c9eb4afe1,

Tree-SHA512: f91ca9e088b8346b20c2affc80870c31640de3aedcfcc0fb98a5e82c77ef64537870b88552f26759d31d8d0956b1fd685e6c25d5acbc92f5feaececd1a7dd37e
2023-03-01 11:07:04 -05:00
Hennadii Stepanov
9c371e50a2
doc: Update Transifex links and slug format in Release Process 2023-03-01 15:01:16 +00:00
fanquake
4d24e9c571
Merge bitcoin/bitcoin#27169: Update translations for 25.0 soft translation string freeze
9172cc672e qt: Update translation source file (Hennadii Stepanov)
7b0cbf444d qt: Bump Transifex slug for 25.x (Hennadii Stepanov)
369023d22d qt: Periodic translation updates from Transifex (Hennadii Stepanov)

Pull request description:

  This PR follows our [Release Process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md).

  Required to open Transifex translations for 25.0 on 2023-03-01 as it's [planned](https://github.com/bitcoin/bitcoin/issues/26549).

  **NOTE.** Translations for the following languages for the latest 24.x Transifex resource have been effectively cancelled/damaged/vandalized:
  - German (de) by [nesbonk83](https://www.transifex.com/user/profile/nesbonk83/) on 2023-01-27
  - Dutch (nl) by [bram00767](https://www.transifex.com/user/profile/bram00767/) on 2022-12-17
  - Spanish, Mexico (es_MX) by [VCFNFT](https://www.transifex.com/user/profile/VCFNFT/) on 2022-08-08

  The first commit ignores changes to translations mentioned above.

ACKs for top commit:
  jarolrod:
    ACK 9172cc672e

Tree-SHA512: 85641facecd11526bbcde934b43629aba1b856c4f97272a956c2ce194af8a1723325a160a0a518fc052af9373f853204848b58d3c0a3bea09788fccfc5d9f557
2023-03-01 14:31:06 +01:00
fanquake
cb40639bdf
Merge bitcoin/bitcoin#27165: Make miniscript_{stable,smart} fuzzers avoid too large scripts
56e37e71a2 Make miniscript fuzzers avoid script size limit (Pieter Wuille)
bcec5ab4ff Make miniscript fuzzers avoid ops limit (Pieter Wuille)
213fffa513 Enforce type consistency in miniscript_stable fuzz test (Pieter Wuille)
e1f30414c6 Simplify miniscript fuzzer NodeInfo struct (Pieter Wuille)
5abb0f5ac3 Do base type propagation in miniscript_stable fuzzer (Pieter Wuille)

Pull request description:

  This adds a number of improvements to the miniscript fuzzers that all amount to rejecting invalid or overly big miniscripts early on:
  * Base type propagation in the miniscript_stable fuzzers prevents constructing a large portion of miniscripts that would be illegal, with just a little bit of type logic in the fuzzer. The fuzzer input format is unchanged.
  * Ops and script size tracking in GenNode means that too-large scripts (either due to script size limit or ops limit) will be detected on the fly during fuzz input processing, before actually constructing the scripts.

  Closes #27147.

ACKs for top commit:
  darosior:
    re-ACK 56e37e71a2
  dergoegge:
    tACK 56e37e71a2

Tree-SHA512: 245584adf9a6644a35fe103bc81b619e5b4f5d467571a761b5809d08b1dec48f7ceaf4d8791ccd8208b45c6b309d2ccca23b3d1ec5399df76cd5bf88f2263280
2023-02-28 17:04:47 +00:00
fanquake
4398cfb22b
Merge bitcoin/bitcoin#27173: valgrind: remove libsecp256k1 suppression
29b62c01c8 valgrind: remove libsecp256k1 suppression (fanquake)

Pull request description:

  I am no-longer been able to recreate this issue, atleast after the most recent libsecp256k1 changes. Can someone else confirm?

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 29b62c01c8
  sipa:
    utACK 29b62c01c8

Tree-SHA512: e61e5b88ac2af3c779f23d999938bec4497f6433a029c07dd57a9481fe9cc104d8d8f63586910b29f41e66bbf0ed094bc7539c0df84754a1783c4b1b15af072e
2023-02-28 16:56:29 +00:00
glozow
a8080c0def
Merge bitcoin/bitcoin#23897: refactor: Move calculation logic out from CheckSequenceLocksAtTip()
75db62ba4c refactor: Move calculation logic out from `CheckSequenceLocksAtTip()` (Hennadii Stepanov)
3bc434f459 refactor: Add `CalculateLockPointsAtTip()` function (Hennadii Stepanov)

Pull request description:

  This PR is follow up for bitcoin/bitcoin#22677 and bitcoin/bitcoin#23683.

  On master (013daed9ac) it is not obvious that `CheckSequenceLocksAtTip()` function can modify its `LockPoints* lp` parameter which leads to https://github.com/bitcoin/bitcoin/pull/22677#discussion_r762040101.

  This PR:
  - separates the lockpoint calculate logic from `CheckSequenceLocksAtTip()` function into a new `CalculateLockPointsAtTip()` one
  - cleans up the `CheckSequenceLocksAtTip()` function interface
  - makes code easier to reason about (hopefully)

ACKs for top commit:
  achow101:
    ACK 75db62ba4c
  stickies-v:
    re-ACK 75db62b

Tree-SHA512: 072c3fd9cd1e1b0e0bfc8960a67b01c80a9f16d6778f374b6944ade03a020415ce8b8ab2593b0f5e787059c8cf90af798290b4c826785d41955092f6e12e7486
2023-02-28 16:53:02 +00:00
Andrew Chow
8303f11e10
Merge bitcoin/bitcoin#27170: refactor: Stop using gArgs global in system.cpp
9a9d5da11f refactor: Stop using gArgs global in system.cpp (Ryan Ofsky)
b20b34f5b3 refactor: Use new GetConfigFilePath function (Ryan Ofsky)

Pull request description:

  Most of the code in `util/system.cpp` that was hardcoded to use the global `ArgsManager` instance `gArgs` has been changed to stop using it (for example in https://github.com/bitcoin/bitcoin/pull/20092). But a few hardcoded references to `gArgs` remain. This commit removes the last ones so these functions aren't reading or writing global state.

  Noticed these `gArgs` references while reviewing #27073

ACKs for top commit:
  achow101:
    ACK 9a9d5da11f
  stickies-v:
    ACK 9a9d5da11
  willcl-ark:
    tACK 9a9d5da11

Tree-SHA512: 2c74b0d5fc83e9ed2ec6562eb26ec735512f75db8876a11a5d5f04e6cdbe0cd8beec19894091aa2cbf29319194d2429ccbf8036f5520ecc394f6fe89a0079a7b
2023-02-28 11:01:21 -05:00
fanquake
9384536eb3
Merge bitcoin/bitcoin#27174: ci: bump lint task to bookworm for git v2.38
a984beeca1 [ci] change lint to bookworm for git v2.38 (glozow)

Pull request description:

  Since 5497c14, verify-commits.py uses `git merge-tree` which requires git v2.38 or later. Fix the lint jobs on master (e.g. https://cirrus-ci.com/task/4971007513985024).

ACKs for top commit:
  achow101:
    ACK a984beeca1
  hebasto:
    re-ACK a984beeca1

Tree-SHA512: dd50598aefb6f9c86cf221dea27fcc521e335cb182f7a3abcb3215d3991794354085be3e07c133ab74ca8b957e3a6acc43722899165957b3d898867d6253ebdc
2023-02-28 15:44:19 +00:00
fanquake
c37fb251f5
Merge bitcoin/bitcoin#27176: docs: GetDataDirNet and GetDataDirBase don't create datadir
fb0dbe9423 docs: GetDataDirNet and GetDataDirBase don't create datadir (stickies-v)

Pull request description:

  Since #27073, the behaviour of `GetDataDir()` [changed](https://github.com/bitcoin/bitcoin/pull/27073/files#diff-19427b0dd1a791adc728c82e88f267751ba4f1c751e19262cac03cccd2822216L435-L443) to only return the datadir path, but not create it if non-existent. This also changed the behaviour of `GetDataDirNet()` and `GetDataDirBase()` but the docs do not yet reflect that.

ACKs for top commit:
  TheCharlatan:
    ACK fb0dbe9423
  theStack:
    ACK fb0dbe9423
  willcl-ark:
    ACK fb0dbe942

Tree-SHA512: 3f10f4871df59882f3649c6d3b2362cae2f8a01ad0bd0c636c5608b0d177d279a2e8712930b819d6d3912e91fa6447b9e54507c33d8afe427f7f39002b013bfb
2023-02-28 15:34:23 +00:00
Andrew Chow
bb136aaf2c
Merge bitcoin/bitcoin#26533: prune: scan and unlink already pruned block files on startup
3141eab9c6 test: add functional test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
e252909e56 test: add unit test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
77557dda4a prune: scan and unlink already pruned block files on startup (Andrew Toth)

Pull request description:

  There are a few cases where we can mark a block and undo file as pruned in our block index, but not actually remove the files from disk.
  1. If we call `FindFilesToPrune` or `FindFilesToPruneManual` and crash before `UnlinkPrunedFiles`.
  2. If on Windows there is an open file handle to the file somewhere else when calling `fs::remove` in `UnlinkPrunedFiles` (https://en.cppreference.com/w/cpp/filesystem/remove, https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilew#remarks). This could be from another process, or if we are calling `ReadBlockFromDisk`/`ReadRawBlockFromDisk` without having a lock on `cs_main` (which has been allowed since ccd8ef65f9).

  This PR mitigates this by scanning all pruned block files on startup after `LoadBlockIndexDB` and unlinking them again.

ACKs for top commit:
  achow101:
    ACK 3141eab9c6
  pablomartin4btc:
    re-ACK with added functional test 3141eab9c6.
  furszy:
    Code review ACK 3141eab9
  theStack:
    Code-review ACK 3141eab9c6

Tree-SHA512: 6c73bc57838ad1b7e5d441af3c4d6bf4c61c4382e2b86485e57fbb74a61240710c0ceeceb8b4834e610ecfa3175c6955c81ea4b2285fee11ca6383f472979d8d
2023-02-28 09:54:10 -05:00
glozow
a984beeca1
[ci] change lint to bookworm for git v2.38
Since 5497c14, verify-commits.py requires git merge-tree which is only
available in git v2.38 or later.
2023-02-28 14:42:48 +00:00
Pieter Wuille
56e37e71a2 Make miniscript fuzzers avoid script size limit
Use the same technique as is using in the FromString miniscript parser to
predict the final script size of the miniscript being generated in the
miniscript_stable and miniscript_smart fuzzers (by counting every unexplored
sub node as 1 script byte, which is possible because every leaf node always
adds at least 1 byte). This allows bailing out early if the script being
generated would exceed the maximum allowed size (before actually constructing
the miniscript, as that may happen only significantly later potentially).

Also add a self-check to make sure this predicted script size matches that
of generated scripts.
2023-02-28 09:42:33 -05:00
Pieter Wuille
bcec5ab4ff Make miniscript fuzzers avoid ops limit
Keep track of the total number of ops the constructed script will have
during miniscript_stable and miniscript_smart fuzzers' GenNode, so it
can abort early if the 201 ops limit would be exceeded.

Also add a self-check that the final constructed node has the predicted
ops size limit, so we know the fuzzer's logic for keeping track of this
is correct.
2023-02-28 09:22:42 -05:00
Pieter Wuille
213fffa513 Enforce type consistency in miniscript_stable fuzz test
Add a self-check to the fuzzer that the constructed types match the expected
types in the miniscript_stable fuzzer too.
2023-02-28 09:22:42 -05:00
Pieter Wuille
e1f30414c6 Simplify miniscript fuzzer NodeInfo struct
Since we now keep track of all expected child node types (even if rudimentary)
in both miniscript_stable and miniscript_smart fuzzers, there is no need anymore
for the former shortcut NodeInfo constructors without sub types.
2023-02-28 09:22:42 -05:00
Pieter Wuille
5abb0f5ac3 Do base type propagation in miniscript_stable fuzzer
Keep track of which base type (B, K, V, or W) is desired in the miniscript_stable
ConsumeStableNode function. This allows aborting early if the constructed node
won't have the right type.

Note that this does not change the fuzzer format; the meaning of inputs in
ConsumeStableNode is unmodified. The only change is that often the fuzzer will
abort early.

The direct motivation is preventing recursing v: wrappers, which are the only
fragment type that does not otherwise increase the overall minimum possible script
size. In a later commit this will be exploited to prevent overly-large scripts from
being constructed.
2023-02-28 09:22:42 -05:00
stickies-v
fb0dbe9423
docs: GetDataDirNet and GetDataDirBase don't create datadir
Since #27073, the behaviour of GetDataDir changed to only return
the datadir path, but not create it. This also changed the behaviour
of GetDataDirNet and GetDataDirBase but the docs do not yet reflect
that.
2023-02-28 12:52:42 +00:00
fanquake
2c9eb4afe1
guix: use cmake-minimal over cmake 2023-02-28 12:15:18 +00:00
fanquake
1475515312
guix: use coreutils-minimal over coreutils 2023-02-28 12:14:52 +00:00
fanquake
4445621415
guix: use bash-minimal over bash 2023-02-28 12:14:51 +00:00
fanquake
29b62c01c8
valgrind: remove libsecp256k1 suppression 2023-02-28 10:45:57 +00:00
fanquake
519ec2650e
Merge bitcoin/bitcoin#27157: init: Return ChainstateLoadStatus::INTERRUPTED when verification was interrupted.
c5825e14f8 doc: add explanation for fail_on_insufficient_dbcache (Ryan Ofsky)
7dff7da4f5 init: Return more fitting ChainStateLoadStatus if verification was interrupted (Martin Zumsande)

Pull request description:

  This addresses two outstanding comments by ryanofsky from #25574:
  * return `ChainstateLoadStatus::INTERRUPTED` instead of `ChainstateLoadStatus::SUCCESS`  if verification was stopped by an interrupt. This would coincide with straightforward expectation, and it avoids a misleading [log entry](c5825e14f8/src/init.cpp (L1526)) in `init` for the block index load time (because that would include the verificiation, which didn't complete). It shouldn't affect node behavior otherwise because the shutdown signal would be caught in init anyway. In test, this would lead to an assert ([link](c5825e14f8/src/test/util/setup_common.cpp (L230))), which also makes more sense because benign interrupts are not expected there during init.
  This can be tested by setting a large value for `-checkblocks`, interrupting the node during block verification and observing the log.
   https://github.com/bitcoin/bitcoin/pull/25574#discussion_r1110050930
  * add documentation for `require_full_verification` https://github.com/bitcoin/bitcoin/pull/25574#discussion_r1110031541

ACKs for top commit:
  MarcoFalke:
    thanks lgtm ACK c5825e14f8

Tree-SHA512: ca1c71a1b046d30083337dd9ef6d52e66fa1ac8c4ecd807716e4aa6a894179a81df41caee916fa30997fd6e0b284412a3c8f2919d19c29d826fb580ffb89fd73
2023-02-28 10:40:24 +00:00
fanquake
e60a58f191
Merge bitcoin/bitcoin#27118: depends: harden libevent
ff4a73aea2 depends: use FORTIFY_SOURCE=3 with libevent (fanquake)

Pull request description:

  Use `FORTIFY_SOURCE=3` when building libevent in depends. I've upstreamed a change to switch libevent from using =2 to =3 as well: https://github.com/libevent/libevent/pull/1418.

  Solves half of #27038, by giving us some fortified funcs in `bitcoin-cli`.

ACKs for top commit:
  TheCharlatan:
    ACK ff4a73aea2

Tree-SHA512: eaf692ec92b288f0cb524c011fc81529f58efa4c43d418a7b3ae7108eba2bccba708a81a28ac6d063267be80ca615637c6e3fccc02497d7367af2eaae0e8d812
2023-02-28 10:28:34 +00:00
Ryan Ofsky
9a9d5da11f refactor: Stop using gArgs global in system.cpp
Most of the code in util/system.cpp that was hardcoded to use the global
ArgsManager instance `gArgs` has been changed to work with explicit ArgsManager
instances (for example in https://github.com/bitcoin/bitcoin/pull/20092). But a
few hardcoded references to `gArgs` remain. This commit removes the last ones
so these functions aren't reading or writing global state.
2023-02-27 14:21:13 -05:00
Ryan Ofsky
b20b34f5b3 refactor: Use new GetConfigFilePath function
New function was introduced by willcl-ark <will@256k1.dev> in commit
56e370fbb9 from
https://github.com/bitcoin/bitcoin/pull/27073 and removes some duplicate code.
2023-02-27 14:14:58 -05:00
Andrew Chow
710cab1d43
Merge bitcoin/bitcoin#26032: wallet: skip R-value signature grinding for external signers
807de2cebd wallet: skip R-value grinding for external signers (Sjors Provoost)
72b763e452 wallet: annotate bools in descriptor SPKM FillPSBT() (Sjors Provoost)

Pull request description:

  When producing a dummy signature for the purpose of estimating the transaction fee, do not assume an external signer performs R-value grinding on the signature.

  In particular, this avoids a scenario where the fee rate is 1 sat / vbyte and a transaction with a 72 byte signature is not accepted into our mempool.

  Suggested testing:
  1. On master, launch with `-signet` and create an external signer wallet using e.g. a Trezor and HWI, see [guide](https://github.com/bitcoin/bitcoin/blob/master/doc/external-signer.md#example-usage) (with the GUI it should "just work" once you have the HWI path configured).
  2. Create a few addresses and fund them from the faucet: https://signet.bc-2.jp/ (wait for confirmation)
  3. Create another address, and now send the entire wallet to it, set the fee to 1 sat/byte
  4. Most likely this transaction never gets broadcast and you won't see it on the [signet explorer](https://explorer.bc-2.jp)

  5. With this PR, try again.
  6. Check the explorer and inspect the transaction. Each input witness starts with either `30440220` (R has 32 bytes) or `30440221` (R has 33 bytes). See this explainer for [DER encoding](https://bitcoin.stackexchange.com/questions/92680/what-are-the-der-signature-and-sec-format).

  Fixes #26030

ACKs for top commit:
  S3RK:
    ACK 807de2cebd
  achow101:
    ACK 807de2cebd
  furszy:
    ACK 807de2ce
  ishaanam:
    utACK 807de2cebd

Tree-SHA512: 64f626a3030ef0ab1e43af86d8fba113151512561baf425e6e5182af53df3a64fa9e85c7f67bf4ed15b5ad6e5d5afc7fbba8b6e1f3bad388e48db51cb9446074
2023-02-27 12:37:46 -05:00
fanquake
82793f1984
Merge bitcoin/bitcoin#27146: Fix various libbitcoinkernel DLL build problems
5da7c0b3e3 build: allow libitcoinkernel dll builds now that exports are fixed (Cory Fields)
130490aef9 build: always build bitcoin-chainstate against static libbitcoinkernel (Cory Fields)
545a74ef32 build: fix bitcoin-chainstate when libbitcoinkernel is static (Cory Fields)
9c253d2398 build: don't define DLL_EXPORT for windows (Cory Fields)

Pull request description:

  Fixes #25008.
  Fixes #19772.

  1. Fixup the build defines so that exports are clean.
  2. Work around a libtool issue wrt dependency calculation
  3. Simplify everything by only ever building in-tree bitcoin-chainstate against a static libbitcoinkernel
  4. Remove Windows-only hack that disabled dll creation

ACKs for top commit:
  TheCharlatan:
    ACK 5da7c0b3e3

Tree-SHA512: 61bab457e13842946387240da703d313509af30d4ca3371a19a26a5ef1716e4d7107b09567323041b549ab1fc97a064aa1d6992406936ab9c491a616bc7f4e7f
2023-02-27 14:41:47 +00:00
fanquake
a2877f7ad3
Merge bitcoin/bitcoin#25227: Handle invalid hex encoding in ParseHex
faab273e06 util: Return empty vector on invalid hex encoding (MarcoFalke)
fa3549a77b test: Add hex parse unit tests (MarcoFalke)

Pull request description:

  Seems a bit confusing to happily accept random bytes and pretend they are hex encoded strings.

ACKs for top commit:
  stickies-v:
    re-ACK faab273e06

Tree-SHA512: a808135f744f50aece03d4bf5a71481c7bdca1fcdd0d5b113abdb0c8b382bf81cafee6d17c239041fb49b59f4e19970f24a475378e7f711c3a47d6438de2bdab
2023-02-27 14:27:50 +00:00
Hennadii Stepanov
9172cc672e
qt: Update translation source file
The diff is produced by running `make -C src translate`.
2023-02-27 14:07:19 +00:00
Hennadii Stepanov
7b0cbf444d
qt: Bump Transifex slug for 25.x 2023-02-27 14:01:14 +00:00
Hennadii Stepanov
369023d22d
qt: Periodic translation updates from Transifex
Pulled from 24.x resource.
Changes to "de", "es_MX" and "nl" have been ignored as they remove
translations altogether.
2023-02-27 13:53:29 +00:00