Commit Graph

29597 Commits

Author SHA1 Message Date
Vasil Dimov
dd4e957dcd
test: ensure banlist can be read from disk after restart
With `banlist.dat` (being written in addrv1 format) if we would try to
write a Tor v3 subnet, it would serialize as a dummy-all-0s IPv6
address and subsequently, when deserialized will not result in the same
subnet.

This problem does not exist with `banlist.json` where the data is saved
in textual, human-readable form.
2021-06-21 15:27:03 +02:00
Vasil Dimov
d197977ae2
banman: save the banlist in a JSON format on disk
Save the banlist in `banlist.json` instead of `banlist.dat`.

This makes it possible to store Tor v3 entries in the banlist on disk
(and any other addresses that cannot be serialized in addrv1 format).

Only read `banlist.dat` if it exists and `banlist.json` does not
exist (first start after an upgrade).

Supersedes https://github.com/bitcoin/bitcoin/pull/20904
Resolves https://github.com/bitcoin/bitcoin/issues/19748
2021-06-21 14:39:44 +02:00
W. J. van der Laan
6a67366fdc
Merge bitcoin/bitcoin#19033: http: Release work queue after event base finish
4e353cb618 http: Release work queue after event base finish (João Barbosa)

Pull request description:

  This fixes a race between `http_request_cb` and `StopHTTPServer` where
  the work queue is used after release.

  Fixes #18856.

ACKs for top commit:
  fjahr:
    Code review ACK 4e353cb618
  achow101:
    ACK 4e353cb618
  LarryRuane:
    ACK 4e353cb618
  hebasto:
    ACK 4e353cb618, tested (rebased on top of master 9313c4e6aa) on Linux Mint 20.1 (x86_64) using MarcoFalke's [patch](https://github.com/bitcoin/bitcoin/pull/19033#issuecomment-640106647), including different `-rpcthreads`/`-rpcworkqueue` cases. The bug is fixed. The code is correct.

Tree-SHA512: 185d2a9744d0d5134d782bf321ac9958ba17b11a5b3d70b4897c8243e6b146dfd3f23c57aef8e10ae9484374120b64389c1949a9cf0a21dccc47ffc934c20930
2021-06-21 11:15:42 +02:00
W. J. van der Laan
f6a25bea82
Merge bitcoin/bitcoin#22147: p2p: Protect last outbound HB compact block peer
30aee2dfe6 tests: Add test for compact block HB selection (Pieter Wuille)
6efbcec4de Protect last outbound HB compact block peer (Suhas Daftuar)

Pull request description:

  If all our high-bandwidth compact block serving peers (BIP 152) stall block
  download, then we can be denied a block for (potentially) a long time. As
  inbound connections are much more likely to be adversarial than outbound
  connections, mitigate this risk by never removing our last outbound HB peer if
  it would be replaced by an inbound.

ACKs for top commit:
  achow101:
    ACK 30aee2dfe6
  ariard:
    Code ACK 30aee2dfe
  jonatack:
    ACK 30aee2dfe6

Tree-SHA512: 5c6c9326e3667b97e0864c371ae2174d2be9054dad479f4366127b9cd3ac60ffa01ec9707b16ef29cac122db6916cf56fd9985733390017134ace483278921d5
2021-06-21 08:18:55 +02:00
W. J. van der Laan
a305a687e7
Merge bitcoin/bitcoin#22244: devtools: Correctly extract symbol versions in symbol-check
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py (W. J. van der Laan)
a33381acf5 devtools: Add xkb version to symbol-check (W. J. van der Laan)
19e598bab0 devtools: Fix verneed section parsing in pixie (W. J. van der Laan)

Pull request description:

  I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section.

  This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454.

  Fix it by correcting the offset computation.

  - xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py`

  This unfortunately brings to light some symbols that have been introduced since and weren't caught (from a gitian compile of master):

  ```
  bitcoin-cli: symbol getrandom from unsupported version GLIBC_2.25
  bitcoin-cli: failed IMPORTED_SYMBOLS
  bitcoind: symbol getrandom from unsupported version GLIBC_2.25
  bitcoind: symbol log from unsupported version GLIBC_2.29
  bitcoind: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoind: symbol pow from unsupported version GLIBC_2.29
  bitcoind: symbol exp from unsupported version GLIBC_2.29
  bitcoind: failed IMPORTED_SYMBOLS
  bitcoin-qt: symbol exp from unsupported version GLIBC_2.29
  bitcoin-qt: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoin-qt: symbol log from unsupported version GLIBC_2.29
  bitcoin-qt: symbol pow from unsupported version GLIBC_2.29
  bitcoin-qt: symbol statx from unsupported version GLIBC_2.28
  bitcoin-qt: symbol getrandom from unsupported version GLIBC_2.25
  bitcoin-qt: symbol renameat2 from unsupported version GLIBC_2.28
  bitcoin-qt: symbol getentropy from unsupported version GLIBC_2.25
  bitcoin-qt: failed IMPORTED_SYMBOLS
  bitcoin-wallet: symbol exp from unsupported version GLIBC_2.29
  bitcoin-wallet: symbol log from unsupported version GLIBC_2.29
  bitcoin-wallet: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoin-wallet: failed IMPORTED_SYMBOLS
  test_bitcoin: symbol getrandom from unsupported version GLIBC_2.25
  test_bitcoin: symbol log from unsupported version GLIBC_2.29
  test_bitcoin: symbol fcntl64 from unsupported version GLIBC_2.28
  test_bitcoin: symbol pow from unsupported version GLIBC_2.29
  test_bitcoin: symbol exp from unsupported version GLIBC_2.29
  test_bitcoin: failed IMPORTED_SYMBOLS
  ```

ACKs for top commit:
  hebasto:
    ACK e8cd3700ee

Tree-SHA512: 8c15e3478eb642f01a1ddaadef03f80583f088f9fa8e3bf171ce16b0ec05ffb4675ec147d7ffc6a4360637ed47fca517c6ca2bac7bb30d794c03783cfb964b79
2021-06-21 07:58:12 +02:00
MarcoFalke
965e937434
Merge bitcoin/bitcoin#22279: fuzz: add missing ECCVerifyHandle to base_encode_decode
906d791311 fuzz: add missing ECCVerifyHandle to base_encode_decode (Andrew Poelstra)

Pull request description:

  It is possible to trigger a fuzztest failure in the `base_encode_decode` by asking it to decode any PSBT that has HD keypaths in it. For example, this one

  ```
  cHNidP8BAFUCAAAAASeaIyOl37UfxF8iD6WLD8E+HjNCeSqF1+Ns1jM7XLw5AAAAAAD/////AaBa6gsAAAAAGXapFP/pwAYQl8w7Y28ssEYPpPxCfStFiKwAAAAAAAEBIJVe6gsAAAAAF6kUY0UgD2jRieGtwN8cTRbqjxTA2+uHIgIDsTQcy6doO2r08SOM1ul+cWfVafrEfx5I1HVBhENVvUZGMEMCIAQktY7/qqaU4VWepck7v9SokGQiQFXN8HC2dxRpRC0HAh9cjrD+plFtYLisszrWTt5g6Hhb+zqpS5m9+GFR25qaAQEEIgAgdx/RitRZZm3Unz1WTj28QvTIR3TjYK2haBao7UiNVoEBBUdSIQOxNBzLp2g7avTxI4zW6X5xZ9Vp+sR/HkjUdUGEQ1W9RiED3lXR4drIBeP4pYwfv5uUwC89uq/hJ/78pJlfJvggg71SriIGA7E0HMunaDtq9PEjjNbpfnFn1Wn6xH8eSNR1QYRDVb1GELSmumcAAACAAAAAgAQAAIAiBgPeVdHh2sgF4/iljB+/m5TALz26r+En/vykmV8m+CCDvRC0prpnAAAAgAAAAIAFAACAAAA=
  ```

  which I took straight from the PSBT test vectors. The reason is that in src/psbt.h we call `DeserializeHDKeypaths`, which in turn calls `CPubKey::IsFullyValid`, which in turn asserts that a secp context has been created.

  The error appears to be masked on many systems by the definition of `instance_of_eccryptoclosure` in src/script/bitcoinconsensus.cpp, which defines a static object which contains an `ECCVerifyHandle`. If you just comment out that line you can reliably trigger the fuzz test failure, e.g. by creating a file `crash` with the above PSBT, and runnnig

  ```
  ASAN_OPTIONS=symbolize=0:detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1 UBSAN_OPTIONS=suppressions=./test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1 FUZZ=base_encode_decode ./src/test/fuzz/fuzz -seed_inputs=crash
  ```

ACKs for top commit:
  practicalswift:
    cr ACK 906d791311

Tree-SHA512: b98b60573c21efe28503fe351883c6f0d9ac99d0dd6f100537b16ac53476617b8a3f899faf0c23d893d34a01b3bbe4a784499ec6f9c7000292e850bed449bd85
2021-06-19 09:40:10 +02:00
MarcoFalke
e172ea8804
Merge bitcoin/bitcoin#22210: test: Use MiniWallet in test_no_inherited_signaling RBF test
fa7d71f270 test: Run pep-8 on touched test (MarcoFalke)
fab7e99c2a test: Use MiniWallet in test_no_inherited_signaling RBF test (MarcoFalke)
fab871f649 test: Remove unused generate() from test (MarcoFalke)
faff3f35b7 test: Add txin.sequence option to MiniWallet (MarcoFalke)

Pull request description:

  This comes with nice benefits:
  * Less code and complexity
  * Test can be run without wallet compiled in

  Also add some additional checks for `getmempoolentry` (#22209) and other cleanups 🎨

ACKs for top commit:
  mjdietzx:
    Tested ACK fa7d71f270 thanks for the explanations, nicely done
  theStack:
    ACK fa7d71f270 🍷

Tree-SHA512: 0e9b8fe985779d8d7034d256deed627125bb374b6ae2972c461b3a220739a51061c6147ad69339bee16282f82716c7f3f8a7a89c693ceb1e47ea50709272332a
2021-06-19 08:47:52 +02:00
Andrew Poelstra
906d791311 fuzz: add missing ECCVerifyHandle to base_encode_decode 2021-06-18 23:13:07 +00:00
W. J. van der Laan
0f47e01d7d
Merge bitcoin/bitcoin#20923: signet miner followups
b3c712cb28 contrib/signet/miner: remove debug code (Anthony Towns)
297e35159f bitcoin-util: use AddCommand / GetCommand (Anthony Towns)
b6d493fd4d contrib/signet/README.md: Update miner description (Anthony Towns)
e66543827c contrib/signet/miner: Automatic timestamp for first block (Anthony Towns)
a383ce5b4a contrib/signet/miner: --grind-cmd is required for calibrate (Anthony Towns)
1a45cd2e51 contrib/signet: Fix typos (Anthony Towns)

Pull request description:

  Followups from #19937

ACKs for top commit:
  laanwj:
    Code review ACK b3c712cb28

Tree-SHA512: a1003f9ee3697438114b60872b50f4300c8b52f0d58551566eb61c421d787525807ae75be205dcab2c24358cd568f53260120880109a9d728773405ff987596f
2021-06-18 19:31:38 +02:00
MarcoFalke
da1e6d5911
Merge bitcoin/bitcoin#14604: tests: Add test and refactor feature_block.py
55311197c4 Added new test for future blocks reacceptance (sanket1729)
511a5af462 Fixed inconsistencies between code and comments (sanket1729)

Pull request description:

  This Commit does 3 things:
  1) Adds a test case for checking reacceptance a previously rejected block which
  was too far in the future.
  ~~2) clean up uses of rehash or calc_sha256 where it was not needed~~
  3) While constructing block 44, this commit makes the code consistent with the expected figure in
  the comment just above it by adding a transaction to the block.
  4) Fix comment describing `sign_tx()` function

ACKs for top commit:
  duncandean:
    reACK 5531119
  brunoerg:
    reACK 55311197c4

Tree-SHA512: d40c72fcdbb0b2a0715adc58441eeea08147ee2ec5e371a4ccc824ebfdc6450698bd40aaeecb7ea7bfdb3cd1b264dd821b890276fff8b8d89b7225cdd9d6b546
2021-06-18 18:08:49 +02:00
MarcoFalke
0844084c13
Merge bitcoin/bitcoin#22249: test: kill process group to avoid dangling processes when using --failfast
451b96f7d2 test: kill process group to avoid dangling processes (S3RK)

Pull request description:

  This is an alternative to #19281

  This PR fixes a problem when after test failure with `--failfast` option there could be dangling nodes. The nodes will continue to occupy rpc/p2p ports on the machine and will cause further test failures.

  If there are any dangling nodes left at the end of the test run we kill the whole process group.
  Pros: the operations is immediate and won't lead to CI timeout
  Cons: the test_runner process is also killed and exit code is 137

  Example output:
  ```
  ...
  Early exiting after test failure

  TEST                           | STATUS    | DURATION

  rpc_decodescript.py            | ✓ Passed  | 2 s
  rpc_deprecated.py              | ✓ Passed  | 2 s
  rpc_deriveaddresses.py         | ✓ Passed  | 2 s
  rpc_dumptxoutset.py            | ✖ Failed  | 2 s

  ALL                            | ✖ Failed  | 8 s (accumulated)
  Runtime: 4 s

  Killed: 9
  > echo $?
  137
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 451b96f7d2
  aitorjs:
    ACK 451b96f7d2. Manual testing with and without **--failfast**.

Tree-SHA512: 87e510a1411b9e7571e63cf7ffc8b9a8935daf9112ffc0f069d6c406ba87743ec439808181f7e13cb97bb200fad528589786c47f0b43cf3a2ef0d06a23cb86dd
2021-06-18 14:22:51 +02:00
W. J. van der Laan
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py
The (ancient) versions specified here were deceptive. Entries older than
MAX_VERSIONS['GLIBC'], which is 2.17, are ignored here. So reorganize
the code to avoid confusion for other people reading this code.
2021-06-18 13:25:57 +02:00
fanquake
da69d9965a
Merge bitcoin/bitcoin#21871: scripts: add checks for minimum required OS versions
aa80b5759d scripts: check macOS SDK version is set (fanquake)
c972345bac scripts: check minimum required Windows version is set (fanquake)
29615aef52 scripts: check minimum required macOS vesion is set (fanquake)
8732f7b6c9 scripts: LIEF 0.11.5 (fanquake)

Pull request description:

  macOS:
  We use a compile flag ([-mmacosx-version-min=10.14](https://github.com/bitcoin/bitcoin/blob/master/depends/hosts/darwin.mk#L96)) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected.

  Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future.

  Windows:
  We use linker flags ([-Wl,--major/minor-subsystem-version](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L683)) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected.

  Gitian builds:
  ```bash
  # macOS:
  8b6fcd61d75001c37b2af3fceb5ae09f5d2fe85e97d361f684214bd91c27954a  bitcoin-f015e1c2cac9-osx-unsigned.dmg
  3c1e412bc7f5a7a5d0f78e2cd84b7096831414e1304c1307211aa3e135d89bbf  bitcoin-f015e1c2cac9-osx-unsigned.tar.gz
  50b7b2804e8481f63c69c78e3e8a71c0d811bf2db8895dd6d3edae9c46a738ae  bitcoin-f015e1c2cac9-osx64.tar.gz
  fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675  src/bitcoin-f015e1c2cac9.tar.gz
  8a20f21b20673dfc8c23e22b20ae0839bcaf65bf0e02f62381cdf5e7922936f0  bitcoin-core-osx-22-res.yml

  # Windows:
  b01fcdc2a5673387050d6c6c4f96f1d350976a121155fde3f76c2af309111f9d  bitcoin-f015e1c2cac9-win-unsigned.tar.gz
  b95bdcbef638804030671d2332d58011f8c4ed4c1db87d6ffd211515c32c9d02  bitcoin-f015e1c2cac9-win64-debug.zip
  350bf180252d24a3d40f05e22398fec7bb00e06d812204eb5a421100a8e10638  bitcoin-f015e1c2cac9-win64-setup-unsigned.exe
  2730ddabe246d99913c9a779e97edcadb2d55309933d46f1dffd0d23ecf9aae5  bitcoin-f015e1c2cac9-win64.zip
  fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675  src/bitcoin-f015e1c2cac9.tar.gz
  aa60d7a753e8cb2d4323cfbbf4d964ad3645e74c918cccd66862888f8646d80f  bitcoin-core-win-22-res.yml
  ```

ACKs for top commit:
  hebasto:
    ACK aa80b5759d, tested by breaking tests:

Tree-SHA512: 10150219910e8131715fbfe20edaa15778387616ef3bfe1a5152c7acd3958fe8f88c74961c3d3641074eb72824680c22764bb1dc01a19e92e946c2d4962a8d2c
2021-06-18 15:21:47 +08:00
fanquake
ad0c8f356e
Merge bitcoin/bitcoin#22238: build: improve detection of eBPF support
8f7704d032 build: improve detection of eBPF support (fanquake)

Pull request description:

  Just checking for the `sys/sdt.h` header isn't enough, as systems like macOS have the header, but it doesn't actually have the `DTRACE_PROBE*` probes, which leads to [compile failures](https://github.com/bitcoin/bitcoin/pull/22006#issuecomment-859559004). The contents of `sys/sdt.h` in the macOS SDK is:
  ```bash
  #ifndef _SYS_SDT_H
  #define _SYS_SDT_H

  /*
   * This is a wrapper header that wraps the mach visible sdt.h header so that
   * the header file ends up visible where software expects it to be.  We also
   * do the C/C++ symbol wrapping here, since Mach headers are technically C
   * interfaces.
   *
   * Note:  The process of adding USDT probes to code is slightly different
   * than documented in the "Solaris Dynamic Tracing Guide".
   * The DTRACE_PROBE*() macros are not supported on Mac OS X -- instead see
   * "BUILDING CODE CONTAINING USDT PROBES" in the dtrace(1) manpage
   *
   */
  #include <sys/cdefs.h>
  __BEGIN_DECLS
  #include <mach/sdt.h>
  __END_DECLS

  #endif  /* _SYS_SDT_H */
  ```

  The `BUILDING CODE CONTAINING USDT PROBES` section from the dtrace manpage is available [here](https://gist.github.com/fanquake/e56c9866d53b326646d04ab43a8df9e2), and outlines the more involved process of using USDT probes on macOS.

ACKs for top commit:
  jb55:
    utACK 8f7704d032
  practicalswift:
    cr ACK 8f7704d032
  hebasto:
    ACK 8f7704d032, tested on macOS Big Sur 11.4 (20F71) and on Linux Mint 20.1 (x86_64) with depends.

Tree-SHA512: 5f1351d0ac2e655fccb22a5454f415906404fdaa336fd89b54ef49ca50a442c44ab92d063cba3f161cb8ea0679c92ae3cd6cfbbcb19728cac21116247a017df5
2021-06-18 15:16:00 +08:00
Samuel Dobson
5c2e2afe99
Merge bitcoin/bitcoin#21365: Basic Taproot signing support for descriptor wallets
458a345b05 Add support for SIGHASH_DEFAULT in RPCs, and make it default (Pieter Wuille)
c0f0c8eccb tests: check spending of P2TR (Pieter Wuille)
a2380127e9 Basic Taproot signing logic in script/sign.cpp (Pieter Wuille)
49487bc3b6 Make GetInputUTXO safer: verify non-witness UTXO match (Pieter Wuille)
fd3f6890f3 Construct and use PrecomputedTransactionData in PSBT signing (Pieter Wuille)
5cb6502ac5 Construct and use PrecomputedTransactionData in SignTransaction (Pieter Wuille)
5d2e22437b Don't nuke witness data when signing fails (Pieter Wuille)
ce9353164b Permit full precomputation in PrecomputedTransactionData (Pieter Wuille)
e841fb503d Add precomputed txdata support to MutableTransactionSignatureCreator (Pieter Wuille)
a91d532338 Add CKey::SignSchnorr function for BIP 340/341 signing (Pieter Wuille)
e77a2839b5 Use HandleMissingData also in CheckSchnorrSignature (Pieter Wuille)
dbb0ce9fbf Add TaprootSpendData data structure, equivalent to script map for P2[W]SH (Pieter Wuille)

Pull request description:

  Builds on top of #22051, adding signing support after derivation support.

  Nothing is changed in descriptor features. Signing works for key path and script path spending, through the normal sending functions, and PSBT-based RPCs. However, PSBT usability is rather low as no extensions have been defined to convey Taproot-specific information, so all script information must be known to the signing wallet.

ACKs for top commit:
  achow101:
    re-ACK 458a345b05
  fjahr:
    Code review ACK 458a345b05
  Sjors:
    ACK 458a345b05

Tree-SHA512: 30ed212cf7754763a4a81624ebc084c51727b8322711ac0b390369213c1a891d367ed8b123882ac08c99595320c11ec57ee42304ff22a69afdc3d1a0d55cc711
2021-06-18 09:12:44 +12:00
MarcoFalke
8cb43077b3
Merge bitcoin/bitcoin#22271: fuzz: Assert roundtrip equality for CPubKey
9550dffa0c fuzz: Assert roundtrip equality for `CPubKey` (Sebastian Falbesoner)

Pull request description:

  This PR is a (quite late) follow-up to #19237 (https://github.com/bitcoin/bitcoin/pull/19237#issuecomment-642203251). Looking at `CPubKey::Serialize` and `CPubKey::Unserialize` I can't think of a scenario where the roundtrip (serialization/deserialization) equality wouldn't hold.

ACKs for top commit:
  jamesob:
    crACK 9550dffa0c pending CI

Tree-SHA512: 640fb9e777d249769b22ee52c0b15a68ff0645b16c986e1c0bce9742155d14f1be601e591833e1dc8dcffebf271966c6b861b90888a44aae1feae2e0248e2c55
2021-06-17 21:40:51 +02:00
W. J. van der Laan
7b45c5e875
Merge bitcoin/bitcoin#20516: Well-defined CAddress disk serialization, and addrv2 anchors.dat
f8866e8c32 Add roundtrip fuzz tests for CAddress serialization (Pieter Wuille)
e2f0548b52 Use addrv2 serialization in anchors.dat (Pieter Wuille)
8cd8f37dfe Introduce well-defined CAddress disk serialization (Pieter Wuille)

Pull request description:

  Alternative to #20509.

  This makes the `CAddress` disk serialization format well defined, and uses it to enable addrv2 support in anchors.dat (in a way that's compatible with older software). The new format is:
  - The first 4 bytes store a format version number. Its low 19 bits are ignored (as those historically stored the `CLIENT_VERSION`), but its high 13 bits specify the actual serialization:
    - 0x00000000: LE64 encoding for `nServices`, V1 encoding for `CService` (like pre-BIP155 network serialization).
    - 0x20000000: CompactSize encoding for `nServices`, V2 encoding for `CService` (like BIP155 network serialization).
    - Any other value triggers an unsupported format error on deserialization, and can be used for future format changes.
  - The `ADDRV2_FORMAT` flag in the stream's version does not determine the actual serialization format; it only sets whether or not V2 encoding is permitted.

ACKs for top commit:
  achow101:
    ACK f8866e8c32
  laanwj:
    Code review ACK f8866e8c32
  vasild:
    ACK f8866e8c32
  jonatack:
    ACK f8866e8c32 tested rebased to master and built/run/restarted with DEBUG_ADDRMAN, peers.dat and anchors ser/deser seems fine
  hebasto:
    ACK f8866e8c32, tested on Linux Mint 20.1 (x86_64).

Tree-SHA512: 3898f8a8c51783a46dd0aae03fa10060521f5dd6e79315fe95ba807689e78f202388ffa28c40bf156c6f7b1fc2ce806b155dcbe56027df73d039a55331723796
2021-06-17 17:43:16 +02:00
Sebastian Falbesoner
9550dffa0c fuzz: Assert roundtrip equality for CPubKey 2021-06-17 17:03:03 +02:00
MarcoFalke
922abe8ca3
Merge bitcoin/bitcoin#22268: fuzz: Add temporary debug assert for oss-fuzz issue
faf1af58f8 fuzz: Add Temporary debug assert for oss-fuzz issue (MarcoFalke)

Pull request description:

  oss-fuzz is acting weird, so add an earlier assert to help troubleshooting

ACKs for top commit:
  practicalswift:
    cr ACK faf1af58f8

Tree-SHA512: 85830d7d47cf6b4edfe91a07bd5aa8f7110db0bade8df93868cf276ed04d5dd17e671f769e6a0fb5092012b86aa82bb411fb171411f15746981104ce634c88c1
2021-06-17 14:57:09 +02:00
MarcoFalke
6eafa81b32
Merge bitcoin/bitcoin#22267: fuzz: Speed up crypto fuzz target
fa483e9f68 fuzz: Speed up crypto fuzz target (MarcoFalke)

Pull request description:

  May fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34962

  Similar solution to https://github.com/bitcoin/bitcoin/pull/22005

ACKs for top commit:
  practicalswift:
    cr ACK fa483e9f68: patch looks correct and rationale makes sense

Tree-SHA512: 3788cf9f6ba0f7a0a217cd3a6a825839689425e99e4d6d657981d291a001b0da7c5abb50a68b4ee1c2a8300b87fb92e4e3ccc1171907792b40251e467c33bd53
2021-06-17 12:54:27 +02:00
MarcoFalke
faf1af58f8
fuzz: Add Temporary debug assert for oss-fuzz issue 2021-06-17 10:55:39 +02:00
MarcoFalke
fa483e9f68
fuzz: Speed up crypto fuzz target 2021-06-17 10:32:59 +02:00
MarcoFalke
dd24567a24
Merge bitcoin/bitcoin#22120: test: p2p_invalid_block: Check that a block rejected due to too-new tim…
754e802274 test: check rejected future block later accepted (Luke Dashjr)

Pull request description:

  (Luke) was unsure if the code sufficiently avoided caching a
  time-too-new rejection, so wrote this test to check it.  It looks like
  despite only exempting BLOCK_MUTATED, it is still okay because header
  failures never cache block invalidity.  This test will help ensure that
  if this ever changes, BLOCK_TIME_FUTURE gets excluded at the same time.

  This PR re-opens https://github.com/bitcoin/bitcoin/pull/17872 which went stale and addresses the nits raised by reviewers there.

ACKs for top commit:
  MarcoFalke:
    review ACK 754e802274

Tree-SHA512: a2bbc8fffb523cf2831e1ecb05f20868e30106a38cc2e369e4973fa549cca06675a668df16f76c49cc4ce3a22925404255e5c53c4232d63ba1b9fca878509aa0
2021-06-17 09:05:52 +02:00
fanquake
d50302625e
Merge bitcoin/bitcoin#22182: guix: Overhaul how guix-{attest,verify} works and hierarchy
e2c40a4ed5 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong)
4cc35daed5 Rewrite guix-{attest,verify} for new hier (Carl Dong)
28a9c9b839 Make SHA256SUMS fragment right after build (Carl Dong)

Pull request description:

  Based on:  #22075
  Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff

  The following changes resolve many usability improvements which were pointed out to me:
  1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents
  2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting
  3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var)
  4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server.

  For those who sign on an separate machine than the one you do builds on, the following steps will work:
  1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest`
  2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine
  3. Sign the `SHA256SUMS` files:
      ```bash
      for i in "<path-to-achow101>/*.SHA256SUMS"; do
          gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,}
      done
      ```
  5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs`

  -----

  After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build:
  ```
  output
  └── x86_64-w64-mingw32
      ├── bitcoin-4e069f7589da-win64-debug.zip
      ├── bitcoin-4e069f7589da-win64-setup-unsigned.exe
      ├── bitcoin-4e069f7589da-win64.zip
      ├── bitcoin-4e069f7589da-win-unsigned.tar.gz
      └── SHA256SUMS.part
  ```

  These `SHA256SUMS.part` fragments look something like:
  ```
  3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825  dist-archive/bitcoin-4e069f7589da.tar.gz
  def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip
  643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe
  a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288  x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz
  fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip
  ```

  Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level

  When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create:

  1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and
  3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs

  Then both files are signed, resulting in the following `guix.sigs` hierarchy:
  ```
  4e069f7589da/
  └── dongcarl
      ├── all.SHA256SUMS
      ├── all.SHA256SUMS.asc
      ├── noncodesigned.SHA256SUMS
      └── noncodesigned.SHA256SUMS.asc
  ```

ACKs for top commit:
  achow101:
    ACK e2c40a4ed5
  hebasto:
    ACK e2c40a4ed5, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only.

Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
2021-06-17 13:10:37 +08:00
fanquake
7c561bea52
Merge bitcoin/bitcoin#21935: Enable external signer support by default, reduce #ifdef
2f5bdcbc31 gui: misc external signer fixes and translation hints (Sjors Provoost)
d672404466 refactor: make ExternalSigner NetworkArg() and m_chain private (Sjors Provoost)
4455145e26 refactor: reduce #ifdef ENABLE_EXTERNAL_SIGNER usage (Sjors Provoost)
5be90c907e build: enable external signer by default (Sjors Provoost)
7d9453041b refactor: clean up external_signer.h includes (Sjors Provoost)
fc0eca31b3 fuzz: fix fuzz binary linking order (Sjors Provoost)

Pull request description:

  This follows the introduction of GUI support in https://github.com/bitcoin-core/gui/pull/4

  I don't think we should expect GUI users to self compile. This also enables external signer support by default for RPC users.

  In addition this PR reduces the number of `#ifdef ENABLE_EXTERNAL_SIGNER`, which also fixes #21919. When compiled with `--disable-external-signer` such wallets can't be created in RPC or GUI, but they can be loaded. Attempting any action that calls HWI will trigger an error.

  Side-note: this PR may or may not (currently) break CI for the GUI repository, as explained here: https://github.com/bitcoin-core/gui/pull/4#issuecomment-769859001

ACKs for top commit:
  achow101:
    ACK 2f5bdcbc31
  hebasto:
    re-ACK 2f5bdcbc31

Tree-SHA512: 1b71c5a8bea2be077ee9fa33a01130c957a0cf90951d4b7b04d3d0ef826bb77e474c3963abddfef2e2c1ea99d9c72cd2302d1eb9b5fcb7ba0bd2a625f006aa05
2021-06-17 12:47:37 +08:00
fanquake
65c4a36e57
Merge bitcoin/bitcoin#22258: build: Disable deprecated-copy warning only when external warnings are enabled
1111457d74 build: Disable deprecated-copy warning only when external warnings are enabled (MarcoFalke)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/18967

  Alternative to https://github.com/bitcoin/bitcoin/pull/22240

ACKs for top commit:
  fanquake:
    tACK 1111457d74

Tree-SHA512: 0fc826f26ebbeab662fa7eed2a5cc1630c6c4e612deb91734885fc8bae0352be657ec48ae94ff55a984ac36d27b95cea8d947cc5cf408231d56addecf79db83f
2021-06-17 11:44:18 +08:00
sanket1729
55311197c4 Added new test for future blocks reacceptance
Adds a test case for checking reacceptance a previously rejected block
that was too far in the future.
2021-06-16 16:43:20 -07:00
sanket1729
511a5af462 Fixed inconsistencies between code and comments
1) Makes the code for block 44 consistent with  the expected figure in
the comment above it by adding a transaction to the block
2) Fixed comment describing sign_tx() function
2021-06-16 16:43:20 -07:00
MarcoFalke
1111457d74
build: Disable deprecated-copy warning only when external warnings are enabled 2021-06-16 15:44:27 +02:00
Sjors Provoost
2f5bdcbc31
gui: misc external signer fixes and translation hints 2021-06-16 10:48:58 +02:00
Sjors Provoost
d672404466
refactor: make ExternalSigner NetworkArg() and m_chain private 2021-06-16 10:48:58 +02:00
Sjors Provoost
4455145e26
refactor: reduce #ifdef ENABLE_EXTERNAL_SIGNER usage
In particular this make the node interface independent on whether external signer support is compiled.
2021-06-16 10:48:58 +02:00
Sjors Provoost
5be90c907e
build: enable external signer by default 2021-06-16 10:48:57 +02:00
Sjors Provoost
7d9453041b
refactor: clean up external_signer.h includes
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-06-16 10:48:38 +02:00
Sjors Provoost
fc0eca31b3
fuzz: fix fuzz binary linking order
We encountered a linking error when attempting to include external_signer_scriptpubkeyman.cpp when configured with --disable-external-signer.

Everywhere else we have LIBBITCOIN_WALLET, it is always before LIBBITCOIN_COMMON. But if you go up to where FUZZ_SUITE_LD_COMMON is first set, you see that we will end up having LIBBITCOIN_COMMON set before LIBBITCOIN_WALLET which means that the linker will have problems linking things common things that the wallet uses. Because the order is correct for the other targets, we only see a linker error for test/fuzz/fuzz.

In this diff, LIBTEST_UTIL and LIBTEST_FUZZ are moved to the top because they include LIBBITCOIN_SERVER and LIBBITCOIN_COMMON. LIBBITCOIN_SERVER always needs to be the first item in the linker order since it has the most dependencies.

The makefiles for making the fuzz and test binaries should be revisited so that the linking order is made consistent with the rest of the code and to avoid other linker order issues that may crop up in the future.

Co-Authored-By: Andrew Chow <achow101-github@achow101.com>
2021-06-16 10:41:24 +02:00
fanquake
6bc1eca01b
Merge bitcoin/bitcoin#22144: Randomize message processing peer order
79c02c88b3 Randomize message processing peer order (Pieter Wuille)

Pull request description:

  Right now, the message handling loop iterates the list of nodes always in the same order: the order they were connected in (see the `vNodes` vector). For some parts of the net processing logic, this order matters. Transaction requests are assigned explicitly to peers since #19988, but many other parts of processing work on a "first-served-by-loop-first" basis, such as block downloading. If peers can predict this ordering, it may be exploited to cause delays.

  As there isn't anything particularly optimal about the current ordering, just make it unpredictable by randomizing.

  Reported by Crypt-iQ.

ACKs for top commit:
  jnewbery:
    ACK 79c02c88b3
  Crypt-iQ:
    ACK 79c02c88b3
  sdaftuar:
    utACK 79c02c88b3
  achow101:
    Code Review ACK 79c02c88b3
  jamesob:
    crACK 79c02c88b3
  jonatack:
    ACK 79c02c88b3
  vasild:
    ACK 79c02c88b3
  theStack:
    ACK 79c02c88b3

Tree-SHA512: 9a87c4dcad47c2d61b76c4f37f59674876b78f33f45943089bf159902a23e12de7a5feae1a73b17cbc3f2e37c980ecf0f7fd86af9e6fa3a68099537a3c82c106
2021-06-16 11:27:16 +08:00
fanquake
8f7704d032
build: improve detection of eBPF support
Just checking for the `sys/sdt.h` header isn't enough, as systems like
macOS have the header, but it doesn't actually have the dtrace probes,
which leads to compile failures.
2021-06-16 10:16:03 +08:00
Luke Dashjr
754e802274
test: check rejected future block later accepted
(Luke) was unsure if the code sufficiently avoided caching a
time-too-new rejection, so wrote this test to check it.  It looks like
despite only exempting BLOCK_MUTATED, it is still okay because header
failures never cache block invalidity.  This test will help ensure that
if this ever changes, BLOCK_TIME_FUTURE gets excluded at the same time.

Co-authored-by: Will Clark <will8clark@gmail.com>
2021-06-15 21:35:29 +01:00
S3RK
451b96f7d2 test: kill process group to avoid dangling processes 2021-06-15 09:37:58 +02:00
fanquake
eb63b1db2c
Merge bitcoin/bitcoin#22247: Switch Appveyor CI to VS2019 stable image
aab7fd0f8d Switch Appveyor CI to VS2019 stable image (Aaron Clauson)

Pull request description:

  The current appveyor config is using the VS2019 preview image so the latest prebuilt Qt5.12.11 binaries can be used, see #22224.

  Appveyor updated the Visual Studio 2019 image to msbuild v16.10.1 on 14th of June. This is the version used to build the latest Qt binaries and removes the need to use the Appveyor VS2019 preview image.

ACKs for top commit:
  MarcoFalke:
    review ACK aab7fd0f8d if green
  practicalswift:
    cr ACK aab7fd0f8d: patch looks correct
  hebasto:
    ACK aab7fd0f8d

Tree-SHA512: 42ea4e6e27a2099ddeed99a3352e8ff014df1e93fbcb1f0f2ebd6f22ec2fb71212275a2adb2a7858516203e566c6ba053367b4ac7e0b74457f7a35e941d6fdd4
2021-06-15 09:25:58 +08:00
Hennadii Stepanov
3f68f02db9
Merge bitcoin-core/gui#362: Add keyboard shortcuts to context menus
e4c916a0ea Bugfix: GUI: Use a different shortcut for "1 d&ay" banning, due to conflict with "&Disconnect" (Luke Dashjr)
94e7cdd7e0 GUI: Add keyboard shortcuts for other context menus (Luke Dashjr)
02b5263cd4 GUI: Restore keyboard shortcuts for context menu entries (Luke Dashjr)

Pull request description:

  Various keyboard shortcuts were lost in #263; this restores them, and also adds new ones for other context menus.

  Note that with a context menu open, simply the shortcut by itself (no Alt) is used.

ACKs for top commit:
  jarolrod:
    Code Review ACK e4c916a
  hebasto:
    ACK e4c916a0ea, tested on Linux Mint 20.1 (Qt 5.12.8).

Tree-SHA512: 949461acf7aac592bc48a1c5abad41b167365830e0cedb3aa11b6a87bd347e16126830ea87936f9c9efc4b7df5b09d3833fae784964d6d119ed45703cfba2ffd
2021-06-15 00:57:18 +03:00
Carl Dong
e2c40a4ed5 guix-attest: Error out if SHA256SUMS is unexpected 2021-06-14 17:05:26 -04:00
Aaron Clauson
aab7fd0f8d
Switch Appveyor CI to VS2019 stable image
The current appveyor config is using the VS2019 preview image so the latest prebuilt Qt5.12.11 binaries can be used, see #22224.

Appveyor updated the Visual Studio 2019 image to msbuild v16.10.1 on 14th of June. This is the version used to build the latest Qt binaries and removes the need to use the Appveyor VS2019 preview image.
2021-06-14 20:35:00 +01:00
W. J. van der Laan
a33381acf5 devtools: Add xkb version to symbol-check
xkb versions symbols (using the prefix `V`), as this library is used by
bitcoin-qt, add it to the valid versions in `symbol-check.py`.
2021-06-14 20:32:09 +02:00
W. J. van der Laan
19e598bab0 devtools: Fix verneed section parsing in pixie
I misunderstood the ELF specification for version symbols (verneed):
The `vn_aux` pointer is relative to the main verneed record, not the
start of the section.

This caused many symbols to not be versioned properly in the return
value of `elf.dyn_symbols`. This was discovered in #21454.

Fix it by correcting the offset computation.
2021-06-14 20:31:58 +02:00
MarcoFalke
3a2c84a6b5
Merge bitcoin/bitcoin#19238: refactor: Make CAddrMan::cs non-recursive
ae98aec9c0 refactor: Make CAddrMan::cs non-recursive (Hennadii Stepanov)
f5d1c7fac7 Add AssertLockHeld to CAddrMan private functions (Hennadii Stepanov)
5ef1d0b698 Add thread safety annotations to CAddrMan public functions (Hennadii Stepanov)
b138973a8b refactor: Avoid recursive locking in CAddrMan::Clear (Hennadii Stepanov)
f79a664314 refactor: Apply consistent pattern for CAddrMan::Check usage (Hennadii Stepanov)
187b7d2bb3 refactor: Avoid recursive locking in CAddrMan::Check (Hennadii Stepanov)
f77d9c79aa refactor: Fix CAddrMan::Check style (Hennadii Stepanov)
06703973c7 Make CAddrMan::Check private (Hennadii Stepanov)
efc6fac951 refactor: Avoid recursive locking in CAddrMan::size (Hennadii Stepanov)
2da95545ea test: Drop excessive locking in CAddrManTest::SimConnFail (Hennadii Stepanov)

Pull request description:

  This PR replaces `RecursiveMutex CAddrMan::cs` with `Mutex CAddrMan::cs`.

  All of the related code branches are covered by appropriate lock assertions to insure that the mutex locking policy has not been changed by accident.

  Related to #19303.

  Based on #22025, and first three commits belong to it.

ACKs for top commit:
  vasild:
    ACK ae98aec9c0

Tree-SHA512: c3a2d3d955a75befd7e497a802b8c10730e393be9111ca263ad0464d32fae6c7edf9bd173ffb6bc9bb61c4b39073a74eba12979d47f26b0b7b4a861d100942df
2021-06-14 16:41:14 +02:00
Hennadii Stepanov
ae98aec9c0
refactor: Make CAddrMan::cs non-recursive 2021-06-14 17:28:38 +03:00
Hennadii Stepanov
f5d1c7fac7
Add AssertLockHeld to CAddrMan private functions 2021-06-14 17:28:38 +03:00
Hennadii Stepanov
5ef1d0b698
Add thread safety annotations to CAddrMan public functions 2021-06-14 17:28:38 +03:00
Hennadii Stepanov
b138973a8b
refactor: Avoid recursive locking in CAddrMan::Clear
Co-authored-by: John Newbery <john@johnnewbery.com>
2021-06-14 17:28:37 +03:00