Commit Graph

28984 Commits

Author SHA1 Message Date
W. J. van der Laan
00a9b0647e
Merge bitcoin/bitcoin#21799: guix: Use gcc-8 across the board
c90f6e5109 guix: Consistently use gcc-8 for $HOST (Carl Dong)

Pull request description:

  Only non-base commit is the last commit: b5abb07d0d

  Right now, here's what we use in Gitian:
  - Linux: Focal's [`g++-8-<arch>-linux-gnu`](https://packages.ubuntu.com/focal/g++-8-aarch64-linux-gnu) (`8.4.0-3ubuntu1cross1`)
  - MinGW-w64: Focal's [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64) (`9.3.0-7ubuntu1+22~exp1ubuntu4`)

  In Guix right now we use `gcc-9` across the board.

  I think it makes more sense to use `gcc-8` across the board, as it doesn't suffer from the `memcmp` bug, and is what debian buster (stable) does, meaning it will be well tested ([`g++-mingw-w64`](https://packages.debian.org/buster/g++-mingw-w64), [`g++-aarch64-linux-gnu`](https://packages.debian.org/buster/g++-aarch64-linux-gnu)).

  We can accomplish this somewhat easily using Guix as we have tighter control over the toolchain (see: b5abb07d0d).

  Let me know your thoughts!

ACKs for top commit:
  MarcoFalke:
    Approach ACK c90f6e5109, haven't reviewed
  laanwj:
    Code review ACK c90f6e5109
  hebasto:
    ACK c90f6e5109, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 3e5b9297305232273323aa745ec417ed1be2418ead0e432db7742f5d5f45efe6e4a2ed44328731512cff4bfde80e5f2dc350a131b8b8fb9207a2ef66bce27ed2
2021-05-05 10:43:09 +02:00
MarcoFalke
779aaa7f03
Merge bitcoin/bitcoin#21754: test: Run feature_cltv with MiniWallet
fa066f1b66 test: Run feature_cltv with MiniWallet (MarcoFalke)
fa5591d38c test: Hide tx rehash in helper (MarcoFalke)
fa5f938cfe test: Remove new_tx reference (MarcoFalke)

Pull request description:

  Allows to run the test even with no wallet compiled in

ACKs for top commit:
  theStack:
    ACK fa066f1b66 💽

Tree-SHA512: 3f659a178ba3ee0baffd70fddf8b8a68e5551d85626c7f254b234d7f75e6a16430a32a7952037db358b579f045b4d296b46156f72e5d226f3e80334dc635ca10
2021-05-05 08:47:28 +02:00
fanquake
dc8da2a685
Merge bitcoin/bitcoin#21824: refactor: [index] Replace deprecated char with uint8_t in serialization
fafb880e88 refactor: [index] Replace deprecated char with uint8_t in serialization (MarcoFalke)

Pull request description:

  All char representations are serialized in the same way, however the `char` one is deprecated according to d22e7ee933/src/serialize.h (L227) . Also, using `uint8_t` directly avoids casts.

ACKs for top commit:
  jonatack:
    Approach ACK fafb880e88
  laanwj:
    Code review ACK fafb880e88
  practicalswift:
    cr ACK fafb880e88: patch looks correct

Tree-SHA512: ed08fb1b18cb75a695e15924bcaa30ff8746bcd5f17cc83e79f94fe5ff8d9f2083435cb49b8245e3341ede2512140940d864299f4746bc40c8ed8bfdbdacac24
2021-05-05 09:59:46 +08:00
MarcoFalke
3f8f238deb
Merge bitcoin/bitcoin#21849: fuzz: Limit toxic test globals to their respective scope
cf83b82cf0 fuzz: Limit toxic test globals to their respective scope (MarcoFalke)

Pull request description:

  Globals in one fuzz target are toxic to all other fuzz targets, because we link all fuzz targets into one binary. Any code called by constructing the global will affect all other targets. This leads to incorrect coverage stats, false-positive crashes, ...

ACKs for top commit:
  practicalswift:
    cr ACK cf83b82cf0: non-toxic is better than toxic!
  laanwj:
    Code review ACK cf83b82cf0

Tree-SHA512: 5b3a37bcb36fce4160c94f877b2c07704527e3e1842092375c793d2eca77b996ae62889326094020855666bb34fa019fcfe92e8ff8430ce0372227f03ab2b907
2021-05-04 17:21:43 +02:00
MarcoFalke
a1c6434e19
Merge bitcoin/bitcoin#21852: ci: Add msan fuzz config
fa0422c251 ci: Add msan fuzz config (MarcoFalke)
fa399a76c6 ci: Use clang-12 in msan task (MarcoFalke)
fab30174af ci: Set BASE_SCRATCH_DIR early, so that it can be used in test configs (MarcoFalke)

Pull request description:

  Similar to the valgrind config, this config is not run by any ci task in this repo, but it can be used by other repos or self-hosted infrastructure.

ACKs for top commit:
  practicalswift:
    cr ACK fa0422c251: patch looks correct

Tree-SHA512: 2122ac0948978a7b952efc80d4aa3674b27d48c6166e0ce917c61ac4ee6b68d701a83e5f71ee6868c208885ee45aae409ca022ebcb23ccbe37819a8c36e34872
2021-05-04 17:04:49 +02:00
MarcoFalke
fa0422c251
ci: Add msan fuzz config 2021-05-04 15:06:00 +02:00
MarcoFalke
fa399a76c6
ci: Use clang-12 in msan task 2021-05-04 15:04:11 +02:00
MarcoFalke
fab30174af
ci: Set BASE_SCRATCH_DIR early, so that it can be used in test configs
Can be reviewed with --color-moved=dimmed-zebra
2021-05-04 15:03:14 +02:00
fanquake
7fc5e865b9
test: install lief in CI 2021-05-04 20:48:53 +08:00
fanquake
955140b326
contrib: consolidate PIE and NX security checks 2021-05-04 20:48:00 +08:00
fanquake
2aa1631822
contrib: use LIEF in PE symbol checks 2021-05-04 20:48:00 +08:00
fanquake
e93ac26b85
contrib: use LIEF in macOS symbol checks 2021-05-04 20:48:00 +08:00
fanquake
a632cbcee5
contrib: use f strings in symbol-check.py 2021-05-04 20:48:00 +08:00
fanquake
0f5d77c8e4
contrib: add PE PIE check to security checks 2021-05-04 20:48:00 +08:00
fanquake
8e1f40dd9a
contrib: use LIEF for PE security checks 2021-05-04 20:47:59 +08:00
fanquake
a25b2e965c
contrib: use LIEF for macOS security checks 2021-05-04 20:47:59 +08:00
fanquake
7e7eae7aa8
contrib: use f strings in security-check.py 2021-05-04 20:47:59 +08:00
fanquake
2e7a9f7ade
guix: install LIEF in Guix container
Co-authored-by: Carl Dong <contact@carldong.me>
2021-05-04 20:47:59 +08:00
fanquake
465967b5ef
gitian: install LIEF in gitian container 2021-05-04 20:47:58 +08:00
fanquake
0ca8b7e7ec
Merge bitcoin/bitcoin#21812: ci: Enable D_GLIBCXX_DEBUG for multiprocess task
fa44f5119a ci: Clarify that previous_releases task is using DEBUG (MarcoFalke)
fad0f21c3c ci: Use clang in multiprocess task to avoid OOM (MarcoFalke)
faeabef4f3 ci: Enable D_GLIBCXX_DEBUG for multiprocess task (MarcoFalke)

Pull request description:

  Enable `-D_GLIBCXX_DEBUG` via the depends `DEBUG` flag. Also `--enable-debug` to get debug symbols in traces.

ACKs for top commit:
  hebasto:
    ACK fa44f5119a, I have reviewed the code and it looks OK, I agree it can be merged, and CI is green.

Tree-SHA512: ab2a216bb44ee462f9dd181ec9025962502bd4201a1118ff52b6a193398e7ea3ca465a45a5eb341e308758fc3ef34ea3521f8a1f85ed64478ef3c1f6c1b8b141
2021-05-04 19:18:35 +08:00
John Newbery
39e19713cd [net processing] Add internal _RelayTransactions()
Callers of the external RelayTransactions() no longer need to lock cs_main.
2021-05-04 09:31:03 +01:00
W. J. van der Laan
ab9a566ab3
Merge bitcoin/bitcoin#21825: net: add I2P hardcoded seeds
142e2da440 net: add I2P seeds to chainparamsseeds (Jon Atack)
e01f173fb9 contrib: add a few I2P seed nodes (Jon Atack)
ea269c7ef1 contrib: parse I2P addresses in generate-seeds.py (Jon Atack)

Pull request description:

  Follow-up to #21560 that updated the fixed seeds infra for BIP155 addresses and then added Tor v3 ones:

  - Update contrib/generate-seeds.py to parse I2P addresses

  - Add a few I2P nodes to contrib/seeds/nodes_main.txt

  - Run generate-seeds.py and add the I2P seeds to chainparamsseeds.h

  Reviewers, see contrib/seeds/README.md for more info and feel free to use the following CLI one-liner to check for and propose additional seeds for contrib/seeds/nodes_main.txt. You can also see how many I2P peers your node knows with cli -addrinfo.

  ```rake
  bitcoin-cli getnodeaddresses 0 | jq '.[] | (select(.address | contains(".b32.i2p"))) | .address' | sort
  ```

  I verified the I2P addresses are correctly BIP155-serialized/deserialized by building with all seeds removed from chainparamsseeds.h except those added here, restarting with `-datadir=newdir -dnsseed=0` and running rpc ` getnodeaddresses 0` that initially returns only the new I2P addresses.

ACKs for top commit:
  laanwj:
    ACK 142e2da440
  vasild:
    ACK 142e2da440

Tree-SHA512: 040576012d5f1f034e2bd566ad654a6fdfd8ff7f6b12fa40c9fda1e948ebf8417fcea64cfc14938a41439370aa4669bab3e97274f9d4f9a6906fa9520afa9cf8
2021-05-04 09:44:46 +02:00
MarcoFalke
cf83b82cf0
fuzz: Limit toxic test globals to their respective scope 2021-05-04 09:24:17 +02:00
MarcoFalke
fac30eec42
refactor: Replace &foo[0] with foo.data() 2021-05-04 06:55:31 +02:00
MarcoFalke
faece47c47
refactor: Avoid &foo[0] on C-Style arrays
This is confusing at best when parts of a class use the
redundant operators and other parts do not.
2021-05-04 06:55:30 +02:00
MarcoFalke
face961109
refactor: Use only one temporary buffer in CreateObfuscateKey 2021-05-04 06:53:37 +02:00
MarcoFalke
fa05dddc42
refactor: Use CPubKey vector constructor where possible 2021-05-04 06:53:32 +02:00
Guido Vranken
fabb6dfe6e
script: Replace address-of idiom with vector data() method 2021-05-04 06:53:20 +02:00
MarcoFalke
e2d4e67a8f
Merge bitcoin/bitcoin#21840: test: Misc refactor to get rid of &foo[0] raw pointers
fa8a88849c bench: Remove duplicate constants (MarcoFalke)
000098f964 test: Use throwing variant accessor (MarcoFalke)
fa2197c8b3 test: Use loop to register RPCs (MarcoFalke)

Pull request description:

  Simplify test code

ACKs for top commit:
  Empact:
    Code Review ACK fa8a88849c
  practicalswift:
    cr ACK fa8a88849c
  promag:
    Code review ACK fa8a88849c.

Tree-SHA512: 6a5bebaa9a3f43e9c332f4fbff606e9ece6dc8b95a769980082cc022f8e9bde6083c1e4a0145dcbf3741f514d6e97b4198f201a1bf1370ebf43bd3a5c0f85981
2021-05-04 06:49:22 +02:00
Carl Dong
d420e5c1c0 guix-attest: Avoid incomplete sigdirs with ERR traps
Sometimes GPG connects to the wrong agent... or you don't have your
smartcard handy...
2021-05-03 16:50:59 -04:00
MarcoFalke
bf5e6a7771
Merge bitcoin/bitcoin#21846: fuzz: Add -fsanitize=integer suppression needed for RPC fuzzer (generateblock)
575792e6ff fuzz: Add -fsanitize=integer suppression needed for RPC fuzzer (practicalswift)

Pull request description:

  Add `-fsanitize=integer` suppression needed for RPC fuzzer (`generateblock`).

  Context: https://github.com/bitcoin-core/qa-assets/pull/59/checks?check_run_id=2494624259

  ```
  miner.cpp:130:21: runtime error: implicit conversion from type 'int64_t' (aka 'long') of value 244763573890 (64-bit, signed) to type 'uint32_t' (aka 'unsigned int') changed the value to 4245405314 (32-bit, unsigned)
      #0 0x56143974eaf3 in BlockAssembler::CreateNewBlock(CScript const&) miner.cpp:130:21
      #1 0x56143993690d in generateblock()::$_4::operator()(RPCHelpMan const&, JSONRPCRequest const&) const rpc/mining.cpp:370:127
  ```

ACKs for top commit:
  practicalswift:
    > review ACK [575792e](575792e6ff), but this function shouldn't be called by the rpc fuzzer, at least not without sanitizing num_blocks
  MarcoFalke:
    review ACK 575792e6ff

Tree-SHA512: c2133d1064bf17df0e7749ef4a0f7664b5c8082040491a1035d39f0c6e5d96997b347ef2354411e285c7f1f973e34515f1c3c88eb3de60fab64ca4d2adf6dd74
2021-05-03 22:18:05 +02:00
practicalswift
575792e6ff fuzz: Add -fsanitize=integer suppression needed for RPC fuzzer 2021-05-03 19:33:46 +00:00
Carl Dong
c90f6e5109 guix: Consistently use gcc-8 for $HOST 2021-05-03 14:42:25 -04:00
MarcoFalke
8e1c06846e
Merge bitcoin/bitcoin#21662: build: update Boost download URL
36c10b9f4b build,boost: update download url. (fdov)

Pull request description:

  - bintray is closing.
      - updated to jfrog.io.

  For reference:
  https://github.com/boostorg/boost/issues/502

ACKs for top commit:
  Sjors:
    ACK 36c10b9
  hebasto:
    ACK 36c10b9f4b

Tree-SHA512: 6746781296c3f395a2824750cf1a578ef9a2d6b8d5c4da196a8da5d084aea01f7f2583b41b2aade7bb94dd02be71efcfd6edf7491a1a25f2d36240b18322a68e
2021-05-03 19:57:43 +02:00
MarcoFalke
ea71726a54
Merge bitcoin/bitcoin#21810: fuzz: Various RPC fuzzer follow-ups
5252f86eb6 fuzz: Reduce maintenance requirements by allowing RPC annotations also for conditionally available RPC commands (such as wallet commands) without the fragility of #ifdef forests (practicalswift)
54549dda31 fuzz: RPC fuzzer post-merge follow-ups. Remove unused includes. Update list of fuzzed RPC commands. (practicalswift)

Pull request description:

  Various RPC fuzzer follow-ups:
  * Remove unused includes.
  * Update list of fuzzed RPC commands.
  * Reduce maintenance requirements by allowing RPC annotations also for conditionally available RPC commands (such as wallet commands) without the fragility of `#ifdef` forests.

  Context: https://github.com/bitcoin/bitcoin/pull/21169#pullrequestreview-646723483

ACKs for top commit:
  MarcoFalke:
    Concept ACK 5252f86eb6

Tree-SHA512: 286d70798131706ffb157758e1c73f7f00ed96ce120c7d9dc849e672b283f1362df47b206cfec9da44d5debb5869225e721761dcd5c38a7d5d1019dc6c912ab2
2021-05-03 19:47:18 +02:00
Carl Dong
feda2c8e31 guix: Skip attesting to dist-archive
We already attest to the relevant dist-archive in inputs.SHA256SUMS,
which is recorded at build-time.

We use a SKIPATTEST.TAG file to indicate output directories which do not
require attestation (much like the CACHEDIR.TAG specification).
Generally, it's better to have build scripts declare properties of
directories instead of introducing name-based special cases in attest
scripts since build scripts have a more detailed context of what is
going on.
2021-05-03 13:18:19 -04:00
Carl Dong
d522d8006b guix: Attest to inputs in inputs.SHA256SUMS
At build/codesigning-time, hash build inputs and output the digest to
${OUTDIR}/inputs.SHA256SUMS, which gets included in the final SHA256SUMS
constructed by guix-attest.

Example final SHA256SUMS:
ee832d2a35b7701bff581dea05a536118b118e3ad0a587a2855b6ee8cd6fba20  inputs/bitcoin-78199266af7b.tar.gz
ca765e70a0c12866dd63c0be228b675278a26329e5f8f5b5c52fd09200fedf21  bitcoin-78199266af7b-powerpc64le-linux-gnu-debug.tar.gz
dae95327d7f2c324e2728c4b73627be6cb2c0d2f2e5bea940d1d5e6463939327  bitcoin-78199266af7b-powerpc64le-linux-gnu.tar.gz
2021-05-03 13:18:19 -04:00
Carl Dong
f9e2960c01 guix: Construct $OUTDIR in ${DISTSRC}/output
While files are being output to $OUTDIR, it will be under
${DISTSRC}/output, and only when everything is done, will
${DISTSRC}/output be moved to the actual $OUTDIR.

This makes it so that a Ctrl-C in the middle of a build is less likely
to result in a partially-constructed $OUTDIR. In fact, if I understand
correctly, if $OUTDIR and $DISTSRC reside on the same filesystem, the
move (rename) is likely atomic.

Also, since the "working $OUTDIR" is under ${DISTSRC}/output, it will be
cleaned properly by the guix-clean script.
2021-05-03 13:18:19 -04:00
Carl Dong
022abc85fc guix: Minor quoting fix in libexec/build.sh 2021-05-03 13:18:19 -04:00
Carl Dong
c83c4fa5b7 guix-attest: Allow skipping GPG signing with NO_SIGN 2021-05-03 13:18:19 -04:00
Carl Dong
0e1c2e448c guix-attest: Use ascii-armor signatures 2021-05-03 13:18:19 -04:00
Carl Dong
b5fd89c4c8 guix-attest: Only use cross-platform flags for find+xargs 2021-05-03 13:18:19 -04:00
Carl Dong
5926432ba6 guix: Add guix-verify script 2021-05-03 13:18:19 -04:00
Carl Dong
30daf76a97 guix: Add guix-attest script 2021-05-03 13:18:19 -04:00
MarcoFalke
3692097d92
Merge bitcoin/bitcoin#21823: script: update REVIEWERS
48bd9ebc5d script: update REVIEWERS (Adam Jonas)

Pull request description:

  Meta: `git show a06eb03` indicates the commit was first made one year ago and the PR was merged in September 2020.

  It might be time for an update, if automated review requests via DrahtBot are operational.

  "Regular contributors are free to add their names to specific directories or files provided that they are willing to provide a review."

  Perhaps we can compile and squash suggested updates here, per that guideline.

ACKs for top commit:
  practicalswift:
    ACK 48bd9ebc5d: patch looks correct :)
  adamjonas:
    ACK 48bd9eb. I ran through the new files added with `*`s using `git ls-files`.
  hebasto:
    ACK 48bd9ebc5d, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 770fd60d78bd64100814c72211e4bd97718c87a55a4c91e136fababa0149d12cdc1b3e89462dee3a55e5f51d3263420cc6d856868469eb9a72ff8c930859c54f
2021-05-03 19:11:40 +02:00
Adam Jonas
48bd9ebc5d
script: update REVIEWERS 2021-05-03 13:16:43 +02:00
MarcoFalke
fa8a88849c
bench: Remove duplicate constants 2021-05-03 11:50:30 +02:00
MarcoFalke
000098f964
test: Use throwing variant accessor
It does not matter if the tests fail due to a BOOST_CHECK failure or
due to a thrown exception. Prefer the exception because it is less
code.

Example fail with the throwing accessor:

unknown location(0): fatal error: in "script_standard_tests/script_standard_ExtractDestinations": std::bad_variant_access: std::get: wrong index for variant
test/script_standard_tests.cpp(314): last checkpoint

*** 1 failure is detected in the test module "Bitcoin Core Test Suite"
2021-05-03 11:50:29 +02:00
MarcoFalke
fa2197c8b3
test: Use loop to register RPCs
The same loop is used by the server, so no need for
the tests to do this differently.
2021-05-03 11:50:05 +02:00
MarcoFalke
b8593616dc
Merge bitcoin/bitcoin#21775: p2p: Limit m_block_inv_mutex
fac96d0265 p2p: Limit m_block_inv_mutex (MarcoFalke)

Pull request description:

  Keeping the lock longer than needed is confusing to reviewers and thread analysis. For example, keeping the lock while appending tx-invs, which requires the mempool lock, will tell thread analysis tools an incorrect lock order of `(1) m_block_inv_mutex, (2) pool.cs`.

ACKs for top commit:
  Crypt-iQ:
    crACK fac96d0265
  jnewbery:
    utACK fac96d0265
  theStack:
    Code-Review ACK fac96d0265

Tree-SHA512: fcfac0f1f8b16df7522513abf716b2eed3d2fc9153f231c8cb61f451e342f29c984a5c872deca6bab3e601e5d651874cc229146c9370e46811b4520747a21f2b
2021-05-03 11:13:41 +02:00