Commit Graph

27256 Commits

Author SHA1 Message Date
Carl Dong
f1694757dd guix: Fix typo 2021-01-21 10:58:12 -05:00
Carl Dong
771c4b98a8 guix: README: Add darwin HOSTS entry 2021-01-21 10:58:12 -05:00
Carl Dong
8dbf18cb1d guix: Check for macOS SDK before building anything 2021-01-21 10:58:12 -05:00
Carl Dong
34b23f597e guix: Set ZERO_AR_DATE for darwin build determinism
See comments inserted in this commit.
2021-01-21 10:58:12 -05:00
Carl Dong
f3835dc6a3 build: Make xorrisofs reproducible with -volume_date
We need this to be after a '--' as '-volume_date' is a xorriso
flag, not a xorrisofs flag. See the respective man pages.

For more details: https://issues.guix.info/issue/35283#2
2021-01-21 10:58:12 -05:00
Carl Dong
c9eb4cf3a0 guix: Add support for darwin builds 2021-01-21 10:58:08 -05:00
Carl Dong
37fe73a092 build: Add var printing target to src/Makefile.am
See 181989f6c9 for more info. I missed
this one last time.
2021-01-21 10:57:31 -05:00
MarcoFalke
1f45e85509
Merge #20972: locks: Annotate CTxMemPool::check to require cs_main
b396467053 locks: Annotate CTxMemPool::check to require cs_main (Carl Dong)

Pull request description:

  ```
  Currently, CTxMemPool::check locks CTxMemPool's own cs member, then
  calls GetSpendHeight which locks cs_main. This can potentially cause an
  undesirable lock invesion since CTxMemPool's cs is supposed to be locked
  after cs_main.

  This does not cause us any problems right now because all callers of
  CTxMemPool already lock cs_main before calling CTxMemPool::check, which
  means that the LOCK(cs_main) in GetSpendHeight becomes benign.

  However, it is currently possible for new code to be added which calls
  CTxMemPool::check without locking cs_main (which would be dangerous).
  Therefore we should make it explicit that cs_main needs to be held
  before calling CTxMemPool::check.

  NOTE: After all review-only assertions are removed in "#20158 |
        tree-wide: De-globalize ChainstateManager", and assuming that we
        keep the changes in "validation: Pass in spendheight to
        CTxMemPool::check", we can re-evaluate to see if this annotation
        is still necessary.
  ```
  -----

  Previous discussions:
  1. https://github.com/bitcoin/bitcoin/pull/20158#discussion_r520639845
  2. https://github.com/bitcoin/bitcoin/pull/20158#pullrequestreview-557117202
  3. https://github.com/bitcoin/bitcoin/pull/20749#discussion_r559425521

ACKs for top commit:
  jnewbery:
    Code review ACK b396467053
  MarcoFalke:
    ACK b396467053
  jonatack:
    ACK b396467053 review and debug built, verified that `cs_main` is held by callers of `CTxMemPool::check()` in `PeerManagerImpl::ProcessOrphanTx()`, `PeerManagerImpl::ProcessMessage()`, and `CChainState::ActivateBestChainStep()`

Tree-SHA512: 4635cddb4aa1af9532bb657b2f9c4deec4568d16ba28c574eae91bb77368cd40e23c3c720a9de11cec78e7ad678a44a5e25af67f13214b86b56e777e0c35a026
2021-01-21 16:45:45 +01:00
MarcoFalke
85fee49c39
Merge #20946: fuzz: Consolidate fuzzing TestingSetup initialization
abb6fa7285 fuzz: Initialize a full TestingSetup where appropriate (Carl Dong)
713314abfa fuzz: Consolidate fuzzing TestingSetup initialization (Carl Dong)

Pull request description:

  ```
  Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways:

  1. Calling InitializeFuzzingContext, which implicitly constructs a static
     const BasicTestingSetup
  2. Directly constructing a static const BasicTestingSetup in the initialize_*
     function
  3. Directly constructing a static TestingSetup and reproducing the
     initialization arguments (I'm assuming because
     InitializeFuzzingContext only initializes a BasicTestingSetup)

  The new, relatively-simple MakeFuzzingContext function allows us to
  consolidate these methods of initialization by being flexible enough to
  be used in all situations. It:

  1. Is templated so that we can choose to initialize any of
     the *TestingSetup classes
  2. Has sane defaults which are often used in fuzzers but are also
     easily overridable
  3. Returns a unique_ptr, explicitly transferring ownership to the caller
     to deal with according to its situation
  ```

  ~~Question for fuzzing people: was it intentional that `src/test/fuzz/net.cpp` would directly instantiate the `BasicTestingSetup` and thus omit the `"-nodebuglogfile"` flag?~~ [Answered](https://github.com/bitcoin/bitcoin/pull/20946#issuecomment-761537108)

ACKs for top commit:
  MarcoFalke:
    ACK abb6fa7285

Tree-SHA512: 96a5ca6f4cd5ea0e9483b60165b31ae3e9003918c700a7f6ade48010f419f2a6312e10b816b3187f1d263798827571866e4c4ac0bbfb2e0c79dfad254cda68e7
2021-01-21 16:04:31 +01:00
Carl Dong
abb6fa7285 fuzz: Initialize a full TestingSetup where appropriate
A full TestingSetup is required for both coins_view and
load_external_block_file as they interact with the active chainstate.
2021-01-21 09:29:42 -05:00
Carl Dong
713314abfa fuzz: Consolidate fuzzing TestingSetup initialization
Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways:

1. Calling InitializeFuzzingContext, which implicitly constructs a static
   const BasicTestingSetup
2. Directly constructing a static const BasicTestingSetup in the initialize_*
   function
3. Directly constructing a static TestingSetup and reproducing the
   initialization arguments (I'm assuming because
   InitializeFuzzingContext only initializes a BasicTestingSetup)

The new, relatively-simple MakeFuzzingContext function allows us to
consolidate these methods of initialization by being flexible enough to
be used in all situations. It:

1. Is templated so that we can choose to initialize any of
   the *TestingSetup classes
2. Has sane defaults which are often used in fuzzers but are also
   easily overridable
3. Returns a unique_ptr, explicitly transferring ownership to the caller
   to deal with according to its situation
2021-01-21 09:29:42 -05:00
fanquake
3734adba39
Merge #20953: test: dedup zmq test setup code (node restart, topics subscription)
4efb6c2d3b zmq test: deduplicate test setup code (node restart, topics subscription) (Sebastian Falbesoner)

Pull request description:

  This PR deduplicates common setup code for the ZMQ functional test. The following steps, previously duplicated in each sub-test, are put into a new method `setup_zmq_test(...)`:
  - create subscriber sockets (`zmq.SUB`) for each topic with the specified timeout (default 60s)
  - restart node0 with specified zmq notifications enabled (`-zmqpub...=tcp://127.0.0.1:...`...)
  - if desired, connect node0 with node1 (note done by default)
  - connect all susbcriber sockets to publisher (running on node0)
  - wait a bit (currently 200ms), to _"Relax so that the subscribers are ready before publishing zmq messages"_

  Note that the last point should be repaced by a more robust method, as this test is still flaky, see #20934 (also #20590 and #20538).

ACKs for top commit:
  instagibbs:
    ACK 4efb6c2d3b
  laanwj:
    Code review ACK 4efb6c2d3b

Tree-SHA512: d49626756a9c669f1133f1b73ce273994b58c760ce0d6a4bdaa384f043a74149dc2b9fa66fe990413d9105f9c3b6ea973e099669e8e02f2902a5b84fa995028c
2021-01-21 16:38:06 +08:00
Carl Dong
b396467053 locks: Annotate CTxMemPool::check to require cs_main
Currently, CTxMemPool::check locks CTxMemPool's own cs member, then
calls GetSpendHeight which locks cs_main. This can potentially cause an
undesirable lock invesion since CTxMemPool's cs is supposed to be locked
after cs_main.

This does not cause us any problems right now because all callers of
CTxMemPool already lock cs_main before calling CTxMemPool::check, which
means that the LOCK(cs_main) in GetSpendHeight becomes benign.

However, it is currently possible for new code to be added which calls
CTxMemPool::check without locking cs_main (which would be dangerous).
Therefore we should make it explicit that cs_main needs to be held
before calling CTxMemPool::check.

NOTE: After all review-only assertions are removed in "#20158 |
      tree-wide: De-globalize ChainstateManager", and assuming that we
      keep the changes in "validation: Pass in spendheight to
      CTxMemPool::check", we can re-evaluate to see if this annotation
      is still necessary.
2021-01-20 16:15:03 -05:00
Samuel Dobson
80486e7e2d
Merge #20952: wallet: Add BerkeleyDB version sanity check at init time
ad57fb756b wallet: Add BerkeleyDB version sanity check at init time (Wladimir J. van der Laan)

Pull request description:

  Detect version conflicts between the run-time BerkeleyDB library and the one used during compilation.

  This is very unsafe (can result in anything from crashes to corruption) so shut down when one is detected.

ACKs for top commit:
  decryp2kanon:
    utACK ad57fb7
  achow101:
    ACK ad57fb756b
  theStack:
    utACK ad57fb756b
  meshcollider:
    Code review ACK ad57fb756b

Tree-SHA512: 99cd7d836bffbdeb3d4e14053f7139cc85a6d42e631a3f9a3058a848042446b364faee127500f5acb374616e6a61ab2bedebfac1ba9bc993b4d6227114c2a6c2
2021-01-20 16:51:42 +13:00
fanquake
977bec1d93
Merge #20937: guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH
1fca9811e1 lint: Skip whitespace lint for guix patches (Carl Dong)
a91c46c57d guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH (Carl Dong)

Pull request description:

  ```
  When building nsis, if VERSION is not specified, it defaults to
  cvs_version which is non-deterministic as it includes the current date.

  This patches nsis to default to SOURCE_DATE_EPOCH if it exists so that
  nsis is reproducible.

  Upstream change: https://github.com/kichik/nsis/pull/13
  ```

  Sidenote: also a good demonstration of how Guix allows us to flexibly patch our tools!

  Note to reviewers: if you want to compare hashes, please build after Jan 16th 2021 without my substitute server enabled!

ACKs for top commit:
  fanquake:
    ACK 1fca9811e1

Tree-SHA512: b800e0ce5f73827ad353739effb9167ec3a6bdb362c725ae20dd3f025ce78660f85c70ce1d75cd0896facf1e8fe38a9e058459ed13dec71ab3a2fe41e20eaa5d
2021-01-20 07:43:44 +08:00
Wladimir J. van der Laan
bc51b99bd5
Merge #20891: rpc: Remove deprecated bumpfee behavior
ea0a7ec949 Remove deprecated bumpfee behavior (Andrew Chow)

Pull request description:

  Removes the deprecation message, behavior, and test.

  This was marked for removal in 22.0.

ACKs for top commit:
  promag:
    ACK ea0a7ec949, maybe add need release notes tag.

Tree-SHA512: d1626906849f6ee37213c32e5f8c1433ad8fb7beabcd88f5801b1964b322171a2341bdfbd9a3a5ab39b2fd9d9c6a05f73298583423a73cab1275653105c03e8e
2021-01-19 17:33:18 +01:00
Wladimir J. van der Laan
dd545c53a5
Merge #20906: contrib: embed C++11 patch in install_db4.sh
92370033a2 contrib: embed C++11 patch in install_db4.sh (jackielove4u)

Pull request description:

  This is a continuation of https://github.com/bitcoin/bitcoin/pull/20665.

  Closes #20722.

ACKs for top commit:
  laanwj:
    ACK 92370033a2
  fanquake:
    ACK 92370033a2.

Tree-SHA512: ebfd16f5301158de1acc1b8eeca43b3d94f0a6d438832133a30648e5e8a88268b4af983be0bb57f3018e3af8459f32f0de676c1b4e8942e199a4497c776631c5
2021-01-19 17:30:16 +01:00
Wladimir J. van der Laan
43f3ada27b
Merge #19866: eBPF Linux tracepoints
22eb7930a6 tracing: add tracing framework (William Casarin)
933ab8a720 build: detect sys/sdt.h for eBPF tracing (William Casarin)

Pull request description:

  Instead of writing ad-hoc logging everywhere (eg: #19509), we can take advantage of linux user static defined traces, aka. USDTs ( not the stablecoin 😅 )

  The linux kernel can hook into these tracepoints at runtime, but otherwise they have little to no performance impact. Traces can pass data which can be printed externally via tools such as bpftrace. For example, here's one that prints incoming and outgoing network messages:

  # Examples

  ## Network Messages

  ```
  #!/usr/bin/env bpftrace

  BEGIN
  {
    printf("bitcoin net msgs\n");
    @start = nsecs;
  }

  usdt:./src/bitcoind:net:push_message
  {
    $ip = str(arg0);
    $peer_id = (int64)arg1;
    $command = str(arg2);
    $data_len = arg3;
    $data = buf(arg3,arg4);
    $t = (nsecs - @start) / 100000;

    printf("%zu outbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);

    @outbound[$command]++;
  }

  usdt:./src/bitcoind:net:process_message
  {
    $ip = str(arg0);
    $peer_id = (int64)arg1;
    $command = str(arg2);
    $data_len = arg3;
    $data = buf(arg3,arg4);
    $t = (nsecs - @start) / 100000;

    printf("%zu inbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);

    @inbound[$ip, $command]++;
  }

  ```

      $ sudo bpftrace netmsg.bt

  output: https://jb55.com/s/b11312484b601fb3.txt

  if you look at the bottom of the output you can see a histogram of all the messages grouped by message type and IP. nice!

  ## IBD Benchmarking

  ```
  #!/usr/bin/env bpftrace
  BEGIN
  {
    printf("IBD to 500,000 bench\n");
  }

  usdt:./src/bitcoind:CChainState:ConnectBlock
  {
    $height = (uint32)arg0;

    if ($height == 1) {
      printf("block 1 found, starting benchmark\n");
      @start = nsecs;
    }

    if ($height >= 500000) {
      @end = nsecs;
      @duration = @end - @start;
      exit();
    }
  }

  END {
    printf("duration %d ms\n", @duration / 1000000)
  }
  ```
  This one hooks into ConnectBlock and prints the IBD time to height 500,000 starting from the first call to ConnectBlock

  Userspace static tracepoints give lots of flexibility without invasive logging code. It's also more flexible than ad-hoc logging code, allowing you to instrument many different aspects of the system without having to enable per-subsystem logging.

  Other ideas: tracepoints for lock contention, threads, what else?

  Let me know what ya'll think and if this is worth adding to bitcoin.

  ## TODO

  - [ ] docs?
  - [x] Integrate systemtap-std-dev/libsystemtap into build (provides the <sys/sdt.h> header)
  - [x] ~dtrace macos support? (is this still a thing?)~ going to focus on linux for now

ACKs for top commit:
  laanwj:
    Tested ACK 22eb7930a6
  0xB10C:
    Tested ACK 22eb7930a6

Tree-SHA512: 69242242112b679c8a12a22b3bc50252c305894fb3055ae6e13d5f56221d858e58af1d698af55e23b69bdb7abedb5565ac6b45fa5144087b77a17acd04646a75
2021-01-18 22:09:05 +01:00
Wladimir J. van der Laan
f7fd76bcc0
Merge #20880: gitian: Use custom MacOS code signing tool
2c403279e2 gitian: Remove codesign_allocate and pagestuff from MacOS build (Andrew Chow)
f55eed2514 gitian: use signapple to create the MacOS code signature (Andrew Chow)
95b06d2185 gitian: use signapple to apply the MacOS code signature (Andrew Chow)
42bb1ea363 gitian: install signapple in gitian-osx-signer.yml (Andrew Chow)

Pull request description:

  The MacOS code signing issues that were encountered during the 0.21.0 release cycle have shown that it is necessary for us to use a code signing tool for which the source code is available and modifiable by us. Given that there appears to not be such a tool available, I have written such a tool, [signapple](https://github.com/achow101/signapple), that we can use. This tool is able to create a valid MacOS code signature, detach it in a way that we were doing previously, and attach it to the unsigned binary. This tool can also verify that the signature is correct.

  This PR implements the usage of that tool in the gitian build for the code signed MacOS binary. The code signer will use this tool to create the detached signature. Gitian builders will use this tool to apply the detached signature. The `gitian-osx-signer.yml` descriptor has been modified to install this tool so that the `detached-sig-apply.sh` script can use it. Additionally, the `codesign_allocate` and `pagestuff` tools are no longer necessary so they are no longer added to the tarball used in code signing. Lastly, both the `detached-sig-create.sh` and `detached-sig-apply.sh` scripts are made to be significantly less complex and to not do unexpected things such as unpacking an already unpacked tarball.

  The detached code signature that signapple creates is almost identical to that which we were previously creating. The only difference is that the cpu architecture name is included in the extension (e.g. we have `bitcoin-qt.x86_64sign` instead of `bitcoin-qt.sign`). This was done in order to support signing universal binaries which we may want to do in the future. However signapple can still apply existing code signatures as it will accept the `.sign` extension. If it is desired, it can be modified to produce signatures with just the `.sign` extension. However I do not think it is necessary to maintain compatibility with the old process.

ACKs for top commit:
  laanwj:
    Code review ACK 2c403279e2

Tree-SHA512: 2a0e01e9133f8859b9de26e7e8fe1d2610d2cbdee2845e6008b12c083c7e3622cbb2d9b83c50a269e2c3074ab95914a8225d3cd4108017f58b77a62bf10951e0
2021-01-18 22:04:27 +01:00
Wladimir J. van der Laan
ca8218301b
Merge #20884: script: Improve robustness of bitcoind.service on startup
9d02654677 doc: Fix systemd spelling and link to doc/init.md (Hennadii Stepanov)
601778c310 script: Add Documentation key to bitcoind.service (Hennadii Stepanov)
d9392b724c script: Improve robustness of bitcoind.service on startup (Hennadii Stepanov)

Pull request description:

  If network interfaces are not properly up the following happens:
  ```
  ...
  2021-01-08T10:17:11Z scheduler thread start
  2021-01-08T10:17:11Z libevent: getaddrinfo: address family for nodename not supported
  2021-01-08T10:17:11Z Binding RPC on address 127.0.0.1 port 8332 failed.
  2021-01-08T10:17:11Z HTTP: creating work queue of depth 16
  2021-01-08T10:17:11Z Using random cookie authentication.
  2021-01-08T10:17:11Z Generated RPC authentication cookie /var/lib/bitcoind/.cookie
  2021-01-08T10:17:11Z HTTP: starting 2 worker threads
  2021-01-08T10:17:11Z init message: Loading banlist...
  2021-01-08T10:17:11Z SetNetworkActive: true
  2021-01-08T10:17:11Z Error: Cannot resolve -externalip address: <EDITED>
  2021-01-08T10:17:11Z Shutdown: In progress...
  2021-01-08T10:17:11Z scheduler thread exit
  2021-01-08T10:17:11Z Shutdown: done
  ```

  This PR improves robustness on startup in such cases in documented way:
  https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

  Also minor doc improvements are added.

ACKs for top commit:
  Sjors:
    ACK 9d02654
  practicalswift:
    ACK 9d02654677: patch looks correct
  darosior:
    ACK 9d02654677 -- been using the first patch too

Tree-SHA512: 38294f5682c09e6ea9008de7d7459098c920cf1b98ad8ef8a5d2ca01f2f781c0fec5591dc40ef36eeb19d94991b0c7fb7cb38c4e716bc7219875c9bcd0a55e1b
2021-01-18 20:02:00 +01:00
Wladimir J. van der Laan
991e612c3b
Merge #20955: test: Fix get_previous_releases.py for aarch64
fa1d5e5137 test: Fix get_previous_releases.py for aarch64 (MarcoFalke)

Pull request description:

  Otherwise it will fail with "Not sure which binary to download..."

ACKs for top commit:
  laanwj:
    Code review ACK fa1d5e5137

Tree-SHA512: 0db71e898a431665757ce835016a4e05c629a95abc4a2951eac9bd9b5876ec3dc3d6f156d58565e2bcdf918cde4f2649183d4a58038ac13c705a7e914c0094d1
2021-01-18 19:20:53 +01:00
Wladimir J. van der Laan
c763cacb88
Merge #20938: build: fix linking against -latomic when building for riscv
54ce4fac80 build: improve macro for testing -latomic requirement (fanquake)
2c010b9c56 add std::atomic include to bitcoin-util.cpp (fanquake)

Pull request description:

  Since the merge of #19937, riscv builds have been failing, due to a link issue with [`std::atomic_exchange`](https://en.cppreference.com/w/cpp/atomic/atomic_exchange) in `bitcoin-util`:
  ```bash
    CXXLD    bitcoin-util
  bitcoin_util-bitcoin-util.o: In function `grind_task':
  /home/ubuntu/build/bitcoin/distsrc-riscv64-linux-gnu/src/bitcoin-util.cpp:98: undefined reference to `__atomic_exchange_1'
  collect2: error: ld returned 1 exit status

  ```

  We have a [macro](https://github.com/bitcoin/bitcoin/blob/master/build-aux/m4/l_atomic.m4) that tries to determine when `-latomic` is required, however it doesn't quite work well enough, as it's currently determining it isn't needed:
  ```bash
  ./autogen.sh
  ./configure --prefix=/home/ubuntu/bitcoin/depends/riscv64-linux-gnu
  ...
  checking whether std::atomic can be used without link library... yes
  ```

  This PR adds a call to `std::atomic_exchange` to the macro, which will get us properly linked against `-latomic` on riscv:
  ```bash
  checking whether std::atomic can be used without link library... no
  checking whether std::atomic needs -latomic... yes
  ```

  Also adds an `<atomic>` include to `bitcoin-util.cpp`.

ACKs for top commit:
  laanwj:
    Tested ACK 54ce4fac80

Tree-SHA512: 963c875097ee96b131163ae8109bcf8fecf4451d20faa2f3d223f9938ea3d8d1ed5604e12ad82c2b4b1c605fd293a9b6b08fefc00dd3e68d09c49e95029c6f50
2021-01-18 18:33:24 +01:00
MarcoFalke
fa1d5e5137
test: Fix get_previous_releases.py for aarch64 2021-01-18 09:06:24 +01:00
Carl Dong
1fca9811e1 lint: Skip whitespace lint for guix patches 2021-01-17 18:43:58 -05:00
Carl Dong
a91c46c57d guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH
When building nsis, if VERSION is not specified, it defaults to
cvs_version which is non-deterministic as it includes the current date.

This patches nsis to default to SOURCE_DATE_EPOCH if it exists so that
nsis is reproducible.

Upstream change: https://github.com/kichik/nsis/pull/13
2021-01-17 18:43:58 -05:00
Wladimir J. van der Laan
7acda55c4f
Merge #20939: build: fix RELOC_SECTION security check for bitcoin-util
c061800bb1 build: fix RELOC_SECTION security check for bitcoin-util (fanquake)

Pull request description:

  The binutils we use for gitian builds strips the reloc section from
  Windows binaries, which breaks ASLR. As a temporary workaround, export
  main(). This is the same workaround as #18702 (bitcoin-cli), and will
  fix the currently failing security check:
  ```bash
  + make -j1 -C src check-security
  make: Entering directory '/home/ubuntu/build/bitcoin/distsrc-x86_64-w64-mingw32/src'
  Checking binary security...
  bitcoin-util.exe: failed RELOC_SECTION
  make: *** [check-security] Error 1
  ```

  Relevant upstream issue:
  https://sourceware.org/bugzilla/show_bug.cgi?id=19011

ACKs for top commit:
  dongcarl:
    ACK c061800bb1
  laanwj:
    ACK c061800bb1

Tree-SHA512: a1a4da0b2cddfc377190b9044a04f42a859ca79f11ce2c2ab4c3d066a2786c34d5446d75f8eec634f308d2d3349ebbd7c9f76dcaebeeb28e471c829851592367
2021-01-17 18:12:08 +01:00
Wladimir J. van der Laan
ad57fb756b wallet: Add BerkeleyDB version sanity check at init time
Detect version conflicts between the run-time BerkeleyDB library and the one used during compilation.

This is very unsafe (can result in anything from crashes to corruption) so shut down when one is detected.
2021-01-17 18:10:20 +01:00
Sebastian Falbesoner
4efb6c2d3b zmq test: deduplicate test setup code (node restart, topics subscription) 2021-01-17 12:58:46 +01:00
Wladimir J. van der Laan
30e664dcce
Merge #20945: Fix 0.21.0 release note to specify correct option BIP 157 support
9a42b5e655 Fix 0.21.0 release note to specify correct option BIP 157 support (benthecarman)

Pull request description:

  https://github.com/bitcoin/bitcoin/blob/master/src/init.cpp#L452

ACKs for top commit:
  mjdietzx:
    ACK 9a42b5e655
  laanwj:
    ACK 9a42b5e655

Tree-SHA512: 901195434046fd8a18a2329104b69e0ecc1bdb2bba1fe0688fb4a7c5992e9bdd66b8e50f46873f4ddf5ffe3c9aed860ccb01584812b06d865bd06fcd2b14c42b
2021-01-16 11:17:23 +01:00
benthecarman
9a42b5e655
Fix 0.21.0 release note to specify correct option BIP 157 support 2021-01-15 14:05:59 -06:00
MarcoFalke
32e59fc371
Merge #20916: rpc: Return wtxid from testmempoolaccept
fa0aa87071 rpc: Return wtxid from testmempoolaccept (MarcoFalke)

Pull request description:

  It would be nice if `testmempoolaccept` returned the unique wtxid directly to avoid a costly `decoderawtransaction` roundtrip

ACKs for top commit:
  mjdietzx:
    utACK fa0aa87071
  stackman27:
    utACK [`fa0aa87`](fa0aa87071)
  glozow:
    cr ACK fa0aa87071

Tree-SHA512: 05dbaf46d93e47e9eedb725c2f57175e6d4e1722da0531fe4f80e74fc2518911da87634f25f61fa2bc8d87a3017e82fd0684b09a0a9706d71deed970035c2e7a
2021-01-15 20:13:34 +01:00
MarcoFalke
0a1cf6c347
Merge #20908: fuzz: Use mocktime in process_message* fuzz targets
fa0a864b38 fuzz: Use mocktime in process_message* fuzz targets (MarcoFalke)

Pull request description:

  Use mocktime to allow time to advance deterministically during execution of a fuzz input. This also allows to drop the call to `JumpOutOfIbd`.

ACKs for top commit:
  practicalswift:
    cr ACK fa0a864b38

Tree-SHA512: e92fc70ec6bd49760173cb202549f364304e22b3f7127b9a4da8447cf9341008e477ad42c2599c2fde167bbcbc0e2d139709b4ef6371788bc2c1c3b7f589e11d
2021-01-15 19:56:18 +01:00
fanquake
c061800bb1
build: fix RELOC_SECTION security check for bitcoin-util
The binutils we use for gitian builds strips the reloc section from
Windows binaries, which breaks ASLR. As a temporary workaround, export
main(). This is the same workaround as #18702 (bitcoin-cli), and will
fix the currently failing security check:
```bash
+ make -j1 -C src check-security
make: Entering directory '/home/ubuntu/build/bitcoin/distsrc-x86_64-w64-mingw32/src'
Checking binary security...
bitcoin-util.exe: failed RELOC_SECTION
make: *** [check-security] Error 1
```

Relevant upstream issue:
https://sourceware.org/bugzilla/show_bug.cgi?id=19011
2021-01-15 11:53:14 +08:00
fanquake
54ce4fac80
build: improve macro for testing -latomic requirement
riscv builds are currently failing because -latomic isn't being linked
against, when it is needed:
```bash
/home/ubuntu/build/bitcoin/distsrc-riscv64-linux-gnu/src/bitcoin-util.cpp:98: undefined reference to `__atomic_exchange_1'
```

This exteneds our macro to ensure that -latomic is linked against when
required.
2021-01-15 10:54:07 +08:00
fanquake
2c010b9c56
add std::atomic include to bitcoin-util.cpp 2021-01-15 10:40:29 +08:00
fanquake
f91587f050
Merge #20834: locks and docs in ATMP and CheckInputsFromMempoolAndCache
2f463f57e3 [doc] for CheckInputsFromMempoolAndCache (gzhao408)
85cc6bed64 lock annotations for MemPoolAccept functions (gzhao408)

Pull request description:

  This is a very small PR that adds some lock annotations to clarify that, now, the `pool.cs` lock is held throughout tx validation for mempool.  The comments in `CheckInputsFromMempoolAndCache` were unclear/outdated so I updated those as well.

  ~This PR is a cleanup. It removes unnecessary code that doesn't do much.~

ACKs for top commit:
  sdaftuar:
    utACK 2f463f57e3
  jnewbery:
    utACK 2f463f57e3
  MarcoFalke:
    cr ACK 2f463f57e3
  fanquake:
    ACK 2f463f57e3

Tree-SHA512: 5863a546b00ef02eba8f208c2c04c32f64671f17c967a5e3c51332fc0f472e5e9addddae075d0b91c77caebe1be9331317646b0bec802e86d9550773fd9621a9
2021-01-14 23:35:41 +08:00
MarcoFalke
29d2aeb4a2
Merge #20828: fuzz: Introduce CallOneOf helper to replace switch-case
fa75d40ef8 fuzz: Introduce CallOneOf helper to replace switch-case (MarcoFalke)

Pull request description:

  The current `switch (fuzzed_data_provider.ConsumeIntegralInRange<int>(0, nn)) { case 0: ... case 1: ... case nn: ...` has several problems:

  * It makes it hard to review newly added targets, because it requires manual counting of cases
  * It makes it hard to update a target, because updating all case labels is trivial, but tedious to review and causes merge conflicts
  * ~~Updating the target raises the question whether the case labels should be preserved to not invalidate the existing fuzz inputs format. Fuzz input format might already change implicitly on every commit, so this isn't something worthwhile to pursue.~~ Edit: This pull doesn't fix this problem.

  Fix all issues by adding a new `CallOneOf` helper

ACKs for top commit:
  ajtowns:
    ACK fa75d40ef8 - code review only
  jnewbery:
    utACK fa75d40ef8

Tree-SHA512: 2daa602b240b86c8e85a024e008f03a57ba60349377eed771f4d21a97a9dba9b66e93fff16ff1992018d4330be7a1a276944c3dfdf698748ce135626c380e563
2021-01-14 11:07:22 +01:00
Wladimir J. van der Laan
ad571bd354
Merge #20931: doc: Add historic 0.21.0 release notes
faea902721 doc: Add historic 0.21.0 release notes (MarcoFalke)

Pull request description:

  Archive the notes

ACKs for top commit:
  laanwj:
    ACK faea902721

Tree-SHA512: a9b5f4ff9c8273b54c81cc9ece0e90dcb2383ee9971a552b9d5ae5ce77ae29b970a4a4240d22e8e07811cc7e64e78adab841592ab3cd5d729c80233e033baf51
2021-01-14 10:47:10 +01:00
MarcoFalke
faea902721
doc: Add historic 0.21.0 release notes 2021-01-14 10:40:43 +01:00
MarcoFalke
cb2c578451
Merge bitcoin-core/gui#148: Bugfix: GUI: Restore SendConfirmationDialog button default to "Yes"
8775691383 Bugfix: GUI: Restore SendConfirmationDialog button default to "Yes" (Luke Dashjr)

Pull request description:

  The SendConfirmationDialog is used for bumping the fee, where "Send" doesn't really make sense

  Originally https://github.com/bitcoin/bitcoin/pull/17463, but rewritten here much simpler based on other merged changes.

ACKs for top commit:
  hebasto:
    ACK 8775691383, tested on Linux Mint 20.1 (x86_64, Qt 5.12.8):

Tree-SHA512: 3953cc9c09613c9a629def8b4dc061b537f148ddcb378430645602e0be0f3a9f1cff083aa685b94b2e9372300d02ec97e0d9ea89db6e3c6feec86795090f0f77
2021-01-13 17:47:46 +01:00
MarcoFalke
22fa9673b0
Merge #20917: doc, rpc: add missing signet mentions in network name lists
fc726e0138 doc, rpc: add missing signet mentions in network name lists (Sebastian Falbesoner)

Pull request description:

  This small PR adds a few missing mentions of signet w.r.t. chain enumerations:

  - RPC `getblockchaininfo`: result description for `"chain"`
  - RPC `getmininginfo`: result description for `"chain"`
  - REST interface documentation:
      - default ports listing for each chain
      - `"chain"` description for `chaininfo` endpoint result

  The instances were identified via `git grep -i "main.*test.*reg"`.

ACKs for top commit:
  ajtowns:
    ACK fc726e0138 -- quick code review only
  benthecarman:
    ACK fc726e0138

Tree-SHA512: 62cdc6ef74fa10db75cc04b9eaf7367183f726b3fee3d21fdf741b3816669dd21508735e89da389ddac980f49773ab229263748d1399553375fefe4526361846
2021-01-13 17:31:47 +01:00
Wladimir J. van der Laan
e7eb37128c
Merge #20913: doc: Add manual page generation for bitcoin-util
bc99ae77e4 scripted-diff: Fix typo in stub manual pages (Wladimir J. van der Laan)
b5e93f873a doc: Add manual page generation for bitcoin-util (Wladimir J. van der Laan)

Pull request description:

  - Add `-version` option to `bitcoin-util`
  - Add `bitcoin-util` call to `gen-manpages.sh`
  - Add stub manual page `bitcoin-util.1`
  - Add install of `bitcoin-util.1` to build system

ACKs for top commit:
  fanquake:
    ACK bc99ae77e4

Tree-SHA512: 948df66c62bbca1cf6da26845dfa63f8f5d036a3d5744add468dd1ce7f442c123d7b0db7011c2e8e3ee6539fd391c7ee2c21b706ec81b21b02821c9501cd077d
2021-01-13 10:16:09 +01:00
Wladimir J. van der Laan
60427ee35f
Merge #20811: refactor: move net_processing implementation details out of header
c97f70c861 net_processing: move Peer definition to .cpp (Anthony Towns)
e0f2e6d2df net_processing: move PeerManagerImpl into cpp file (Anthony Towns)
a568b82feb net_processing: split PeerManager into interface and implementation classes (Anthony Towns)
0df3d3fd6b net_processing: make more of PeerManager private (Anthony Towns)
0d246a59b6 net, net_processing: move NetEventsInterface method docs to net.h (Anthony Towns)

Pull request description:

  Moves the implementation details of `PeerManager` and all of `struct Peer` into net_processing.cpp.

ACKs for top commit:
  jnewbery:
    ACK c97f70c861
  Sjors:
    ACK c97f70c861
  laanwj:
    Code review ACK c97f70c861
  vasild:
    ACK c97f70c861

Tree-SHA512: 2e081e491d981c61bd78436a6a6c2eb41d3c2d86a1a8ef9d4d6f801b82cb64a8bf707a96db3429418d1704cffb60a657d1037a0336cbc8173fb79aef9eb72001
2021-01-13 09:48:06 +01:00
Wladimir J. van der Laan
8ffaf5c2f5
Merge #19935: Move SaltedHashers to separate file and add some new ones
281fd1a4a0 Replace KeyIDHasher with SaltedSipHasher (Andrew Chow)
210b693db6 Add generic SaltedSipHasher (Andrew Chow)
95e61c1cf2 Move Hashers to util/hasher.{cpp/h} (Andrew Chow)

Pull request description:

  There are existing `SaltedOutPointHasher` and `SaltedTxidHasher` classes used for `std::unordered_map` and `std::unordered_set` that could be useful in other places in the codebase. So we these to their own `saltedhash.{cpp/h}` file. An existing `KeyIDHasher` is moved there too. Additionally, `ScriptIDHasher`, `SaltedPubkeyHasher`, and `SaltedScriptHasher` are added so that they can be used in future work.

  `KeyIDHasher` and `ScriptIDHasher` are not salted so that equality comparisons of maps and sets keyed by `CKeyID` and `CScriptID` will actually work.

  Split from #19602 (and a few other PRs/branches I have).

ACKs for top commit:
  laanwj:
    Code review ACK 281fd1a4a0
  jonatack:
    ACK 281fd1a4a0, code review, debug build and ran bitcoind after rebasing to master @ dff0f6f753
  fjahr:
    utACK 281fd1a4a0

Tree-SHA512: bb03b231ccf3c9ecefc997b8da9c3770af4819f9be5b0a72997a103864e84046a2ac39b8eadf0dc9247bdccd53f86f433642e3a098882e6748341a9e7736271b
2021-01-13 08:49:17 +01:00
MarcoFalke
fa0a864b38
fuzz: Use mocktime in process_message* fuzz targets 2021-01-13 07:48:41 +01:00
MarcoFalke
fa0aa87071
rpc: Return wtxid from testmempoolaccept 2021-01-12 18:43:43 +01:00
Sebastian Falbesoner
fc726e0138 doc, rpc: add missing signet mentions in network name lists 2021-01-12 18:43:31 +01:00
Wladimir J. van der Laan
bc99ae77e4 scripted-diff: Fix typo in stub manual pages
-BEGIN VERIFY SCRIPT-
sed -i 's/placefolder/placeholder/' $(git ls-files doc/man/\*.1)
-END VERIFY SCRIPT-
2021-01-12 16:46:55 +01:00
Wladimir J. van der Laan
b5e93f873a doc: Add manual page generation for bitcoin-util
- Add `-version` option to `bitcoin-util`
- Add `bitcoin-util` call to `gen-manpages.sh`
- Add stub manual page `bitcoin-util.1`
- Add install of `bitcoin-util.1` to build system
2021-01-12 14:09:21 +01:00
Wladimir J. van der Laan
7b975639ef
Merge #19937: signet mining utility
595a34dbea contrib/signet: Document miner script in README.md (Anthony Towns)
ff7dbdc08a contrib/signet: Add script for generating a signet chain (Anthony Towns)
13762bcc96 Add bitcoin-util command line utility (Anthony Towns)
95d5d5e625 rpc: allow getblocktemplate for test chains when unconnected or in IBD (Anthony Towns)
81c54dec20 rpc: update getblocktemplate with signet rule, include signet_challenge (Anthony Towns)

Pull request description:

  Adds `contrib/signet/miner` for mining signet blocks.

  Adds `bitcoin-util` cli utility, with the idea being it can provide bitcoin related functionality that does not rely on the ability to access a running node. Only subcommand currently is "grind" which takes a hex-encoded header and grinds its nonce until its nBits is satisfied.

  Updates `getblocktemplate` to include `signet_challenge` field, and makes `getblocktemplate` require the signet rule when invoked on the signet change. Removes connectivity and IBD checks from `getblocktemplate` when applied to a test chain (regtest, testnet, signet).

ACKs for top commit:
  laanwj:
    code review ACK 595a34dbea

Tree-SHA512: 8d43297710fdc1edc58acd9b53e1bd1671e5724f7097b40ab73653715dc8becc70534c4496cbba9290f4dd6538a7a3d5830eb85f83391ea31a3bb5b9d3378cc3
2021-01-12 12:53:45 +01:00