Commit Graph

30056 Commits

Author SHA1 Message Date
Jon Atack
2962640c49
contrib, p2p: update I2P hardcoded seeds 2021-07-30 11:03:44 +02:00
MarcoFalke
9c9939ce54
Merge bitcoin/bitcoin#22584: test: Add temporary sanitizer suppression implicit-signed-integer-truncation:netaddress.cpp
fa865287e5 test: Add temporary sanitizer suppression implicit-signed-integer-truncation:netaddress.cpp (MarcoFalke)

Pull request description:

  This is required to unbreak the fuzzers while a fix is being worked on.

  https://cirrus-ci.com/task/4787303177519104?logs=ci#L3020

  ```
  netaddress.cpp:1190:18: runtime error: implicit conversion from type 'int' of value -1 (32-bit, signed) to type 'uint8_t' (aka 'unsigned char') changed the value to 255 (8-bit, unsigned)

ACKs for top commit:
  practicalswift:
    cr ACK fa865287e5
  tryphe:
    untested ACK fa865287e5
  lsilva01:
    ACK fa865287e5

Tree-SHA512: 4a54ec68c014c7a4c9ab268c3a04321db5eb9b2857646b41406d8d4908a3d349848b4549e80aea6afd9a0c3639522a48fe578527139519b12439eae9f0c4c46c
2021-07-30 08:29:35 +02:00
MarcoFalke
fa865287e5
test: Add temporary sanitizer suppression implicit-signed-integer-truncation:netaddress.cpp 2021-07-29 18:51:27 +02:00
fanquake
d23570098c
Merge bitcoin/bitcoin#21882: build: Fix undefined reference to __mulodi4
e4c8bb62e4 build: Fix undefined reference to __mulodi4 (Hennadii Stepanov)

Pull request description:

  When compiling with clang on 32-bit systems the `__mulodi4` symbol is defined in compiler-rt only.

  Fixes #21294.

  See more:
  - https://bugs.llvm.org/show_bug.cgi?id=16404
  - https://bugs.llvm.org/show_bug.cgi?id=28629

ACKs for top commit:
  MarcoFalke:
    tested-only ACK e4c8bb62e4
  luke-jr:
    utACK e4c8bb62e4
  fanquake:
    ACK e4c8bb62e4 - it's a bit of an awkward workaround to carry, but at-least it's contained to the fuzzers.

Tree-SHA512: 93edb4ed568027702b1b9aba953ad50889b834ef97fde3cb99d1ce70076d9c00aa13f95c86b12d6f59b24fa90108d93742f920e15119901a2848fb337ab859a1
2021-07-29 20:53:36 +08:00
fanquake
19434fa22a
Merge bitcoin/bitcoin#22516: macdeploy: alternative info to download the macOS SDK
f8f772dc49 macdeploy: alternative info to download the macOS SDK (Antoine Poinsot)

Pull request description:

  The previous link wasn't accessible for me, this adds some instructions
  given to me by Hebasto on #bitcoin-core-builds as well as a shasum for
  the archive to quickly check the downloaded one is the right one before
  processing with the entire Guix build.

ACKs for top commit:
  fanquake:
    ACK f8f772dc49

Tree-SHA512: 620160b593ed8fa4ae4a748b8e72d67b93ff0ec9e6b8ef3c3ac5402c1c48ec0ac325a527b6278cdf84aaf51ba8194d4c366c412ffad141d0412add2710efcff5
2021-07-29 19:52:05 +08:00
Antoine Poinsot
f8f772dc49
macdeploy: alternative info to download the macOS SDK
The previous link wasn't accessible for me, this adds some instructions
given to me by Hebasto on #bitcoin-core-builds as well as a shasum for
the archive to quickly check the downloaded one is the right one before
processing with the entire Guix build.

This also corrects a link to an older version of the SDK currently in
use.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-07-29 10:59:14 +02:00
MarcoFalke
24fb69dca4
Merge bitcoin/bitcoin#22139: test: add type annotations to util.get_rpc_proxy
fbeb8c43bc test: add type annotations to util.get_rpc_proxy (fanquake)

Pull request description:

  Split out from #22092 while we address the functional test failure.

ACKs for top commit:
  instagibbs:
    ACK fbeb8c43bc

Tree-SHA512: 031ef8703202ae5271787719fc3fea8693574b2eb937ccf852875de95798d7fa3c39a8db7c91993d0c946b45d9b4d6de570bd1102e0344348784723bd84803a8
2021-07-29 09:52:17 +02:00
fanquake
8f76f9f613
Merge bitcoin/bitcoin#22531: guix: Fixes to guix-{attest,verify}
9b313dfef1 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information (Andrew Chow)
43225f0a2a guix: Remove extra \r from all.SHA256SUMS line ending (Andrew Chow)
d080c27066 guix, doc: Add a note that codesigners need to rebuild after tagging (Andrew Chow)
4a466388a0 guix: Allow changing the base manifest in guix-verify (Andrew Chow)
33455c7696 guix: Make all.SHA256SUMS rather than codesigned.SHA256SUMS (Andrew Chow)

Pull request description:

  `guix-verify` expects `all.SHA256SUMS` but `guix-attest` produces `codesigned.SHA256SUMS`. Since `all.SHA256SUMS` makes more sense (as the file contains all the sha256sums, not just the codesigned ones), `guix-attest` has been changed to output a file of that name.

  As a quality of life improvement, `guix-verify` can take `SIGNER` and use the signer's manifest as the base to compare against. This makes it easier to compare a single person's attestations with everyone else's and can make it more obvious when one builder is clearly mismatching with everyone else.

  Lastly `release-process.md` is updated with a note about a gotcha that can cause a mismatch in the codesigned attestation.

ACKs for top commit:
  fanquake:
    ACK 9b313dfef1

Tree-SHA512: 0d60627def38288dbd3059ad1e72cad224f9205da11b1a561c082ef28250a074df5cc5f2797c91a7be027bc486a3fda3319c2e496a8724e5b539337236c6f990
2021-07-29 11:13:12 +08:00
fanquake
be92dc5876
Merge bitcoin/bitcoin#22538: doc: fix command typo in guix README
198ceb82f9 script, doc: guix touchups (jonatack)
d7b7f610a5 Updated Readme, Corrected the codesign typo (h)

Pull request description:

ACKs for top commit:
  jamesob:
    ACK 198ceb82f9
  jonatack:
    ACK 198ceb82f9

Tree-SHA512: 408360cebb51cff330fdd5d5d8ae91a168cdc99fb1377913fd9119e6eba536e58f87ff5c5b479e21a21fa3403323b137c338005bbd67e6fd24314929cdff9325
2021-07-29 11:03:18 +08:00
Andrew Chow
9b313dfef1 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information
If the user has set log.showSignature=true in their git config, then the
git log will always output GPG signature information. Since git log is
used to set EPOCH_SOURCE_DATE, this will mistakenly have GPG signature
information in it which causes issues for the build. To avoid this
issue, we override the config and force log.showSignature=false.
2021-07-28 12:59:47 -04:00
Andrew Chow
43225f0a2a guix: Remove extra \r from all.SHA256SUMS line ending
guix-attest mistakenly added an extra \r to the line endings in
all.SHA256SUMS, causing guix-verify to erroneously fail.

Co-Authored-By: Carl Dong <contact@carldong.me>
2021-07-28 12:59:47 -04:00
Andrew Chow
d080c27066 guix, doc: Add a note that codesigners need to rebuild after tagging
One of the issues observed during the 22.0rc1 release process was that a
codesigner's attestation mismatched non-codesigner attestations because
the guix-codesign step was performed prior to tagging the version in
bitcoin-detached-sigs.
2021-07-28 12:59:47 -04:00
MarcoFalke
4b1fb50def
Merge bitcoin/bitcoin#22528: refactor: move GetTransaction to node/transaction.cpp
f685a13bef doc: GetTransaction()/getrawtransaction follow-ups to #22383 (John Newbery)
abc57e1f08 refactor: move `GetTransaction(...)` to node/transaction.cpp (Sebastian Falbesoner)

Pull request description:

  ~This PR is based on #22383, which should be reviewed first~ (merged by now).

  In [yesterday's PR review club session to PR 22383](https://bitcoincore.reviews/22383), the idea of moving the function `GetTransaction(...)` from src/validation.cpp to src/node/transaction.cpp came up. With this, the circular dependency "index/txindex -> validation -> index/txindex" is removed (see change in `lint-circular-dependencies.sh`). Thanks to jnewbery for suggesting and to sipa for providing historical background.

  Relevant IRC log:
  ```
  17:52 <jnewbery> Was anyone surprised that GetTransaction() is in validation.cpp? It seems to me that node/transaction.cpp would be a more appropriate place for it.
  17:53 <raj_> jnewbery, +1
  17:53 <stickies-v> agreed!
  17:54 <glozow> jnewbery ya
  17:54 <jnewbery> seems weird that validation would call into txindex. I wonder if we remove this function, then validation would no longer need to #include txindex
  17:54 <sipa> GetTransaction predates node/transaction.cpp, and even the generic index framework itself :)
  17:55 <sipa> (before 0.8, validation itself used the txindex)
  17:55 <jnewbery> (and GetTransaction() seems like a natural sibling to BroadcastTransaction(), which is already in node/transaction.cpp)
  17:55 <jnewbery> sipa: right, this is not meant as a criticism of course. Just wondering if we can organize things a bit more rationally now that we have better separation between things.
  17:55 <sipa> jnewbery: sure, just providing background
  17:56 <sipa> seems very reasonable to move it elsewhere now
  ```

  The commit should be trivial to review with `--color-moved`.

ACKs for top commit:
  jnewbery:
    Code review ACK f685a13bef
  rajarshimaitra:
    tACK f685a13bef
  mjdietzx:
    crACK f685a13bef
  LarryRuane:
    Code review, test ACK f685a13bef

Tree-SHA512: 0e844a6ecb1be04c638b55bc4478c2949549a4fcae01c984eee078de74d176fb19d508fc09360a62ad130677bfa7daf703b67870800e55942838d7313246248c
2021-07-28 18:19:50 +02:00
MarcoFalke
67b9416540
Merge bitcoin/bitcoin#21562: [net processing] Various tidying up of PeerManagerImpl ctor
fde1bf4f61 [net processing] Default initialize m_recent_confirmed_transactions (John Newbery)
37dcd12d53 scripted-diff: Rename recentRejects (John Newbery)
cd9902ac50 [net processing] Default initialize recentRejects (John Newbery)
a28bfd1d4c [net processing] Default initialize m_stale_tip_check_time (John Newbery)
9190b01d8d [net processing] Add Orphanage empty consistency check (John Newbery)

Pull request description:

  - Use default initialization of PeerManagerImpl members where possible
  - Remove unique_ptr indirection where it's not needed

ACKs for top commit:
  MarcoFalke:
    ACK fde1bf4f61 👞
  theStack:
    re-ACK fde1bf4f61

Tree-SHA512: 7ddedcc972df8e933e1fbe5c88b8ea17df89e1e58fc769518512c5540e49dc8eddb3f47e78d1329a6fc5644d2c1d11c981f681fd633f5218bfa4b3e6a86f3d7b
2021-07-28 16:31:41 +02:00
W. J. van der Laan
31fef69c03
Merge bitcoin/bitcoin#22047: index, rpc: Coinstatsindex follow-ups
779e638ca9 coinstats: Add comments for new coinstatsindex values (Fabian Jahr)
5b3d4e724f Index: Improve logging in coinstatsindex (Fabian Jahr)
d4356d4e48 rpc: Block until synced if coinstatsindex is used in gettxoutsetinfo (Fabian Jahr)
a5f6791139 rpc: Add missing gettxoutsetinfo help docs (Fabian Jahr)
01386bfd88 Index: Return early from failed coinstatsindex init (Fabian Jahr)
1e3842385b index: Use batch writing in coinstatsindex WriteBlock (Fabian Jahr)
fb65dde147 scripted-diff: Fix coinstats data member names (Fabian Jahr)
8ea8c927ac index: Avoid unnecessary type casts in coinstatsindex (Fabian Jahr)

Pull request description:

  This is a collection of smaller follow-ups to #19521, addressing several post-merge review comments.

ACKs for top commit:
  Sjors:
    re-utACK 779e638ca9
  jonatack:
    re-ACK 779e638ca9 diff since last review involves doc changes only; rebased to current master and verified clean debug build/no silent conflicts, unit tests, and feature_coinstatsindex functional test
  laanwj:
    Code review ACK 779e638ca9
  Talkless:
    re-utACK 779e638ca9 after cosmetic changes.

Tree-SHA512: cb0d038d230c582d7fe3041c89b1e04d39971fab3739d540c609cf826754c6c513b12ded08ac92180aec7a9d7a70114ece50357bd1a902de4adaae9f30b8d699
2021-07-28 15:19:34 +02:00
MarcoFalke
5e213822f8
Merge bitcoin/bitcoin#22530: log: sort logging categories alphabetically
d596dba987 test: assert logging categories are sorted in rpc and help (Jon Atack)
17bbff3b88 log, refactor: use guard clause in LogCategoriesList() (Jon Atack)
7c57297319 log: sort LogCategoriesList and LogCategoriesString alphabetically (Jon Atack)
f720cfa824 test: verify number of categories returned by logging RPC (Jon Atack)

Pull request description:

  Sorting the logging categories seems more user-friendly with the number of categories we now have, allowing CLI users to more quickly find a particular category.

  before
  ```
  $ bitcoin-cli help logging
  ...
  The valid logging categories are: net, tor, mempool, http, bench, zmq, walletdb, rpc, estimatefee, addrman, selectcoins, reindex, cmpctblock, rand, prune, proxy, mempoolrej, libevent, coindb, qt, leveldb, validation, i2p, ipc

  $ bitcoind -h | grep -A8 "debug=<category>"
    -debug=<category>
         ...
         output all debugging information. <category> can be: net, tor,
         mempool, http, bench, zmq, walletdb, rpc, estimatefee, addrman,
         selectcoins, reindex, cmpctblock, rand, prune, proxy, mempoolrej,
         libevent, coindb, qt, leveldb, validation, i2p, ipc.

  $ bitcoin-cli logging [] '["addrman"]'
  {
    "net": false,
    "tor": true,
    "mempool": false,
    "http": false,
    "bench": false,
    "zmq": false,
    "walletdb": false,
    "rpc": false,
    "estimatefee": false,
    "addrman": false,
    "selectcoins": false,
    "reindex": false,
    "cmpctblock": false,
    "rand": false,
    "prune": false,
    "proxy": true,
    "mempoolrej": false,
    "libevent": false,
    "coindb": false,
    "qt": false,
    "leveldb": false,
    "validation": false,
    "i2p": true,
    "ipc": false
  }
  ```

  after

  ```
  $ bitcoin-cli help logging
  ...
  The valid logging categories are: addrman, bench, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb, libevent, mempool, mempoolrej, net, proxy, prune, qt, rand, reindex, rpc, selectcoins, tor, validation, walletdb, zmq

  $ bitcoind -h | grep -A8 "debug=<category>"
    -debug=<category>
         ...
         output all debugging information. <category> can be: addrman,
         bench, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb,
         libevent, mempool, mempoolrej, net, proxy, prune, qt, rand,
         reindex, rpc, selectcoins, tor, validation, walletdb, zmq.

  $ bitcoin-cli logging [] '["addrman"]'
  {
    "addrman": false,
    "bench": false,
    "cmpctblock": false,
    "coindb": false,
    "estimatefee": false,
    "http": false,
    "i2p": false,
    "ipc": false,
    "leveldb": false,
    "libevent": false,
    "mempool": false,
    "mempoolrej": false,
    "net": false,
    "proxy": false,
    "prune": false,
    "qt": false,
    "rand": false,
    "reindex": false,
    "rpc": false,
    "selectcoins": false,
    "tor": false,
    "validation": false,
    "walletdb": false,
    "zmq": false
  }
  ```

ACKs for top commit:
  theStack:
    re-ACK d596dba987

Tree-SHA512: d546257f562b0a288d1b19a028f1a510aaf21bd21da058e7c84653d305ea8662ecb4647ebefd2b97411f845fe5b0b841d40d3fe6814eefcb8ce82df341dfce22
2021-07-28 14:31:11 +02:00
MarcoFalke
93878d2ab5
Merge bitcoin/bitcoin#22423: test: wallet_listtransactions improvements (speedup, cleanup, logging)
a006d7d730 test: add logging to wallet_listtransactions (Sebastian Falbesoner)
47915b1187 test: remove unneeded/redundant code in wallet_listtransactions (Sebastian Falbesoner)
fb6c6a7938 test: speedup wallet_listtransactions by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)

Pull request description:

  This PR improves the test `wallet_listtransactions.py` in three ways:
  * speeds up runtime by a factor of 2-3x by using the good ol' immediate tx relay trick (`-whitelist=noban@127.0.0.1`)
  * removes unneeded/redundant code
  * adds log messages, mostly by turning comments into `self.log.info(...)` calls

ACKs for top commit:
  jonatack:
    ACK a006d7d730
  kristapsk:
    ACK a006d7d730

Tree-SHA512: a91a19f5ebc4d05f0b96c5419683c4c57ac0ef44b64eeb8dd550bd72296fd3a2857a3ba83f755fe4b0b3bd06439973f226070b5d0ce2dee58344dae78cb50290
2021-07-28 14:15:40 +02:00
MarcoFalke
548ca1d3a5
Merge bitcoin/bitcoin#22550: test: improve test_signing_with_{csv,cltv} subtests (speed, prevent timeout)
12f094ec21 test: use constants for CSV/CLTV activation heights in rpc_signrawtransaction (Sebastian Falbesoner)
746f203f19 test: introduce `generate_to_height` helper, use in rpc_signrawtransaction (Sebastian Falbesoner)
e3237b1cd0 test: check that CSV/CLTV are active in rpc_signrawtransaction (Sebastian Falbesoner)

Pull request description:

  This PR primarily aims to solve the current RPC timeout problem for test rpc_signrawtransaction.py, as described in #22542. In the course of that the test is also improved in other ways (see https://github.com/bitcoin/bitcoin/pull/22542#pullrequestreview-714297804).

  Reviewers guideline:
  * In `test_signing_with_cltv()`, a comment is fixed -- it wrongly referred to CSV, it should be CLTV.
  * As preparation, assertions are added that ensure that CSV and CLTV have been really activated after generating blocks by checking the 'softforks' output of the getblockchaininfo() RPC. Right now in master, one could remove (or decrease, like in #22542) the generate calls and the test would still pass, when it shouldn't.
  * A helper `generate_to_height()` is introduced which improves the previous way of reaching a block height in two ways:
      - instead of blindly generating TH blocks to reach target block height >= TH, the current block height CH is taken into account, and only (TH - CH) are generated in total
      - to avoid potential RPC timeouts, the block generation is split up into multiple generatetoaddress RPC calls ([as suggested by laanwj](https://github.com/bitcoin/bitcoin/pull/22542#issuecomment-886237866)); here chunks of 200 blocks have been chosen
   * The helper is used in the affected sub-tests, which should both speed-up the test (from ~18s to ~12s on my machine) and avoid potential timeouts
   * Finally, the activation constants for CSV and CLTV are used instead of using magic numbers 500 and 1500

  Open questions:
  * Any good naming alternatives for `generate_to_height()`? Not really happy with the name, happy to hear suggestions
  * Where to put the CSV and CLTV activation height constants in the test_framewor folder? I guess importing constants from other tests isn't really the desired way to go

ACKs for top commit:
  laanwj:
    Code review and tested ACK 12f094ec21
  rajarshimaitra:
    reACK 12f094ec21

Tree-SHA512: 14509f6d3e5a5a05d6a30a3145bb82cd96a29d9d8a589abf1944a8bf34291cde78ce711195f52e9426588dc822b3618ec9b455e057360021ae46152bb7613516
2021-07-28 13:58:56 +02:00
Jon Atack
d596dba987
test: assert logging categories are sorted in rpc and help 2021-07-28 12:36:28 +02:00
fanquake
be175cef24
Merge bitcoin/bitcoin#22557: fuzz: silence a compiler warning about unused CBanEntry comparator
787296eb67 fuzz: silence a compiler warning about unused CBanEntry comparator (Vasil Dimov)

Pull request description:

  ```
  test/fuzz/banman.cpp:35:13: warning: unused function 'operator==' [-Wunused-function]
  static bool operator==(const CBanEntry& lhs, const CBanEntry& rhs)
              ^
  1 warning generated.
  ```

  See https://github.com/bitcoin/bitcoin/pull/22517#issuecomment-886177699

ACKs for top commit:
  MarcoFalke:
    cr ACK 787296eb67
  practicalswift:
    cr ACK 787296eb67
  hebasto:
    ACK 787296eb67

Tree-SHA512: 72e483cef249170160879cf4b69b787fb6c539d61dda423f618e2c5f130bee8c42897487751e5b58e7679cdb0153eb80efcb104e8a85095daa60d47e39ce78b8
2021-07-28 10:31:48 +08:00
W. J. van der Laan
61cefde7a7
Merge bitcoin/bitcoin#22006: tracing: first tracepoints and documentation on User-Space, Statically Defined Tracing (USDT)
8f37f5c2a5 tracing: Tracepoint for connected blocks (0xb10c)
4224dec22b tracing: Tracepoints for in- and outbound P2P msgs (0xb10c)
469b71ae62 doc: document systemtap dependency (0xb10c)
84ace9aef1 doc: Add initial USDT documentation (0xb10c)

Pull request description:

  This PR adds documentation for User-Space, Statically Defined Tracing (USDT) as well as three tracepoints (including documentation and usage examples).

  ## Context
  The `TRACEx` macros for tracepoints and build system changes for USDT were merged in https://github.com/bitcoin/bitcoin/pull/19866 earlier this year. Issue https://github.com/bitcoin/bitcoin/issues/20981 contains discussion about potential tracepoints and guidelines for adding them (also documented with this PR). USDT was a topic in a [core-dev-meeting discussion](https://bitcoin.jonasschnelli.ch/ircmeetings/logs/bitcoin-core-dev/2021/bitcoin-core-dev.2021-01-21-19.00.moin.txt) on 21st Jan, 2021.

  - [collabora.com: An eBPF overview, part 1: Introduction](https://www.collabora.com/news-and-blog/blog/2019/04/05/an-ebpf-overview-part-1-introduction/)
  - [collabora.com: An eBPF overview, part 2: Machine & bytecode](https://www.collabora.com/news-and-blog/blog/2019/04/15/an-ebpf-overview-part-2-machine-and-bytecode/)
  - [Brendan D. Gregg's blog posts, and book on on eBPF](http://www.brendangregg.com/)
  - [Brendan D. Gregg: Linux bcc/BPF Node.js USDT Tracing](http://www.brendangregg.com/blog/2016-10-12/linux-bcc-nodejs-usdt.html)

  ## USDT? Stablecoin?

  User-space, Statically Defined Tracing (USDT) allows for more observability during development, debugging, code review, and production usage. The tracepoints make it possible to keep track of custom statistics and enable detailed monitoring of otherwise hidden internals and have little to no performance impact when unused. Linux kernels (4.x or newer) can hook into the tracepoints and execute [eBPF] programs in a kernel VM once the tracepoint is called.

  This PR includes, for example, tracepoints for in- and outbound P2P messages.

  ```
  USDT and eBPF Overview
  ======================

                  ┌──────────────────┐            ┌──────────────┐
                  │ tracing script   │            │ bitcoind     │
                  │==================│      2.    │==============│
                  │  eBPF  │ tracing │      hooks │              │
                  │  code  │ logic   │      into┌─┤►tracepoint 1─┼───┐ 3.
                  └────┬───┴──▲──────┘          ├─┤►tracepoint 2 │   │ pass args
              1.       │      │ 4.              │ │ ...          │   │ to eBPF
      User    compiles │      │ pass data to    │ └──────────────┘   │ program
      Space    & loads │      │ tracing script  │                    │
      ─────────────────┼──────┼─────────────────┼────────────────────┼───
      Kernel           │      │                 │                    │
      Space       ┌──┬─▼──────┴─────────────────┴────────────┐       │
                  │  │  eBPF program                         │◄──────┘
                  │  └───────────────────────────────────────┤
                  │ eBPF kernel Virtual Machine (sandboxed)  │
                  └──────────────────────────────────────────┘

  1. The tracing script compiles the eBPF code and loads the eBFP program into a kernel VM
  2. The eBPF program hooks into one or more tracepoints
  3. When the tracepoint is called, the arguments are passed to the eBPF program
  4. The eBPF program processes the arguments and returns data to the tracing script
  ```

  The two main [eBPF] front-ends with support for USDT are [bpftrace] an [BPF Compiler Collection (BCC)]. BCC is used for complex tools and daemons and `bpftrace` is preferred for one-liners and shorter scripts. Example tracing scripts for both are provided with this PR.

  [eBPF]: https://ebpf.io/
  [bpftrace]: https://github.com/iovisor/bpftrace
  [BPF Compiler Collection (BCC)]: https://github.com/iovisor/bcc

  This PR adds three tracepoints:
  - `net:inbound_message`
  - `net:outbound_message`
  - `valildation:block_connected`

  See `doc/tracing.md` and `contrib/tracing/` for documentation and example tracing scripts.

  ## Open Questions (Not in scope for this PR)
  -  How to use these tracepoints under macOS?
  -  Release builds with USDT support?
  -  Should and can the tracepoints be automatically tested?

  ## Todo (before undraft)
  - [x] bcc example showing how to read raw P2P messages up to 32kb
  - [x] document that you need `sys/sdt.h` from `systemtap` for USDT support in Bitcoin Core (`apt install systemtap-sdt-dev` on debian-like). See 933ab8a720
  - [ ] release notes?

ACKs for top commit:
  laanwj:
    re-ACK 8f37f5c2a5
  jb55:
    ACK 8f37f5c2a5

Tree-SHA512: a92a8a2dfcd28465f58a6e5f50d39486817ef5f51214ec40bdb02a6843b9c08ea154fadb31558825ff3a4687477b90f2a5da5d6451989eef978e128a264c289d
2021-07-27 19:47:16 +02:00
0xb10c
8f37f5c2a5
tracing: Tracepoint for connected blocks
Can, for example, be used to benchmark block connections.
2021-07-27 17:12:38 +02:00
0xb10c
4224dec22b
tracing: Tracepoints for in- and outbound P2P msgs
Can be used to monitor in- and outbound node traffic.

Based on ealier work by jb55.

Co-authored-by: William Casarin <jb55@jb55.com>
2021-07-27 17:12:16 +02:00
0xb10c
469b71ae62
doc: document systemtap dependency 2021-07-27 16:33:43 +02:00
0xb10c
84ace9aef1
doc: Add initial USDT documentation
Both added files are extended in the following commits.

doc/usdt.md is based on earlier work by laanwj.

Co-authored-by: W. J. van der Laan <laanwj@protonmail.com>
2021-07-27 16:32:01 +02:00
Vasil Dimov
787296eb67
fuzz: silence a compiler warning about unused CBanEntry comparator
```
test/fuzz/banman.cpp:35:13: warning: unused function 'operator==' [-Wunused-function]
static bool operator==(const CBanEntry& lhs, const CBanEntry& rhs)
            ^
1 warning generated.
```
2021-07-27 13:58:38 +02:00
Hennadii Stepanov
e4c8bb62e4
build: Fix undefined reference to __mulodi4
When compiling with clang on 32-bit systems the __mulodi4 symbol is
defined in compiler-rt only.
2021-07-27 14:53:24 +03:00
MarcoFalke
979f410e69
Merge bitcoin/bitcoin#22561: Fix typo in comment
4148c5228f Fix typo in comment (Cuong V. Nguyen)

Pull request description:

ACKs for top commit:
  theStack:
    ACK 4148c5228f

Tree-SHA512: 03166a3398135fc0f8e0552ff8fd85e0a0b406b847a2b4c101ccf5db13d1b948582b642d22a2160c8a5eb587ccbc281d986728026bab48527be56d5af6c1f033
2021-07-27 11:29:17 +02:00
MarcoFalke
f372623807
Merge bitcoin/bitcoin#22495: p2p: refactor: tidy up PeerManagerImpl::Misbehaving(...)
8858e88c84 p2p: refactor: tidy up `PeerManagerImpl::Misbehaving(...)` (Sebastian Falbesoner)

Pull request description:

  This simple refactoring PR has the goal to improve the readability of the `Misbehaving` method by

  - introducing constant variables `score_before` and  `score_now` (to avoid repeatedly calculating the former)
  - deduplicating calls to LogPrint(), eliminates else-branch

ACKs for top commit:
  jnewbery:
    utACK 8858e88c84
  rajarshimaitra:
    tACK 8858e88c84

Tree-SHA512: 1d4dd5ac1d16ee9595edf4fa46e4960915a203641d74e6c33cffaba62ea71328834309a4451256fb45daf759f0cf6f4f199c46815afff6c89c0746e2ad4d4092
2021-07-27 11:26:39 +02:00
MarcoFalke
7075a52b67
Merge bitcoin/bitcoin#22155: wallet test: Add test for subtract fee from recipient behavior
fe6dc76b7c wallet test: Add test for subtract fee from recipient behavior (Russell Yanofsky)
2565478c81 wallet test refactor: add CreateSyncedWallet function (Russell Yanofsky)

Pull request description:

  This adds test coverage for wallet subtract from recipient behavior without changing it. Behavior seems to have changed recently in a minor way in #17331 without being noticed.

ACKs for top commit:
  achow101:
    ACK fe6dc76b7c
  glozow:
    ACK fe6dc76b7c
  promag:
    Code review ACK fe6dc76b7c.

Tree-SHA512: e00c5dfe467e4ccef5edb0dd4fff6c53f35a37828a4327bea2e166751e5ef971d519ffca7b8f735b12912bb4a547980626356bc1855981005aed1a6c2a57be0b
2021-07-27 11:21:46 +02:00
Cuong V. Nguyen
4148c5228f
Fix typo in comment 2021-07-27 14:01:08 +07:00
fanquake
fbeb8c43bc
test: add type annotations to util.get_rpc_proxy
Remove proxy.url assignment:
error: "AuthServiceProxy" has no attribute "url"
2021-07-27 11:26:14 +08:00
Sebastian Falbesoner
12f094ec21 test: use constants for CSV/CLTV activation heights in rpc_signrawtransaction 2021-07-27 00:14:57 +02:00
Sebastian Falbesoner
746f203f19 test: introduce generate_to_height helper, use in rpc_signrawtransaction
This will speed up the test a bit and avoid potential .generate() RPC
timeouts (in sub-test `test_signing_with_cltv()`) on slower machines.
2021-07-27 00:14:52 +02:00
Sebastian Falbesoner
8858e88c84 p2p: refactor: tidy up PeerManagerImpl::Misbehaving(...)
- introduce constant variables `score_before` and
  `score_after` in order to improve readability
- deduplicate calls to LogPrint(), eliminates else-branch
2021-07-26 15:51:14 +02:00
Sebastian Falbesoner
e3237b1cd0 test: check that CSV/CLTV are active in rpc_signrawtransaction
Without this check, the tests would also pass if the CSV and
CLTV activation heights are not reached yet (e.g. if the .generate()
calls before are removed), as the operations OP_CSV and OP_CLTV
simply behave as NOPs.
Also fixes a comment in the sub-test `test_signing_with_cltv()`.
2021-07-25 23:44:29 +02:00
Fabian Jahr
779e638ca9
coinstats: Add comments for new coinstatsindex values 2021-07-25 21:02:12 +02:00
Fabian Jahr
5b3d4e724f
Index: Improve logging in coinstatsindex
More accurate logging of a warning should make clear if the recovery condition was hit while catching the results of the previous block.
2021-07-25 20:59:49 +02:00
Fabian Jahr
d4356d4e48
rpc: Block until synced if coinstatsindex is used in gettxoutsetinfo
During initial sync after startup the gettxoutsetinfo RPC will still return an error while catching up. However, after the initial sync the index will not error immediately anymore when it's in the process of syncing to the tip while being called. Instead it will block until synced and then return the response.
2021-07-25 20:59:49 +02:00
Fabian Jahr
a5f6791139
rpc: Add missing gettxoutsetinfo help docs 2021-07-25 20:59:45 +02:00
MarcoFalke
1488f55fa5
Merge bitcoin/bitcoin#22454: fuzz: Limit max ops in tx_pool fuzz targets
fa33ed4b3f fuzz: Limit max ops in tx_pool fuzz targets (MarcoFalke)

Pull request description:

  Without a size limit on the input data, the runtime is unbounded. Fix this by picking an upper bound on the maximum number of fuzz operations.

  Reproducer from OSS-Fuzz (without bug report):
  [clusterfuzz-testcase-tx_pool_standard-5963992253202432.log](https://github.com/bitcoin/bitcoin/files/6822465/clusterfuzz-testcase-tx_pool_standard-5963992253202432.log)

ACKs for top commit:
  practicalswift:
    cr ACK fa33ed4b3f

Tree-SHA512: 32098d573880afba12d510ac83519dc886a6c65d5207edb810f92c7c61edf5e2fc9c57e7b7a1ae656c02ce14e3595707dd6b93caf7956beb2bc817609e14d23d
2021-07-25 12:40:52 +02:00
MarcoFalke
2aa937e97a
Merge bitcoin/bitcoin#22453: fuzz: Limit max ops in rolling_bloom_filter fuzz target
faa86b71ac fuzz: Use ConsumeUInt256 helper to simplify rolling_bloom_filter fuzz test (MarcoFalke)
aaaa61fd30 fuzz: Speed up rolling_bloom_filter fuzz test (MarcoFalke)

Pull request description:

  Without a size limit on the input data, the runtime is unbounded. Fix this by picking an upper bound on the maximum number of fuzz operations.

  Reproducer from OSS-Fuzz (without bug report):
  [clusterfuzz-testcase-rolling_bloom_filter-5980807721254912.log](https://github.com/bitcoin/bitcoin/files/6822159/clusterfuzz-testcase-rolling_bloom_filter-5980807721254912.log)

ACKs for top commit:
  practicalswift:
    cr ACK faa86b71ac
  theStack:
    Concept and code review ACK faa86b71ac

Tree-SHA512: eace588509dfddb2ba97baf86379fa713fa6eb758184abff676cb95807ff8ff36905eeaddeba05665b8464c35c57e2138f88caec71cbfb255e546bbe76558da0
2021-07-25 12:34:55 +02:00
MarcoFalke
2735e111eb
Merge bitcoin/bitcoin#22444: fuzz: Limit max ops in prevector fuzz target
faafda232e fuzz: Speed up prevector fuzz target (MarcoFalke)

Pull request description:

  Without a size limit on the input data, the runtime is unbounded. Fix this by picking an upper bound on the maximum number of fuzz operations.

  Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=35981

ACKs for top commit:
  practicalswift:
    cr ACK faafda232e

Tree-SHA512: 1bf166c4a99a8ce88bdc030cd6a32ce1da5251b73873772e0e9c001ec2bacafebb183f7c8c88806d0ab633aada2cff8b78791f5c9c0c6f2cc8ef5f0875c4b2ef
2021-07-25 12:31:53 +02:00
MarcoFalke
8bc4a11409
Merge bitcoin/bitcoin#22442: util: improves error messages on get_previous_releases script
179a051704 util: improves error messages on get_previous_releases script (Nelson Galdeman)

Pull request description:

  When previous releases are fetched and the specified version wasn't added to the checksum list we used to get a "Checksum did not match" which isn't true (https://github.com/bitcoin-core/bitcoincore.org/issues/753#issuecomment-879546719).

  If the specified version number is not on the list, it now logs cannot do the comparison instead.

ACKs for top commit:
  practicalswift:
    cr ACK 179a051704
  theStack:
    tACK 179a051704, tested on Debian bullseye/sid

Tree-SHA512: 2a07ce75232f853fd311c43581f8faf12d423668946ae6ad784feece5b4d0edd57fc018ba1f0c5a73bfaccb326e0df9a643580d16bf427c1ec3ff34a9cdbc80c
2021-07-25 10:19:04 +02:00
MarcoFalke
401db600b0
Merge bitcoin/bitcoin#22517: fuzz: Temporarily disable failing assert in banman fuzz test
fa8bed6a47 fuzz: Temporarily disable failing assert in banman fuzz test (MarcoFalke)

Pull request description:

  Otherwise the remainder of the fuzz test can't be fuzzed without running into crashes

ACKs for top commit:
  practicalswift:
    cr ACK fa8bed6a47

Tree-SHA512: ec6606292e2cfd26484c7f6caf1c418c377da54111b332990fce68373f0438defda71d931a42ca34431527fbc172dd2fdf29b260afca15b34910ee137de1c365
2021-07-25 10:15:20 +02:00
MarcoFalke
fd557ceb88
Merge bitcoin/bitcoin#13533: [tests] Reduced number of validations in tx_validationcache_tests
c3e111a7da Reduced number of validations in `tx_validationcache_tests` to keep the run time reasonable. (lucash-dev)

Pull request description:

  Following a suggestion in the comments, changed `ValidateCheckInputsForAllFlags` from testing all possible flag combinations to testing a random subset. Also created a new enum constant for the highest flag, so that this test doesn’t keep testing an incomplete subset in case a new flag is added.

  Timing for `checkinputs_test`:
  ```
  Before:   6.8s
  After:    3.7s
  ----------------
  Saved:    3.1s (45%)
  ```

  This PR was split from #13050. Also see #10026.

ACKs for top commit:
  leonardojobim:
    tACK c3e111a7da.
  kallewoof:
    ACK c3e111a7da
  theStack:
    re-ACK c3e111a7da

Tree-SHA512: bef49645bdd4f61ec73cc77a9f028b95d9856db9446d2e7fc9a48867a6f0e94c2c9f150cb771a30fe852db0efb0a1bd15d38b00d712651793ccb59ff6157a7b4
2021-07-24 10:36:58 +02:00
jonatack
198ceb82f9 script, doc: guix touchups 2021-07-23 21:23:37 +05:30
h
d7b7f610a5 Updated Readme, Corrected the codesign typo 2021-07-23 20:15:16 +05:30
fanquake
2b5563bb1e
Merge bitcoin/bitcoin#22533: guix/build: Remove vestigial SKIPATTEST.TAG
9f01feda0a guix/build: Remove vestigial SKIPATTEST.TAG (Carl Dong)

Pull request description:

  No longer needed or referenced by anything. A relic from prior to the great hierarchy overhaul of #22182

ACKs for top commit:
  achow101:
    ACK 9f01feda0a
  fanquake:
    ACK 9f01feda0a

Tree-SHA512: a94cf63f0c5cb8dbacf1025b6c0e81b219c2a3c93b3cbcefc239ccde29e602ecd4b717b1d93dbe53cb791a5017236fb09823c034aec42b0c31894fc9e0ab8b21
2021-07-23 14:03:24 +08:00
fanquake
9df1906091
Merge bitcoin/bitcoin#22511: guix: Silence getent(1) invocation, doc fixups
a884a1edcd guix/INSTALL: Misc fixups (Carl Dong)
3c4d2c418e guix: Silence getent(1) invocation (Carl Dong)

Pull request description:

  Otherwise the `getent(1)` checks will print out the default http, https, and ftp ports, making it seem like something is being spawned that is listening on those ports, which is not the case.

ACKs for top commit:
  fanquake:
    ACK a884a1edcd

Tree-SHA512: 7706a98fe5f2bcd766fd3a16bfffab899ec45e80d72c485b7bed2a83d2024eddbb44ae4a77e2352e308740ca203c163421a11a5a2327fa94d2032ecceef4d63f
2021-07-23 09:07:58 +08:00