Commit Graph

30085 Commits

Author SHA1 Message Date
W. J. van der Laan
efd6f904c7
Merge bitcoin/bitcoin#22570: Ignore banlist.dat
fa1eddb1a3 Fix whitespace in touched files (MarcoFalke)
fa4e6afdae Remove unused CSubNet serialize code (MarcoFalke)
fa384fdd0b Ignore banlist.dat (MarcoFalke)

Pull request description:

  The code to read `banlist.dat` should be removed eventually. The major release (22.x) can be used to translate a `banlist.dat` into a `banlist.json`. Thus, it is now possible to remove the reading code.

ACKs for top commit:
  Zero-1729:
    re-ACK fa1eddb1a3
  laanwj:
    concept and code review ACK fa1eddb1a3
  vasild:
    ACK fa1eddb1a3
  jonatack:
    Light code review utACK fa1eddb1a3

Tree-SHA512: e136193b7c0ba1d6d2e79c7fb4106ba4af75fa229ed7214675ee64e98e59bb4808779e7a8a09eecce62f7a5d4bc6e16b8a5ad4596129357c8fc5e3b88f214249
2021-08-02 13:27:17 +02:00
MarcoFalke
2f60d9fce6
Merge bitcoin/bitcoin#21940: refactor: Mark CAddrMan::Select and GetAddr const
fae108ceb5 Fix incorrect whitespace in addrman (MarcoFalke)
fa32024d51 Add missing GUARDED_BY to CAddrMan::insecure_rand (MarcoFalke)
fab755b77f fuzz: Actually use const addrman (MarcoFalke)
fae0c79351 refactor: Mark CAddrMan::GetAddr const (MarcoFalke)
fa02934c8c refactor: Mark CAddrMan::Select const (MarcoFalke)

Pull request description:

  To clarify that a call to this only changes the random state and nothing else.

ACKs for top commit:
  jnewbery:
    Code review ACK fae108ceb5
  theStack:
    re-ACK fae108ceb5 🍦

Tree-SHA512: 3ffb211d4715cc3daeb3bfcdb3fcc6b108ca96df5fa565510436fac0e8da86c93b30c9c4aad0563e27d84f615fcd729481072009a4e2360c8b3d40787ab6506a
2021-08-02 12:08:45 +02:00
MarcoFalke
bb60960750
Merge bitcoin/bitcoin#22593: test: refactor: remove hex_str_to_bytes helper
ca6c154ef1 test: refactor: remove `hex_str_to_bytes` helper (Sebastian Falbesoner)

Pull request description:

  Use the built-in class method `bytes.fromhex()` instead, which is available since Python 3.0 (https://docs.python.org/3.0/library/stdtypes.html?highlight=fromhex#bytes.fromhex).

  This would be nice to solve with a scripted-diff, but it's tricky to also tackle the imports (which need to be removed in the same commit, otherwise the linter would complain).

ACKs for top commit:
  theStack:
    For the previous ACK (thanks Zero-1729!), it can be trivially re-reviewed via `git range-diff ee5462683...ca6c154ef`
  practicalswift:
    cr ACK ca6c154ef1
  tryphe:
    Concept ACK, cr ACK ca6c154ef1
  Zero-1729:
    re-ACK ca6c154ef1, clean rebase.

Tree-SHA512: 1cda70e7bad523ed937239f6e999c2b43754533a507b18b3ff2503fd0b0ab604efbef23e155726818b970621569d5bcfd42927f1e329184fc1bcbc92651e47ca
2021-08-02 12:02:28 +02:00
W. J. van der Laan
fd9c22ca69
Merge bitcoin/bitcoin#22589: net, doc: update I2P hardcoded seeds and docs for 22.0
d2dffd5be4 doc: add info to i2p.md about IBD time and multiple networks (Jon Atack)
2962640c49 contrib, p2p: update I2P hardcoded seeds (Jon Atack)

Pull request description:

  - Update the I2P seeds with the latest ones I can connect to (feedback welcome).

  - Update the I2P doc with frequently asked and reported info: IBD speed and running with other networks.

ACKs for top commit:
  tryphe:
    ACK d2dffd5be4
  duncandean:
    ACK d2dffd5
  willcl-ark:
    ACK d2dffd5be4
  laanwj:
    ACK d2dffd5be4
  vasild:
    ACK d2dffd5be4

Tree-SHA512: 1edda0eb77b4e224b44961849b36c5a13e1eb14bdc50f909ec44d7fe926c259cada62404906594a1cf68ffd96254647ac49f24248214b42403ecebae45135e63
2021-08-02 11:03:29 +02:00
Sebastian Falbesoner
ca6c154ef1 test: refactor: remove hex_str_to_bytes helper
Use the built-in class method bytes.fromhex() instead,
which is available since Python 3.0.
2021-08-01 19:26:51 +02:00
MarcoFalke
f2e41d1109
Merge bitcoin/bitcoin#22429: test: refactor: fix segwit terminology (s/witness_program/witness_script/)
8a2b58db9e test: fix segwit terminology (s/witness_program/witness_script/) (Sebastian Falbesoner)

Pull request description:

  This PR fixes wrong uses of the term "witness program", which according to [BIP141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Witness_program)  is defined as follows:
  > A scriptPubKey (or redeemScript as defined in BIP16/P2SH) that consists of a 1-byte push opcode (for 0 to 16) followed by a data push between 2 and 40 bytes gets a new special meaning. The value of the first push is called the "version byte". **The following byte vector pushed is called the "witness program".**

  In most cases where "witness program" is used in tests (concerns comments, variable names and in one instance even a function name) what we really want to denote is the "witness script". Thanks to [MarcoFalke for pointing this out in a review comment](https://github.com/bitcoin/bitcoin/pull/22363#discussion_r666794261)!

  Some historical background: At the time when the P2P segwit tests were first introduced (commit 330b0f31ee, PR #8149), the term "witness program" was not used consistently in BIP141: https://bitcoin.stackexchange.com/questions/46451/what-is-the-precise-definition-of-witness-program
  This was fixed in PR https://github.com/bitcoin/bips/pull/416 later.

  So in some way, this PR can be seen as a very late follow-up to the BIP141 fix that also reflects these changes in the tests.

ACKs for top commit:
  josibake:
    tACK 8a2b58db9e

Tree-SHA512: f36bb9e53d1b54b86bfa87ec12f33e3ebca64b5f59d97e9662fe35ba12c25e1c9a4f93a5425d0eaa3879dce9e50368d345555b927bfab76945511f873396892b
2021-08-01 16:59:19 +02:00
MarcoFalke
6499928bfb
Merge bitcoin/bitcoin#22573: fuzz: document faster throughput configuration
8a4f0fcd3f Document faster throughput configuration (Alex Groce)

Pull request description:

  This is a small change to the fuzzing doc that I think might help more people improve the corpus coverage, which I think is low partly just due to lack of long, low-overhead, runs, in addition to the need to apply a more diverse set of fuzzers and coverage notions.

ACKs for top commit:
  practicalswift:
    ACK 8a4f0fcd3f
  tryphe:
    ACK 8a4f0fcd3f

Tree-SHA512: 0f1802f5c551d6ade7393cd2ac439ffd485786b17c4fd0f1a321f69f8ed0db1167ae04b5cae7bf904e89aba03e89b6d974bff564bfc6a78a571893719f323434
2021-07-31 09:26:12 +02:00
MarcoFalke
da1c0c64fd
Merge bitcoin/bitcoin#22330: test: use MiniWallet for simple doublespend sub-test in feature_rbf.py
aa02c64540 test: use MiniWallet for simple doublespend test in feature_rbf.py (Sebastian Falbesoner)
a3f6397c73 test: feature_rbf.py: make MiniWallet instance available for all sub-tests (Sebastian Falbesoner)
84c874794c test: remove unneeded initialization code in feature_rbf.py (Sebastian Falbesoner)

Pull request description:

  This PR's goal is to prepare the functional test `feature_rbf.py` for more MiniWallet usage. It first gets rid of unused initialization code (I guess that was needed at times when the nodes were still in IBD at the start of tests?), then makes the MiniWallet instance introduced in #22210 available for all sub-tests, and finally, uses that instance in the first sub-test `test_simple_doublespend`.

  Note that the same idea of replacing the `make_utxo` calls with MiniWallet can be also applied to other sub-tests too; this just serves as a first proof-of-concept.

ACKs for top commit:
  MarcoFalke:
    re-ACK aa02c64540 🌯

Tree-SHA512: 2902dd15d493d83b0790028c92d14fbd99ca05ace704c7011fb38261ce6517aeb810ef9f360fcb701d95887975b6a2911cfe538858d38fceb2c1c2a40afdbe6b
2021-07-30 14:01:02 +02:00
MarcoFalke
78f040a6b3
Merge bitcoin/bitcoin#22490: test: Disable automatic connections per default in the functional tests
8ca51af1ec test: Disable automatic connections by default (Martin Zumsande)

Pull request description:

  A node normally doesn't make automatic connections to peers in the functional tests because neither DNS seeds nor hardcoded peers are available on regtest. However, when random entries are inserted into addrman as part of a functional test (e.g. while testing addr relay), `ThreadOpenConnections` will periodically try to connect to them, resulting in log entries such as:
  `[opencon] [net.cpp:400] [ConnectNode] trying connection 18.166.1.1:8333 lastseen=0.0hrs`

  I don't think it's desirable that functional tests try to connect to random computers on the internet, aside from the possibility that at some point in time someone out there might actually answer in a way to ruin a test.

  This PR fixes this problem by disabling  `ThreadOpenConnections` by adding `-connect=0` to the default args, and adding exceptions only when needed for the test to pass.

ACKs for top commit:
  tryphe:
    Concept ACK, light code review ACK 8ca51af1ec

Tree-SHA512: bcfb2de610e6c35a97a2bd7ad6267e968b1ac7529638d99276993cd5bc93ce9919d54e22d6dc84e1b02ecd626ab6554e201693552ea065c29794eece38c43f7d
2021-07-30 13:26:50 +02:00
MarcoFalke
b6c3fceed6
Merge bitcoin/bitcoin#22520: test: improve rpc_blockchain.py tests and assert on time and mediantime
ef5e9304cd test: update logging and docstring in rpc_blockchain.py (Jon Atack)
d548dc71e4 test: replace magic values by constants in rpc_blockchain.py (Jon Atack)
78c361086f test: assert on mediantime in getblockheader and getblockchaininfo (Jon Atack)
0a9129c588 test: assert on the value of getblockchaininfo#time (Jon Atack)

Pull request description:

  Follow-up to #22407 improving test coverage per https://github.com/bitcoin/bitcoin/pull/22407#pullrequestreview-702077013.

ACKs for top commit:
  tryphe:
    untested ACK ef5e9304cd

Tree-SHA512: f746d56f430331bc6a2ea7ecd27b21b06275927966aacf1f1127d8d5fdfd930583cabe72e23df3adb2e005da904fc05dc573b8e5eaa2f86e0e193b89a17a5734
2021-07-30 11:44:25 +02:00
MarcoFalke
fa1eddb1a3
Fix whitespace in touched files
Leaving the incorrect indentation would be frustrating because:
* Some editor may fix up the whitespace when editing a file, so before
  commiting the whitespace changes need to be undone.
* It makes it harder to use clang-format-diff on a change.

Can be trivially reviewed with --word-diff-regex=. --ignore-all-space
2021-07-30 11:24:59 +02:00
MarcoFalke
fa4e6afdae
Remove unused CSubNet serialize code 2021-07-30 11:24:41 +02:00
Jon Atack
d2dffd5be4
doc: add info to i2p.md about IBD time and multiple networks 2021-07-30 11:24:34 +02:00
MarcoFalke
fa384fdd0b
Ignore banlist.dat
This also allows to remove the "dirty" argument, which can now be
deduced from the return value of Read().
2021-07-30 11:21:51 +02:00
Jon Atack
2962640c49
contrib, p2p: update I2P hardcoded seeds 2021-07-30 11:03:44 +02:00
MarcoFalke
9c9939ce54
Merge bitcoin/bitcoin#22584: test: Add temporary sanitizer suppression implicit-signed-integer-truncation:netaddress.cpp
fa865287e5 test: Add temporary sanitizer suppression implicit-signed-integer-truncation:netaddress.cpp (MarcoFalke)

Pull request description:

  This is required to unbreak the fuzzers while a fix is being worked on.

  https://cirrus-ci.com/task/4787303177519104?logs=ci#L3020

  ```
  netaddress.cpp:1190:18: runtime error: implicit conversion from type 'int' of value -1 (32-bit, signed) to type 'uint8_t' (aka 'unsigned char') changed the value to 255 (8-bit, unsigned)

ACKs for top commit:
  practicalswift:
    cr ACK fa865287e5
  tryphe:
    untested ACK fa865287e5
  lsilva01:
    ACK fa865287e5

Tree-SHA512: 4a54ec68c014c7a4c9ab268c3a04321db5eb9b2857646b41406d8d4908a3d349848b4549e80aea6afd9a0c3639522a48fe578527139519b12439eae9f0c4c46c
2021-07-30 08:29:35 +02:00
MarcoFalke
fa865287e5
test: Add temporary sanitizer suppression implicit-signed-integer-truncation:netaddress.cpp 2021-07-29 18:51:27 +02:00
fanquake
d23570098c
Merge bitcoin/bitcoin#21882: build: Fix undefined reference to __mulodi4
e4c8bb62e4 build: Fix undefined reference to __mulodi4 (Hennadii Stepanov)

Pull request description:

  When compiling with clang on 32-bit systems the `__mulodi4` symbol is defined in compiler-rt only.

  Fixes #21294.

  See more:
  - https://bugs.llvm.org/show_bug.cgi?id=16404
  - https://bugs.llvm.org/show_bug.cgi?id=28629

ACKs for top commit:
  MarcoFalke:
    tested-only ACK e4c8bb62e4
  luke-jr:
    utACK e4c8bb62e4
  fanquake:
    ACK e4c8bb62e4 - it's a bit of an awkward workaround to carry, but at-least it's contained to the fuzzers.

Tree-SHA512: 93edb4ed568027702b1b9aba953ad50889b834ef97fde3cb99d1ce70076d9c00aa13f95c86b12d6f59b24fa90108d93742f920e15119901a2848fb337ab859a1
2021-07-29 20:53:36 +08:00
fanquake
19434fa22a
Merge bitcoin/bitcoin#22516: macdeploy: alternative info to download the macOS SDK
f8f772dc49 macdeploy: alternative info to download the macOS SDK (Antoine Poinsot)

Pull request description:

  The previous link wasn't accessible for me, this adds some instructions
  given to me by Hebasto on #bitcoin-core-builds as well as a shasum for
  the archive to quickly check the downloaded one is the right one before
  processing with the entire Guix build.

ACKs for top commit:
  fanquake:
    ACK f8f772dc49

Tree-SHA512: 620160b593ed8fa4ae4a748b8e72d67b93ff0ec9e6b8ef3c3ac5402c1c48ec0ac325a527b6278cdf84aaf51ba8194d4c366c412ffad141d0412add2710efcff5
2021-07-29 19:52:05 +08:00
Antoine Poinsot
f8f772dc49
macdeploy: alternative info to download the macOS SDK
The previous link wasn't accessible for me, this adds some instructions
given to me by Hebasto on #bitcoin-core-builds as well as a shasum for
the archive to quickly check the downloaded one is the right one before
processing with the entire Guix build.

This also corrects a link to an older version of the SDK currently in
use.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-07-29 10:59:14 +02:00
MarcoFalke
24fb69dca4
Merge bitcoin/bitcoin#22139: test: add type annotations to util.get_rpc_proxy
fbeb8c43bc test: add type annotations to util.get_rpc_proxy (fanquake)

Pull request description:

  Split out from #22092 while we address the functional test failure.

ACKs for top commit:
  instagibbs:
    ACK fbeb8c43bc

Tree-SHA512: 031ef8703202ae5271787719fc3fea8693574b2eb937ccf852875de95798d7fa3c39a8db7c91993d0c946b45d9b4d6de570bd1102e0344348784723bd84803a8
2021-07-29 09:52:17 +02:00
fanquake
8f76f9f613
Merge bitcoin/bitcoin#22531: guix: Fixes to guix-{attest,verify}
9b313dfef1 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information (Andrew Chow)
43225f0a2a guix: Remove extra \r from all.SHA256SUMS line ending (Andrew Chow)
d080c27066 guix, doc: Add a note that codesigners need to rebuild after tagging (Andrew Chow)
4a466388a0 guix: Allow changing the base manifest in guix-verify (Andrew Chow)
33455c7696 guix: Make all.SHA256SUMS rather than codesigned.SHA256SUMS (Andrew Chow)

Pull request description:

  `guix-verify` expects `all.SHA256SUMS` but `guix-attest` produces `codesigned.SHA256SUMS`. Since `all.SHA256SUMS` makes more sense (as the file contains all the sha256sums, not just the codesigned ones), `guix-attest` has been changed to output a file of that name.

  As a quality of life improvement, `guix-verify` can take `SIGNER` and use the signer's manifest as the base to compare against. This makes it easier to compare a single person's attestations with everyone else's and can make it more obvious when one builder is clearly mismatching with everyone else.

  Lastly `release-process.md` is updated with a note about a gotcha that can cause a mismatch in the codesigned attestation.

ACKs for top commit:
  fanquake:
    ACK 9b313dfef1

Tree-SHA512: 0d60627def38288dbd3059ad1e72cad224f9205da11b1a561c082ef28250a074df5cc5f2797c91a7be027bc486a3fda3319c2e496a8724e5b539337236c6f990
2021-07-29 11:13:12 +08:00
fanquake
be92dc5876
Merge bitcoin/bitcoin#22538: doc: fix command typo in guix README
198ceb82f9 script, doc: guix touchups (jonatack)
d7b7f610a5 Updated Readme, Corrected the codesign typo (h)

Pull request description:

ACKs for top commit:
  jamesob:
    ACK 198ceb82f9
  jonatack:
    ACK 198ceb82f9

Tree-SHA512: 408360cebb51cff330fdd5d5d8ae91a168cdc99fb1377913fd9119e6eba536e58f87ff5c5b479e21a21fa3403323b137c338005bbd67e6fd24314929cdff9325
2021-07-29 11:03:18 +08:00
Alex Groce
8a4f0fcd3f
Document faster throughput configuration 2021-07-28 13:03:08 -07:00
Andrew Chow
9b313dfef1 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information
If the user has set log.showSignature=true in their git config, then the
git log will always output GPG signature information. Since git log is
used to set EPOCH_SOURCE_DATE, this will mistakenly have GPG signature
information in it which causes issues for the build. To avoid this
issue, we override the config and force log.showSignature=false.
2021-07-28 12:59:47 -04:00
Andrew Chow
43225f0a2a guix: Remove extra \r from all.SHA256SUMS line ending
guix-attest mistakenly added an extra \r to the line endings in
all.SHA256SUMS, causing guix-verify to erroneously fail.

Co-Authored-By: Carl Dong <contact@carldong.me>
2021-07-28 12:59:47 -04:00
Andrew Chow
d080c27066 guix, doc: Add a note that codesigners need to rebuild after tagging
One of the issues observed during the 22.0rc1 release process was that a
codesigner's attestation mismatched non-codesigner attestations because
the guix-codesign step was performed prior to tagging the version in
bitcoin-detached-sigs.
2021-07-28 12:59:47 -04:00
MarcoFalke
4b1fb50def
Merge bitcoin/bitcoin#22528: refactor: move GetTransaction to node/transaction.cpp
f685a13bef doc: GetTransaction()/getrawtransaction follow-ups to #22383 (John Newbery)
abc57e1f08 refactor: move `GetTransaction(...)` to node/transaction.cpp (Sebastian Falbesoner)

Pull request description:

  ~This PR is based on #22383, which should be reviewed first~ (merged by now).

  In [yesterday's PR review club session to PR 22383](https://bitcoincore.reviews/22383), the idea of moving the function `GetTransaction(...)` from src/validation.cpp to src/node/transaction.cpp came up. With this, the circular dependency "index/txindex -> validation -> index/txindex" is removed (see change in `lint-circular-dependencies.sh`). Thanks to jnewbery for suggesting and to sipa for providing historical background.

  Relevant IRC log:
  ```
  17:52 <jnewbery> Was anyone surprised that GetTransaction() is in validation.cpp? It seems to me that node/transaction.cpp would be a more appropriate place for it.
  17:53 <raj_> jnewbery, +1
  17:53 <stickies-v> agreed!
  17:54 <glozow> jnewbery ya
  17:54 <jnewbery> seems weird that validation would call into txindex. I wonder if we remove this function, then validation would no longer need to #include txindex
  17:54 <sipa> GetTransaction predates node/transaction.cpp, and even the generic index framework itself :)
  17:55 <sipa> (before 0.8, validation itself used the txindex)
  17:55 <jnewbery> (and GetTransaction() seems like a natural sibling to BroadcastTransaction(), which is already in node/transaction.cpp)
  17:55 <jnewbery> sipa: right, this is not meant as a criticism of course. Just wondering if we can organize things a bit more rationally now that we have better separation between things.
  17:55 <sipa> jnewbery: sure, just providing background
  17:56 <sipa> seems very reasonable to move it elsewhere now
  ```

  The commit should be trivial to review with `--color-moved`.

ACKs for top commit:
  jnewbery:
    Code review ACK f685a13bef
  rajarshimaitra:
    tACK f685a13bef
  mjdietzx:
    crACK f685a13bef
  LarryRuane:
    Code review, test ACK f685a13bef

Tree-SHA512: 0e844a6ecb1be04c638b55bc4478c2949549a4fcae01c984eee078de74d176fb19d508fc09360a62ad130677bfa7daf703b67870800e55942838d7313246248c
2021-07-28 18:19:50 +02:00
Sebastian Falbesoner
aa02c64540 test: use MiniWallet for simple doublespend test in feature_rbf.py 2021-07-28 18:15:53 +02:00
MarcoFalke
67b9416540
Merge bitcoin/bitcoin#21562: [net processing] Various tidying up of PeerManagerImpl ctor
fde1bf4f61 [net processing] Default initialize m_recent_confirmed_transactions (John Newbery)
37dcd12d53 scripted-diff: Rename recentRejects (John Newbery)
cd9902ac50 [net processing] Default initialize recentRejects (John Newbery)
a28bfd1d4c [net processing] Default initialize m_stale_tip_check_time (John Newbery)
9190b01d8d [net processing] Add Orphanage empty consistency check (John Newbery)

Pull request description:

  - Use default initialization of PeerManagerImpl members where possible
  - Remove unique_ptr indirection where it's not needed

ACKs for top commit:
  MarcoFalke:
    ACK fde1bf4f61 πŸ‘ž
  theStack:
    re-ACK fde1bf4f61

Tree-SHA512: 7ddedcc972df8e933e1fbe5c88b8ea17df89e1e58fc769518512c5540e49dc8eddb3f47e78d1329a6fc5644d2c1d11c981f681fd633f5218bfa4b3e6a86f3d7b
2021-07-28 16:31:41 +02:00
W. J. van der Laan
31fef69c03
Merge bitcoin/bitcoin#22047: index, rpc: Coinstatsindex follow-ups
779e638ca9 coinstats: Add comments for new coinstatsindex values (Fabian Jahr)
5b3d4e724f Index: Improve logging in coinstatsindex (Fabian Jahr)
d4356d4e48 rpc: Block until synced if coinstatsindex is used in gettxoutsetinfo (Fabian Jahr)
a5f6791139 rpc: Add missing gettxoutsetinfo help docs (Fabian Jahr)
01386bfd88 Index: Return early from failed coinstatsindex init (Fabian Jahr)
1e3842385b index: Use batch writing in coinstatsindex WriteBlock (Fabian Jahr)
fb65dde147 scripted-diff: Fix coinstats data member names (Fabian Jahr)
8ea8c927ac index: Avoid unnecessary type casts in coinstatsindex (Fabian Jahr)

Pull request description:

  This is a collection of smaller follow-ups to #19521, addressing several post-merge review comments.

ACKs for top commit:
  Sjors:
    re-utACK 779e638ca9
  jonatack:
    re-ACK 779e638ca9 diff since last review involves doc changes only; rebased to current master and verified clean debug build/no silent conflicts, unit tests, and feature_coinstatsindex functional test
  laanwj:
    Code review ACK 779e638ca9
  Talkless:
    re-utACK 779e638ca9 after cosmetic changes.

Tree-SHA512: cb0d038d230c582d7fe3041c89b1e04d39971fab3739d540c609cf826754c6c513b12ded08ac92180aec7a9d7a70114ece50357bd1a902de4adaae9f30b8d699
2021-07-28 15:19:34 +02:00
MarcoFalke
5e213822f8
Merge bitcoin/bitcoin#22530: log: sort logging categories alphabetically
d596dba987 test: assert logging categories are sorted in rpc and help (Jon Atack)
17bbff3b88 log, refactor: use guard clause in LogCategoriesList() (Jon Atack)
7c57297319 log: sort LogCategoriesList and LogCategoriesString alphabetically (Jon Atack)
f720cfa824 test: verify number of categories returned by logging RPC (Jon Atack)

Pull request description:

  Sorting the logging categories seems more user-friendly with the number of categories we now have, allowing CLI users to more quickly find a particular category.

  before
  ```
  $ bitcoin-cli help logging
  ...
  The valid logging categories are: net, tor, mempool, http, bench, zmq, walletdb, rpc, estimatefee, addrman, selectcoins, reindex, cmpctblock, rand, prune, proxy, mempoolrej, libevent, coindb, qt, leveldb, validation, i2p, ipc

  $ bitcoind -h | grep -A8 "debug=<category>"
    -debug=<category>
         ...
         output all debugging information. <category> can be: net, tor,
         mempool, http, bench, zmq, walletdb, rpc, estimatefee, addrman,
         selectcoins, reindex, cmpctblock, rand, prune, proxy, mempoolrej,
         libevent, coindb, qt, leveldb, validation, i2p, ipc.

  $ bitcoin-cli logging [] '["addrman"]'
  {
    "net": false,
    "tor": true,
    "mempool": false,
    "http": false,
    "bench": false,
    "zmq": false,
    "walletdb": false,
    "rpc": false,
    "estimatefee": false,
    "addrman": false,
    "selectcoins": false,
    "reindex": false,
    "cmpctblock": false,
    "rand": false,
    "prune": false,
    "proxy": true,
    "mempoolrej": false,
    "libevent": false,
    "coindb": false,
    "qt": false,
    "leveldb": false,
    "validation": false,
    "i2p": true,
    "ipc": false
  }
  ```

  after

  ```
  $ bitcoin-cli help logging
  ...
  The valid logging categories are: addrman, bench, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb, libevent, mempool, mempoolrej, net, proxy, prune, qt, rand, reindex, rpc, selectcoins, tor, validation, walletdb, zmq

  $ bitcoind -h | grep -A8 "debug=<category>"
    -debug=<category>
         ...
         output all debugging information. <category> can be: addrman,
         bench, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb,
         libevent, mempool, mempoolrej, net, proxy, prune, qt, rand,
         reindex, rpc, selectcoins, tor, validation, walletdb, zmq.

  $ bitcoin-cli logging [] '["addrman"]'
  {
    "addrman": false,
    "bench": false,
    "cmpctblock": false,
    "coindb": false,
    "estimatefee": false,
    "http": false,
    "i2p": false,
    "ipc": false,
    "leveldb": false,
    "libevent": false,
    "mempool": false,
    "mempoolrej": false,
    "net": false,
    "proxy": false,
    "prune": false,
    "qt": false,
    "rand": false,
    "reindex": false,
    "rpc": false,
    "selectcoins": false,
    "tor": false,
    "validation": false,
    "walletdb": false,
    "zmq": false
  }
  ```

ACKs for top commit:
  theStack:
    re-ACK d596dba987

Tree-SHA512: d546257f562b0a288d1b19a028f1a510aaf21bd21da058e7c84653d305ea8662ecb4647ebefd2b97411f845fe5b0b841d40d3fe6814eefcb8ce82df341dfce22
2021-07-28 14:31:11 +02:00
MarcoFalke
93878d2ab5
Merge bitcoin/bitcoin#22423: test: wallet_listtransactions improvements (speedup, cleanup, logging)
a006d7d730 test: add logging to wallet_listtransactions (Sebastian Falbesoner)
47915b1187 test: remove unneeded/redundant code in wallet_listtransactions (Sebastian Falbesoner)
fb6c6a7938 test: speedup wallet_listtransactions by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)

Pull request description:

  This PR improves the test `wallet_listtransactions.py` in three ways:
  * speeds up runtime by a factor of 2-3x by using the good ol' immediate tx relay trick (`-whitelist=noban@127.0.0.1`)
  * removes unneeded/redundant code
  * adds log messages, mostly by turning comments into `self.log.info(...)` calls

ACKs for top commit:
  jonatack:
    ACK a006d7d730
  kristapsk:
    ACK a006d7d730

Tree-SHA512: a91a19f5ebc4d05f0b96c5419683c4c57ac0ef44b64eeb8dd550bd72296fd3a2857a3ba83f755fe4b0b3bd06439973f226070b5d0ce2dee58344dae78cb50290
2021-07-28 14:15:40 +02:00
MarcoFalke
548ca1d3a5
Merge bitcoin/bitcoin#22550: test: improve test_signing_with_{csv,cltv} subtests (speed, prevent timeout)
12f094ec21 test: use constants for CSV/CLTV activation heights in rpc_signrawtransaction (Sebastian Falbesoner)
746f203f19 test: introduce `generate_to_height` helper, use in rpc_signrawtransaction (Sebastian Falbesoner)
e3237b1cd0 test: check that CSV/CLTV are active in rpc_signrawtransaction (Sebastian Falbesoner)

Pull request description:

  This PR primarily aims to solve the current RPC timeout problem for test rpc_signrawtransaction.py, as described in #22542. In the course of that the test is also improved in other ways (see https://github.com/bitcoin/bitcoin/pull/22542#pullrequestreview-714297804).

  Reviewers guideline:
  * In `test_signing_with_cltv()`, a comment is fixed -- it wrongly referred to CSV, it should be CLTV.
  * As preparation, assertions are added that ensure that CSV and CLTV have been really activated after generating blocks by checking the 'softforks' output of the getblockchaininfo() RPC. Right now in master, one could remove (or decrease, like in #22542) the generate calls and the test would still pass, when it shouldn't.
  * A helper `generate_to_height()` is introduced which improves the previous way of reaching a block height in two ways:
      - instead of blindly generating TH blocks to reach target block height >= TH, the current block height CH is taken into account, and only (TH - CH) are generated in total
      - to avoid potential RPC timeouts, the block generation is split up into multiple generatetoaddress RPC calls ([as suggested by laanwj](https://github.com/bitcoin/bitcoin/pull/22542#issuecomment-886237866)); here chunks of 200 blocks have been chosen
   * The helper is used in the affected sub-tests, which should both speed-up the test (from ~18s to ~12s on my machine) and avoid potential timeouts
   * Finally, the activation constants for CSV and CLTV are used instead of using magic numbers 500 and 1500

  Open questions:
  * Any good naming alternatives for `generate_to_height()`? Not really happy with the name, happy to hear suggestions
  * Where to put the CSV and CLTV activation height constants in the test_framewor folder? I guess importing constants from other tests isn't really the desired way to go

ACKs for top commit:
  laanwj:
    Code review and tested ACK 12f094ec21
  rajarshimaitra:
    reACK 12f094ec21

Tree-SHA512: 14509f6d3e5a5a05d6a30a3145bb82cd96a29d9d8a589abf1944a8bf34291cde78ce711195f52e9426588dc822b3618ec9b455e057360021ae46152bb7613516
2021-07-28 13:58:56 +02:00
Jon Atack
d596dba987
test: assert logging categories are sorted in rpc and help 2021-07-28 12:36:28 +02:00
fanquake
be175cef24
Merge bitcoin/bitcoin#22557: fuzz: silence a compiler warning about unused CBanEntry comparator
787296eb67 fuzz: silence a compiler warning about unused CBanEntry comparator (Vasil Dimov)

Pull request description:

  ```
  test/fuzz/banman.cpp:35:13: warning: unused function 'operator==' [-Wunused-function]
  static bool operator==(const CBanEntry& lhs, const CBanEntry& rhs)
              ^
  1 warning generated.
  ```

  See https://github.com/bitcoin/bitcoin/pull/22517#issuecomment-886177699

ACKs for top commit:
  MarcoFalke:
    cr ACK 787296eb67
  practicalswift:
    cr ACK 787296eb67
  hebasto:
    ACK 787296eb67

Tree-SHA512: 72e483cef249170160879cf4b69b787fb6c539d61dda423f618e2c5f130bee8c42897487751e5b58e7679cdb0153eb80efcb104e8a85095daa60d47e39ce78b8
2021-07-28 10:31:48 +08:00
Sebastian Falbesoner
a3f6397c73 test: feature_rbf.py: make MiniWallet instance available for all sub-tests
also document on why we start scanning blocks at height 76
2021-07-27 23:09:48 +02:00
W. J. van der Laan
61cefde7a7
Merge bitcoin/bitcoin#22006: tracing: first tracepoints and documentation on User-Space, Statically Defined Tracing (USDT)
8f37f5c2a5 tracing: Tracepoint for connected blocks (0xb10c)
4224dec22b tracing: Tracepoints for in- and outbound P2P msgs (0xb10c)
469b71ae62 doc: document systemtap dependency (0xb10c)
84ace9aef1 doc: Add initial USDT documentation (0xb10c)

Pull request description:

  This PR adds documentation for User-Space, Statically Defined Tracing (USDT) as well as three tracepoints (including documentation and usage examples).

  ## Context
  The `TRACEx` macros for tracepoints and build system changes for USDT were merged in https://github.com/bitcoin/bitcoin/pull/19866 earlier this year. Issue https://github.com/bitcoin/bitcoin/issues/20981 contains discussion about potential tracepoints and guidelines for adding them (also documented with this PR). USDT was a topic in a [core-dev-meeting discussion](https://bitcoin.jonasschnelli.ch/ircmeetings/logs/bitcoin-core-dev/2021/bitcoin-core-dev.2021-01-21-19.00.moin.txt) on 21st Jan, 2021.

  - [collabora.com: An eBPF overview, part 1: Introduction](https://www.collabora.com/news-and-blog/blog/2019/04/05/an-ebpf-overview-part-1-introduction/)
  - [collabora.com: An eBPF overview, part 2: Machine & bytecode](https://www.collabora.com/news-and-blog/blog/2019/04/15/an-ebpf-overview-part-2-machine-and-bytecode/)
  - [Brendan D. Gregg's blog posts, and book on on eBPF](http://www.brendangregg.com/)
  - [Brendan D. Gregg: Linux bcc/BPF Node.js USDT Tracing](http://www.brendangregg.com/blog/2016-10-12/linux-bcc-nodejs-usdt.html)

  ## USDT? Stablecoin?

  User-space, Statically Defined Tracing (USDT) allows for more observability during development, debugging, code review, and production usage. The tracepoints make it possible to keep track of custom statistics and enable detailed monitoring of otherwise hidden internals and have little to no performance impact when unused. Linux kernels (4.x or newer) can hook into the tracepoints and execute [eBPF] programs in a kernel VM once the tracepoint is called.

  This PR includes, for example, tracepoints for in- and outbound P2P messages.

  ```
  USDT and eBPF Overview
  ======================

                  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”            β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                  β”‚ tracing script   β”‚            β”‚ bitcoind     β”‚
                  β”‚==================β”‚      2.    β”‚==============β”‚
                  β”‚  eBPF  β”‚ tracing β”‚      hooks β”‚              β”‚
                  β”‚  code  β”‚ logic   β”‚      intoβ”Œβ”€β”€β–Ίtracepoint 1─┼───┐ 3.
                  β””β”€β”€β”€β”€β”¬β”€β”€β”€β”΄β”€β”€β–²β”€β”€β”€β”€β”€β”€β”˜          β”œβ”€β”€β–Ίtracepoint 2 β”‚   β”‚ pass args
              1.       β”‚      β”‚ 4.              β”‚ β”‚ ...          β”‚   β”‚ to eBPF
      User    compiles β”‚      β”‚ pass data to    β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚ program
      Space    & loads β”‚      β”‚ tracing script  β”‚                    β”‚
      ─────────────────┼──────┼─────────────────┼────────────────────┼───
      Kernel           β”‚      β”‚                 β”‚                    β”‚
      Space       β”Œβ”€β”€β”¬β”€β–Όβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       β”‚
                  β”‚  β”‚  eBPF program                         β”‚β—„β”€β”€β”€β”€β”€β”€β”˜
                  β”‚  └────────────────────────────────────────
                  β”‚ eBPF kernel Virtual Machine (sandboxed)  β”‚
                  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

  1. The tracing script compiles the eBPF code and loads the eBFP program into a kernel VM
  2. The eBPF program hooks into one or more tracepoints
  3. When the tracepoint is called, the arguments are passed to the eBPF program
  4. The eBPF program processes the arguments and returns data to the tracing script
  ```

  The two main [eBPF] front-ends with support for USDT are [bpftrace] an [BPF Compiler Collection (BCC)]. BCC is used for complex tools and daemons and `bpftrace` is preferred for one-liners and shorter scripts. Example tracing scripts for both are provided with this PR.

  [eBPF]: https://ebpf.io/
  [bpftrace]: https://github.com/iovisor/bpftrace
  [BPF Compiler Collection (BCC)]: https://github.com/iovisor/bcc

  This PR adds three tracepoints:
  - `net:inbound_message`
  - `net:outbound_message`
  - `valildation:block_connected`

  See `doc/tracing.md` and `contrib/tracing/` for documentation and example tracing scripts.

  ## Open Questions (Not in scope for this PR)
  -  How to use these tracepoints under macOS?
  -  Release builds with USDT support?
  -  Should and can the tracepoints be automatically tested?

  ## Todo (before undraft)
  - [x] bcc example showing how to read raw P2P messages up to 32kb
  - [x] document that you need `sys/sdt.h` from `systemtap` for USDT support in Bitcoin Core (`apt install systemtap-sdt-dev` on debian-like). See 933ab8a720
  - [ ] release notes?

ACKs for top commit:
  laanwj:
    re-ACK 8f37f5c2a5
  jb55:
    ACK 8f37f5c2a5

Tree-SHA512: a92a8a2dfcd28465f58a6e5f50d39486817ef5f51214ec40bdb02a6843b9c08ea154fadb31558825ff3a4687477b90f2a5da5d6451989eef978e128a264c289d
2021-07-27 19:47:16 +02:00
0xb10c
8f37f5c2a5
tracing: Tracepoint for connected blocks
Can, for example, be used to benchmark block connections.
2021-07-27 17:12:38 +02:00
0xb10c
4224dec22b
tracing: Tracepoints for in- and outbound P2P msgs
Can be used to monitor in- and outbound node traffic.

Based on ealier work by jb55.

Co-authored-by: William Casarin <jb55@jb55.com>
2021-07-27 17:12:16 +02:00
0xb10c
469b71ae62
doc: document systemtap dependency 2021-07-27 16:33:43 +02:00
0xb10c
84ace9aef1
doc: Add initial USDT documentation
Both added files are extended in the following commits.

doc/usdt.md is based on earlier work by laanwj.

Co-authored-by: W. J. van der Laan <laanwj@protonmail.com>
2021-07-27 16:32:01 +02:00
Vasil Dimov
787296eb67
fuzz: silence a compiler warning about unused CBanEntry comparator
```
test/fuzz/banman.cpp:35:13: warning: unused function 'operator==' [-Wunused-function]
static bool operator==(const CBanEntry& lhs, const CBanEntry& rhs)
            ^
1 warning generated.
```
2021-07-27 13:58:38 +02:00
Hennadii Stepanov
e4c8bb62e4
build: Fix undefined reference to __mulodi4
When compiling with clang on 32-bit systems the __mulodi4 symbol is
defined in compiler-rt only.
2021-07-27 14:53:24 +03:00
MarcoFalke
979f410e69
Merge bitcoin/bitcoin#22561: Fix typo in comment
4148c5228f Fix typo in comment (Cuong V. Nguyen)

Pull request description:

ACKs for top commit:
  theStack:
    ACK 4148c5228f

Tree-SHA512: 03166a3398135fc0f8e0552ff8fd85e0a0b406b847a2b4c101ccf5db13d1b948582b642d22a2160c8a5eb587ccbc281d986728026bab48527be56d5af6c1f033
2021-07-27 11:29:17 +02:00
MarcoFalke
f372623807
Merge bitcoin/bitcoin#22495: p2p: refactor: tidy up PeerManagerImpl::Misbehaving(...)
8858e88c84 p2p: refactor: tidy up `PeerManagerImpl::Misbehaving(...)` (Sebastian Falbesoner)

Pull request description:

  This simple refactoring PR has the goal to improve the readability of the `Misbehaving` method by

  - introducing constant variables `score_before` and  `score_now` (to avoid repeatedly calculating the former)
  - deduplicating calls to LogPrint(), eliminates else-branch

ACKs for top commit:
  jnewbery:
    utACK 8858e88c84
  rajarshimaitra:
    tACK 8858e88c84

Tree-SHA512: 1d4dd5ac1d16ee9595edf4fa46e4960915a203641d74e6c33cffaba62ea71328834309a4451256fb45daf759f0cf6f4f199c46815afff6c89c0746e2ad4d4092
2021-07-27 11:26:39 +02:00
MarcoFalke
7075a52b67
Merge bitcoin/bitcoin#22155: wallet test: Add test for subtract fee from recipient behavior
fe6dc76b7c wallet test: Add test for subtract fee from recipient behavior (Russell Yanofsky)
2565478c81 wallet test refactor: add CreateSyncedWallet function (Russell Yanofsky)

Pull request description:

  This adds test coverage for wallet subtract from recipient behavior without changing it. Behavior seems to have changed recently in a minor way in #17331 without being noticed.

ACKs for top commit:
  achow101:
    ACK fe6dc76b7c
  glozow:
    ACK fe6dc76b7c
  promag:
    Code review ACK fe6dc76b7c.

Tree-SHA512: e00c5dfe467e4ccef5edb0dd4fff6c53f35a37828a4327bea2e166751e5ef971d519ffca7b8f735b12912bb4a547980626356bc1855981005aed1a6c2a57be0b
2021-07-27 11:21:46 +02:00
Cuong V. Nguyen
4148c5228f
Fix typo in comment 2021-07-27 14:01:08 +07:00
fanquake
fbeb8c43bc
test: add type annotations to util.get_rpc_proxy
Remove proxy.url assignment:
error: "AuthServiceProxy" has no attribute "url"
2021-07-27 11:26:14 +08:00
Sebastian Falbesoner
12f094ec21 test: use constants for CSV/CLTV activation heights in rpc_signrawtransaction 2021-07-27 00:14:57 +02:00