Commit Graph

25162 Commits

Author SHA1 Message Date
Amiti Uttarwar
35839e963b [net] Fix bug where AddrFetch connections would be counted as outbound full relay
The desired logic is for us to only open feeler connections after we have hit
the max count for outbound full relay connections.  A short lived AddrFetch
connection (previously called oneshot) could cause ThreadOpenConnections to
miscount and mistakenly open a feeler instead of full relay.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
4972c21b67 [net/refactor] Clarify logic for selecting connections in ThreadOpenConnections 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
60156f5fc4 [net/refactor] Remove fInbound flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
7b322df629 [net/refactor] Remove m_addr_fetch member var from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
14923422b0 [net/refactor] Remove fFeeler flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
49efac5cae [net/refactor] Remove m_manual_connection flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
d3698b5ee3 [net/refactor] Add connection type as a member var to CNode
- Directly maintaining the connection type prevents having to deduce it from
  several flags.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
46578c03e9 [doc] Describe different connection types 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
442abae2ba [net/refactor] Add AddrFetch connections to ConnectionType enum
- AddrFetch connections are short lived connections used to getaddr from a peer
- previously called "one shot" connections
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
af59feb052 [net/refactor] Extract m_addr_known logic from initializer list 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
e1bc29812d [net/refactor] Add block relay only connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
0e52a659a2 [net/refactor] Add feeler connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
1521c47438 [net/refactor] Add manual connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
26304b4100 [net/refactor] Introduce an enum to distinguish type of connection
- extract inbound & outbound types
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
3f1b7140e9 scripted-diff: Rename OneShot to AddrFetch
-BEGIN VERIFY SCRIPT-
sed -i 's/a oneshot/an addrfetch/g' src/chainparams.cpp #comment
sed -i 's/oneshot/addrfetch/g' src/net.cpp #comment
sed -i 's/AddOneShot/AddAddrFetch/g' src/net.h src/net.cpp
sed -i 's/cs_vOneShots/m_addr_fetches_mutex/g' src/net.h src/net.cpp
sed -i 's/vOneShots/m_addr_fetches/g' src/net.h src/net.cpp
sed -i 's/fOneShot/m_addr_fetch/g' src/net.h src/net.cpp src/net_processing.cpp
sed -i 's/ProcessOneShot/ProcessAddrFetch/g' src/net.h src/net.cpp
-END VERIFY SCRIPT-
2020-08-07 17:18:12 -07:00
MarcoFalke
a41ae68053
Merge #19613: travis: Re-enable s390x
fae656144e travis: Re-enable s390x (MarcoFalke)

Pull request description:

  According to travis, the issue has been solved. Quote

  > I would like to confirm that we have resolved this issue and most of our users are reported that this issue has been resolved on their end as well. Could you please re-check and see if that still exists for you?

ACKs for top commit:
  theStack:
    ACK fae656144e

Tree-SHA512: cf42f96d25474a9dcf0817a049e30e29714731d708f73c40a3042b0c70a71ff08f07dd96a89f0dcd5a50a63a355cf30b3511172a32b8af7d5a2e13ad222a4b49
2020-07-28 17:51:58 +02:00
MarcoFalke
fae656144e
travis: Re-enable s390x 2020-07-28 16:01:53 +02:00
MarcoFalke
24ead1a923
Merge #19519: ci: Increase CCACHE_SIZE in some builds on Travis
fdf697fe75 ci: Increase CCACHE_SIZE in some builds on Travis (Hennadii Stepanov)

Pull request description:

  This PR will decrease build time for the longest Travis builds.
  Up to ~20 min improvement with warmed up cache is expected.

Top commit has no ACKs.

Tree-SHA512: ee3a22162d03537be11e60b53de043247f6a65e35e630e0807b758a846b7e05ef2059d18846644aafa3cd5dc7d3e3f56a8ccb94cc71b95227debc9201d3142a2
2020-07-28 10:25:11 +02:00
fanquake
e0ed4cc50d
Merge #19530: depends: build LTO support into Apple's ld64
5962522fbc depends: bump native_cctools for fixed lto with external clang (Cory Fields)
00d1ba7aaa depends: enable lto support for Apple's ld64 (Cory Fields)

Pull request description:

  This didn't work for a few reasons (various toolchain compatibility issues) the last time I tested it, but after the last round of bumps it works with no apparent issues.

  Note that this does not _enable_ LTO by default in any way, only hooks up the machinery for ```-flto``` to work correctly when specified.

  Lines were split for an easier rebase after #17919 is merged.

ACKs for top commit:
  fanquake:
    ACK 5962522fbc. The relevant option upstream is [here](https://github.com/tpoechtrager/cctools-port/blob/master/cctools/m4/llvm.m4#L4).

Tree-SHA512: df2775e74e7bc847e6cef94cb8457d503d6c9e2fdea861e51386fa6ed5a7ba688241db3685561ae1a32f66724c1b3801727252025f00c04b90a3bdc8a4f6f93b
2020-07-28 16:01:26 +08:00
fanquake
cea39fc268
Merge #19595: Fix .gitignore for src/test/fuzz directory
623f66554d Fix .gitignore for src/test/fuzz directory (Hennadii Stepanov)

Pull request description:

  On master (31d2b4098a):
  ```
  $ git ls-files --ignored --exclude-standard
  contrib/init/org.bitcoin.bitcoind.plist
  contrib/macdeploy/fancy.plist
  src/qt/Makefile
  src/qt/test/Makefile
  src/test/Makefile
  src/test/fuzz/FuzzedDataProvider.h
  src/test/fuzz/addition_overflow.cpp
  src/test/fuzz/addrdb.cpp
  src/test/fuzz/asmap.cpp
  src/test/fuzz/asmap_direct.cpp
  src/test/fuzz/autofile.cpp
  src/test/fuzz/banman.cpp
  src/test/fuzz/base_encode_decode.cpp
  src/test/fuzz/bech32.cpp
  src/test/fuzz/block.cpp
  src/test/fuzz/block_header.cpp
  src/test/fuzz/blockfilter.cpp
  src/test/fuzz/bloom_filter.cpp
  src/test/fuzz/buffered_file.cpp
  src/test/fuzz/chain.cpp
  src/test/fuzz/checkqueue.cpp
  src/test/fuzz/coins_view.cpp
  src/test/fuzz/crypto.cpp
  src/test/fuzz/crypto_aes256.cpp
  src/test/fuzz/crypto_aes256cbc.cpp
  src/test/fuzz/crypto_chacha20.cpp
  src/test/fuzz/crypto_chacha20_poly1305_aead.cpp
  src/test/fuzz/crypto_common.cpp
  src/test/fuzz/crypto_hkdf_hmac_sha256_l32.cpp
  src/test/fuzz/crypto_poly1305.cpp
  src/test/fuzz/cuckoocache.cpp
  src/test/fuzz/decode_tx.cpp
  src/test/fuzz/descriptor_parse.cpp
  src/test/fuzz/deserialize.cpp
  src/test/fuzz/eval_script.cpp
  src/test/fuzz/fee_rate.cpp
  src/test/fuzz/fees.cpp
  src/test/fuzz/flatfile.cpp
  src/test/fuzz/float.cpp
  src/test/fuzz/fuzz.cpp
  src/test/fuzz/fuzz.h
  src/test/fuzz/golomb_rice.cpp
  src/test/fuzz/hex.cpp
  src/test/fuzz/http_request.cpp
  src/test/fuzz/integer.cpp
  src/test/fuzz/key.cpp
  src/test/fuzz/key_io.cpp
  src/test/fuzz/kitchen_sink.cpp
  src/test/fuzz/load_external_block_file.cpp
  src/test/fuzz/locale.cpp
  src/test/fuzz/merkleblock.cpp
  src/test/fuzz/message.cpp
  src/test/fuzz/multiplication_overflow.cpp
  src/test/fuzz/net_permissions.cpp
  src/test/fuzz/netaddress.cpp
  src/test/fuzz/p2p_transport_deserializer.cpp
  src/test/fuzz/parse_hd_keypath.cpp
  src/test/fuzz/parse_iso8601.cpp
  src/test/fuzz/parse_numbers.cpp
  src/test/fuzz/parse_script.cpp
  src/test/fuzz/parse_univalue.cpp
  src/test/fuzz/policy_estimator.cpp
  src/test/fuzz/policy_estimator_io.cpp
  src/test/fuzz/pow.cpp
  src/test/fuzz/prevector.cpp
  src/test/fuzz/primitives_transaction.cpp
  src/test/fuzz/process_message.cpp
  src/test/fuzz/process_messages.cpp
  src/test/fuzz/protocol.cpp
  src/test/fuzz/psbt.cpp
  src/test/fuzz/random.cpp
  src/test/fuzz/rbf.cpp
  src/test/fuzz/rolling_bloom_filter.cpp
  src/test/fuzz/script.cpp
  src/test/fuzz/script_bitcoin_consensus.cpp
  src/test/fuzz/script_descriptor_cache.cpp
  src/test/fuzz/script_flags.cpp
  src/test/fuzz/script_interpreter.cpp
  src/test/fuzz/script_ops.cpp
  src/test/fuzz/script_sigcache.cpp
  src/test/fuzz/script_sign.cpp
  src/test/fuzz/scriptnum_ops.cpp
  src/test/fuzz/signature_checker.cpp
  src/test/fuzz/span.cpp
  src/test/fuzz/spanparsing.cpp
  src/test/fuzz/string.cpp
  src/test/fuzz/strprintf.cpp
  src/test/fuzz/system.cpp
  src/test/fuzz/timedata.cpp
  src/test/fuzz/transaction.cpp
  src/test/fuzz/tx_in.cpp
  src/test/fuzz/tx_out.cpp
  src/test/fuzz/util.h
  src/univalue/gen/gen.cpp
  test/functional/data/wallets/high_minversion/db.log
  test/functional/data/wallets/high_minversion/wallet.dat
  ```

  With this PR:
  ```
  $ git ls-files --ignored --exclude-standard
  contrib/init/org.bitcoin.bitcoind.plist
  contrib/macdeploy/fancy.plist
  src/qt/Makefile
  src/qt/test/Makefile
  src/test/Makefile
  src/univalue/gen/gen.cpp
  test/functional/data/wallets/high_minversion/db.log
  test/functional/data/wallets/high_minversion/wallet.dat
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 623f66554d seems like an improvement when writing new fuzz tests
  practicalswift:
    ACK 623f66554d -- thanks for fixing! ❤️
  theStack:
    tested ACK 623f66554d

Tree-SHA512: 16b3854bf4fd8c3096d915a4efc5cbc63d28b18854b051bafee374508dfbb5871ae7dc6f303dbf57469473082d2c3a7df0a8170da22d60d13878544679363b5c
2020-07-28 15:19:00 +08:00
fanquake
2979a7aff0
Merge #19583: p2p: clean up Misbehaving()
a8865f8b72 [net processing] Tidy up Misbehaving() (John Newbery)
d15b3afb4c [net processing] Always supply debug message to Misbehaving() (John Newbery)
634144a1c2 [net processing] Fixup MaybeDiscourageAndDisconnect() style (John Newbery)

Pull request description:

  This PR makes a few minor clean-ups to `Misbehaving()` in preparation to move it out of the cs_main lock.

  There are very minor logging changes but otherwise no functional changes.

ACKs for top commit:
  troygiorshev:
    tACK a8865f8b72
  jonatack:
    ACK a8865f8
  fjahr:
    Code review ACK a8865f8b72
  promag:
    Code review ACK a8865f8b72.

Tree-SHA512: 98fb4f5f76399715545a1ea19290dcebfc8cb4eff72a1d3555dd3de6e184040bb8668c9651dab21db0dfd8e674e53a5977105ef76547146c9f6fa6b4b9d2ba59
2020-07-28 15:15:35 +08:00
fanquake
a1da180b1b
Merge #19589: rpc: Avoid useless mempool query in gettxoutproof
fa5979d12f rpc: Avoid useless mempool query in gettxoutproof (MarcoFalke)
fa1f7f28cb rpc: Style fixups in gettxoutproof (MarcoFalke)

Pull request description:

  `GetTransaction` implicitly and unconditionally asks the mempool global for a transaction. This is problematic for several reasons:

  * `gettxoutproof` is for on-chain txs only and asking the mempool for on-chain txs is confusing and minimally wasteful
  * Globals are confusing and make code harder to test with unit tests

  Fix both issues by passing in an optional mempool. This also helps with #19556

ACKs for top commit:
  hebasto:
    re-ACK fa5979d12f
  jnewbery:
    utACK fa5979d12f
  promag:
    Code review ACK fa5979d12f.

Tree-SHA512: 048361b82abfcc40481181bd44f70cfc9e97d5d6356549df34bbe30b9de7a0a72d2207a3ad0279b21f06293509b284d8967f58ca7e716263a22b20aa4e7f9c54
2020-07-28 14:54:00 +08:00
Wladimir J. van der Laan
b62fbf9e1c
Merge #19593: refactor: Drop unused CBufferedFile::Seek()
7b3851e947 refactor: Drop unused CBufferedFile::Seek() (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK 7b3851e947 -- deleted code is better than unused untested code:)
  MarcoFalke:
    ACK 7b3851e947, assuming that removing this should either be correct or result in a compile failure
  jonasschnelli:
    utACK 7b3851e947
  promag:
    Code review ACK 7b3851e947.

Tree-SHA512: 7bfd172aa4bbe349855c1303fd9cd58093d66833fefe46bd29081bfcca4ab434b84c6b84e76e94d06b8749a5abe1dc1e184f5189136cd1403d0e5bc25ad6d456
2020-07-27 13:57:34 +02:00
Hennadii Stepanov
623f66554d
Fix .gitignore for src/test/fuzz directory 2020-07-27 00:56:37 +03:00
Hennadii Stepanov
7b3851e947
refactor: Drop unused CBufferedFile::Seek() 2020-07-26 22:46:28 +03:00
MarcoFalke
31d2b4098a
Merge #19508: Work around memory-aliasing in descriptor ParsePubkey
fa2ae0ac8d span: Add Span::empty() and use it in script/descriptor (MarcoFalke)
fa8a992589 Work around memory-aliasing in descriptor ParsePubkey (MarcoFalke)

Pull request description:

  While this is not undefined behaviour, the memory aliasing trick is confusing when reading the code. Having `a.size()==0` and then access `a[0]` works in this particular case, but should probably be avoided to harden the code for the future.

ACKs for top commit:
  theStack:
    re-ACK fa2ae0ac8d
  elichai:
    ACK fa2ae0ac8d
  jonatack:
    ACK fa2ae0ac8d

Tree-SHA512: 0ec7b09eef45504973a195923cdf1aa8522117c8e2f69b453e5ce9aa8a7e327c71138518022c32d05133dc99cb861101ed0f60fa891814ee3e9dab3a6fa61a84
2020-07-26 19:01:51 +02:00
MarcoFalke
fa5979d12f
rpc: Avoid useless mempool query in gettxoutproof 2020-07-26 16:44:07 +02:00
MarcoFalke
fa1f7f28cb
rpc: Style fixups in gettxoutproof 2020-07-26 15:43:35 +02:00
John Newbery
a8865f8b72 [net processing] Tidy up Misbehaving()
- Make const things const.
- Replace conditional return with assert.
- Don't log the peer's IP address.
- Log the name Misbehaving directly instead of relying on __func__.
2020-07-25 15:52:23 +01:00
John Newbery
d15b3afb4c [net processing] Always supply debug message to Misbehaving()
Misbehaving() could optionally take a debug string for printing to the
log file. Make this mandatory and always provide the string.

A couple of additional minor changes:

- remove the unnecessary forward declaration of Misbehaving()
- don't include the nodeid or newline in the passed debug message.
Misbehaving() adds these itself.
2020-07-25 15:50:34 +01:00
John Newbery
634144a1c2 [net processing] Fixup MaybeDiscourageAndDisconnect() style
Based on review comments from Marco Falke and Jon Atack.
2020-07-25 15:49:24 +01:00
Wladimir J. van der Laan
40a04814d1
Merge #19472: [net processing] Reduce cs_main scope in MaybeDiscourageAndDisconnect()
655b195747 [net processing] Continue SendMessages processing if not disconnecting peer (John Newbery)
a49781e56d [net processing] Only call MaybeDiscourageAndDisconnect from SendMessages (John Newbery)
a1d5a428a2 [net processing] Fix bad indentation in SendMessages() (John Newbery)
1a1c23f8d4 [net processing] Change cs_main TRY_LOCK to LOCK in SendMessages() (John Newbery)

Pull request description:

  The motivation for this PR is to reduce the scope of cs_main locking in misbehavior logic. It is the first set of commits from a larger branch to move the misbehavior data out of CNodeState and into a new struct that doesn't take cs_main.

  There are some very minor behavior changes in this branch, such as:

  - Not checking for discouragement/disconnect in `ProcessMessages()` (and instead relying on the following check in `SendMessages()`)
  - Checking for discouragement/disconnect as the first action in `SendMessages()` (and not doing ping message sending first)
  - Continuing through `SendMessages()` if `MaybeDiscourageAndDisconnect()` doesn't disconnect the peer (rather than dropping out of `SendMessages()`

ACKs for top commit:
  jonatack:
    re-ACK 655b195 per `git range-diff 505b4ed f54af5e 655b195`, code/commit messages review, a bit of code history, and debug build.
  MarcoFalke:
    ACK 655b195747 only some style-nits 🚁
  promag:
    Code review ACK 655b195747.
  ariard:
    Code Review ACK 655b195

Tree-SHA512: fd6d7bc6bb789f5fb7771fb6a45f61a8faba32af93b766554f562144f9631d15c9cc849a383e71743ef73e610b4ee14853666f6fbf08a3ae35176d48c76c65d3
2020-07-24 17:20:58 +02:00
fanquake
007e15dcd7
Merge #19565: build: call AC_PATH_TOOL for dsymutil in macOS cross-compile
ef3d4ce4c3 build: call AC_PATH_TOOL for dsymutil in macOS cross-compile (fanquake)

Pull request description:

  While testing #19530 I noticed that we couldn't call [`dsymutil`](https://www.llvm.org/docs/CommandGuide/dsymutil.html) after LTO:
  ```bash
  ../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
  ```

  This updates configure to call `AC_PATH_TOOL` so that we end up with the
  full path to dsymutil, similar to `otool` and `install_name_tool`, ie:
  `/bitcoin/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-dsymutil`.

ACKs for top commit:
  laanwj:
    Code review ACK ef3d4ce4c3
  theuni:
    ACK ef3d4ce4c3.

Tree-SHA512: e4fa93e7f9f7945289143dfe2a6645ad8ee7f3bee0793412b3509901a30566d6f952e3b39e0e525a54f8dbd0c480f8da70fc6cb80b07800d11b0c6071fbb7466
2020-07-24 08:57:51 +08:00
MarcoFalke
f4cfa6d019
Merge #15935: Add <datadir>/settings.json persistent settings storage
9c69cfe4c5 Add <datadir>/settings.json persistent settings storage. (Russell Yanofsky)
eb682c5700 util: Add ReadSettings and WriteSettings functions (Russell Yanofsky)

Pull request description:

  Persistent settings are used in followup PRs #15936 to unify gui settings between bitcoin-qt and bitcoind, and #15937 to add a load_on_startup flag to the loadwallet RPC and maintain a dynamic list of wallets that should be loaded on startup that also can be shared between bitcoind and bitcoin-qt.

ACKs for top commit:
  MarcoFalke:
    Approach re-ACK 9c69cfe4c5 🌾
  jnewbery:
    utACK 9c69cfe4c5

Tree-SHA512: 39fcc6051717117c9141e934de1d0d3f739484be4685cdf97d54de967c8c816502b4fd0de12114433beaa5c5b7060c810fd8ae4e2b3ce7c371eb729ac01ba2e1
2020-07-23 18:39:42 +02:00
MarcoFalke
6ee36a263c
Merge #19473: net: Add -networkactive option
2aac093a3d test: Add test coverage for -networkactive option (Hennadii Stepanov)
3c58129b12 net: Log network activity status change unconditionally (Hennadii Stepanov)
62fe6aa87e net: Add -networkactive option (Hennadii Stepanov)

Pull request description:

  Some Bitcoin Core activity is completely local (offline), e.g., reindexing.

  The `setnetworkactive` RPC command is already present. This PR adds the corresponding command-line argument / config option, and allows to start the client with disabled p2p network by providing `-networkactive=0` or `-nonetworkactive`.

  This was done while reviewing #16981.

ACKs for top commit:
  MarcoFalke:
    re-ACK 2aac093a3d 🏠
  LarryRuane:
    ACK 2aac093a3d

Tree-SHA512: 446d791b46d7b556d7694df7b1f88cd4fbc09301fe4eaf036b45cb8166ed806156353cc03788a07b633d5887d5eee30a7c02a2d4307141c8ccc75e0a88145636
2020-07-23 18:32:59 +02:00
Samuel Dobson
9d4b3d86b6
Merge #19334: wallet: Introduce WalletDatabase abstract class
d416ae560e walletdb: Introduce WalletDatabase abstract class (Andrew Chow)
2179dbcbcd walletdb: Add BerkeleyDatabase::Open dummy function (Andrew Chow)
71d28e7cdc walletdb: Introduce AddRef and RemoveRef functions (Andrew Chow)
27b2766384 walletdb: Move BerkeleyDatabase::Flush(true) to Close() (Andrew Chow)

Pull request description:

  A `WalletDatabase` abstract class is created from `BerkeleyDatabase` and is implemented by `BerkeleyDatabase`. First, to get to the point that this is possible, 4 functions need to be added to `BerkeleyDatabase`: `AddRef`, `RemoveRef`, `Open`, and `Close`.

  First the increment and decrement of `mapFileUseCount` is refactored into separate functions `AddRef` and `RemoveRef`.

  `Open` is introduced as a dummy function. This will raise an exception so that it always fails.

  `Close` is refactored from `Flush`. The `shutdown` argument in `Flush` is removed and instead `Flush(true)` is now the `Close` function.

  Split from #18971

  Requires #19325

ACKs for top commit:
  ryanofsky:
    Code review ACK d416ae560e. Only changes since last review were rebasing after base PR #19334 merge, and adding cs_db lock in BerkeleyDatabase destructor, which should avoid races accessing env->m_databases and env->m_fileids
  fjahr:
    Code review ACK d416ae560e
  meshcollider:
    Code review & test run ACK d416ae560e

Tree-SHA512: 98d05ec093d7446c4488e2b0914584222a331e9a2f4d5be6af98e3f6d78fdd8e75526c12f91a8a52d4820c25bce02aa02aabe92d38bee7eb2fce07d0691b7b0d
2020-07-23 15:22:25 +12:00
Hennadii Stepanov
2aac093a3d
test: Add test coverage for -networkactive option 2020-07-22 22:55:48 +03:00
Hennadii Stepanov
3c58129b12
net: Log network activity status change unconditionally 2020-07-22 22:55:47 +03:00
Hennadii Stepanov
62fe6aa87e
net: Add -networkactive option
The `setnetworkactive' RPC command is already present.
This new option allows to start the client with disabled p2p network
activity for testing or reindexing.
2020-07-22 22:55:11 +03:00
Wladimir J. van der Laan
ccef10261e
Merge #18044: Use wtxid for transaction relay
0a4f1422cd Further improve comments around recentRejects (Suhas Daftuar)
0e20cfedb7 Disconnect peers sending wtxidrelay message after VERACK (Suhas Daftuar)
cacd85209e test: Use wtxid relay generally in functional tests (Fabian Jahr)
8d8099e97a test: Add tests for wtxid tx relay in segwit test (Fabian Jahr)
9a5392fdf6 test: Update test framework p2p protocol version to 70016 (Fabian Jahr)
dd78d1d641 Rename AddInventoryKnown() to AddKnownTx() (Suhas Daftuar)
4eb515574e Make TX_WITNESS_STRIPPED its own rejection reason (Suhas Daftuar)
97141ca442 Delay getdata requests from peers using txid-based relay (Suhas Daftuar)
46d78d47de Add p2p message "wtxidrelay" (Suhas Daftuar)
2d282e0cba ignore non-wtxidrelay compliant invs (Anthony Towns)
ac88e2eb61 Add support for tx-relay via wtxid (Suhas Daftuar)
8e68fc246d Add wtxids to recentRejects instead of txids (Suhas Daftuar)
144c385820 Add wtxids of confirmed transactions to bloom filter (Suhas Daftuar)
85c78d54af Add wtxid-index to orphan map (Suhas Daftuar)
08b39955ec Add a wtxid-index to mapRelay (Suhas Daftuar)
60f0acda71 Just pass a hash to AddInventoryKnown (Suhas Daftuar)
c7eb6b4f1f Add wtxid to mempool unbroadcast tracking (Amiti Uttarwar)
2b4b90aa8f Add a wtxid-index to the mempool (Suhas Daftuar)

Pull request description:

  Using txids (a transaction's hash, without witness) for transaction relay is problematic, post-segwit -- if a peer gives us a segwit transaction that fails policy checks, it could be because the txid associated with the transaction is definitely unacceptable to our node (regardless of the witness), or it could be that the transaction was malleated and with a different witness, the txid could be accepted to our mempool.

  We have a bloom filter of recently rejected transactions, whose purpose is to help us avoid redownloading and revalidating transactions that fail to be accepted, but because of this potential for witness malleability to interfere with relay of valid transactions, we do not use the filter for segwit transactions.  This issue is discussed at some length in #8279.  The effect of this is that whenever a segwit transaction that fails policy checks is relayed, a node would download that transaction from every peer announcing it, because it has no way presently to cache failure.  Historically this hasn't been a big problem, but if/when policy for accepting segwit transactions were to change (eg taproot, or any other change), we could expect older nodes talking to newer nodes to be wasting bandwidth because of this.

  As discussed in that issue, switching to wtxid-based relay solves this problem -- by using an identifier for a transaction that commits to all the data in our relay protocol, we can be certain if a transaction that a peer is announcing is one that we've already tried to process, or if it's something new.  This PR introduces support for wtxid-based relay with peers that support it (and remains backwards compatible with peers that use txids for relay, of course).

  Apart from code correctness, one issue to be aware of is that by downloading from old and new peers alike, we should expect there to be some bandwidth wasted, because sometimes we might download the same transaction via txid-relay as well as wtxid-relay.  The last commit in this PR implements a heuristic I want to analyze, which is to just delay relay from txid-relay peers by 2 seconds, if we have at least 1 wtxid-based peer.  I've just started running a couple nodes with this heuristic so I can measure how well it works, but I'm open to other ideas for minimizing that issue.  In the long run, I think this will be essentially a non-issue, so I don't think it's too big a concern, we just need to bite the bullet and deal with it during upgrade.

  Finally, this proposal would need a simple BIP describing the changes, which I haven't yet drafted.  However, review and testing of this code in the interim would be welcome.

  To do items:
  - [x] Write BIP explaining the spec here (1 new p2p message for negotiating wtxid-based relay, along with a new INV type)
  - [ ] Measure and evaluate a heuristic for minimizing how often a node downloads the same transaction twice, when connected to old and new nodes.

ACKs for top commit:
  naumenkogs:
    utACK 0a4f1422cd
  laanwj:
    utACK 0a4f1422cd

Tree-SHA512: d8eb8f0688cf0cbe9507bf738e143edab1f595551fdfeddc2b6734686ea26e7f156b6bfde38bad8bbbe8bec1857c7223e1687f8f018de7463dde8ecaa8f450df
2020-07-22 20:58:55 +02:00
Wladimir J. van der Laan
1397afc5ec
Merge #19526: log: Avoid treating remote misbehvior as local system error
fa56eda58e log: Avoid treating remote misbehvior as local system error (MarcoFalke)
fa492895b5 refactor: Switch ValidationState mode to C++11 enum class (MarcoFalke)

Pull request description:

  When logging failures of `CheckBlockHeader` (high-hash), they are always logged as system error. This is problematic for several reasons:

  * Submitting a blockheader that fails `CheckBlockHeader` over RPC will result in a debug log line that starts with `ERROR`. Proper behaviour should be to log not anything and instead only return the failure reason to the RPC user. This pull does not fix this issue entirely, but is a good first step in the right direction.

  * A misbehaving peer that sends us an invalid block header that fails `CheckBlockHeader` will result in a debug log line that starts with `ERROR`. Proper behavior should be to log the remote peer misbehavior if logging for that category was enabled. This pull fixes this issue for `CheckBlockHeader` and other functions can be adjusted as well if needed in follow-ups. This should be a good first step in the right direction.

ACKs for top commit:
  practicalswift:
    re-ACK fa56eda58e

Tree-SHA512: 9793191f5cb57bdff7c93926e94877e8ca2ef89dcebcf9eb155899c733961839ec7c3f9b9f001dc082ada4234fe6e75f6df431301678d6822325840771166d77
2020-07-22 19:48:55 +02:00
Wladimir J. van der Laan
93decbc7a4
Merge #19370: Static asserts for consistency of fee defaults
1554b54d47 Static asserts for consistency of fee defaults. (Daniel Kraft)

Pull request description:

  This adds `static_assert`'s that ensure that the default values given for fee levels in the wallet (minimum fee and incremental feerate increase) are at least as high as the corresponding levels configured in the core node policy.  Since the core policy values are enforced by the network, it makes sense for the wallet to be conservative and above (or at least not below) this.

ACKs for top commit:
  laanwj:
    code review ACK 1554b54d47, these assumptions seem straightforward

Tree-SHA512: 50e5adf082f467062334377f82a3ee75bcfd436afc65bd0eb33c8d0549d6d90fd1f48c31f60cabe523eb59be9efa8ae0879e9e09cd51ca9c1bd466631ce03cf4
2020-07-22 19:25:07 +02:00
Wladimir J. van der Laan
c7b4968552
Merge #19553: build: pass -fcommon when building genisoimage
2a701a1c42 build: pass -fcommon when building genisoimage (fanquake)

Pull request description:

  Starting with the 10.1 release, GCC [defaults to -fno-common](https://gcc.gnu.org/gcc-10/porting_to.html). This causes
  linking issues when building genisoimage:
  ```bash
  [ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
  [100%] Linking C executable genisoimage
  /usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  ```

  Rather than patching genisoimage further, just pass -fcommon to preserve
  the legacy GCC behaviour.

  Noticed [while testing #19530](https://github.com/bitcoin/bitcoin/pull/19530#issuecomment-659802512).

ACKs for top commit:
  laanwj:
    ACK 2a701a1c42
  hebasto:
    ACK 2a701a1c42, tested on Fedora 32 (x86_64, GCC 10.1.1):

Tree-SHA512: 0e6dd1b4c7c3f5b6997616da27a7ea54f757c81677766ee6813c776356527787a32b959f834e699fee9798553b13bcb142469f442c5e7a04f810f2c06e3a9505
2020-07-22 13:57:22 +02:00
fanquake
ef3d4ce4c3
build: call AC_PATH_TOOL for dsymutil in macOS cross-compile
While testing #19530 I noticed that we couldn't call dsymutil after LTO:
```bash
../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
```

This updates configure to call `AC_PATH_TOOL` so that we end up with the
full path to dsymutil, similar to `otool` and `install_name_tool`, ie:
`/bitcoin/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-otool`.
2020-07-22 18:22:56 +08:00
fanquake
2031aa92a3
Merge #19562: test: Fix fuzzer compilation on macOS
c8992e8959 test: Fix fuzzer compilation on macOS fixes #19557 (freenancial)

Pull request description:

  fixes #19557

  Before the fix:
  ```
  ➜  bitcoin git:(fix-fuzzer-macos) make
  Making all in src
    CXX      test/fuzz/addition_overflow-addition_overflow.o
  In file included from test/fuzz/addition_overflow.cpp:7:
  ./test/fuzz/util.h:335:13: error: no matching function for call to 'AdditionOverflow'
          if (AdditionOverflow((uint64_t)fuzzed_file->m_offset, random_bytes.size())) {
              ^~~~~~~~~~~~~~~~
  ./test/fuzz/util.h:201:16: note: candidate template ignored: deduced conflicting types for parameter 'T' ('unsigned long long' vs. 'unsigned long')
  NODISCARD bool AdditionOverflow(const T i, const T j) noexcept
                 ^
  ./test/fuzz/util.h:346:13: error: no matching function for call to 'AdditionOverflow'
          if (AdditionOverflow(fuzzed_file->m_offset, n)) {
              ^~~~~~~~~~~~~~~~
  ./test/fuzz/util.h:201:16: note: candidate template ignored: deduced conflicting types for parameter 'T' ('long long' vs. 'long')
  NODISCARD bool AdditionOverflow(const T i, const T j) noexcept
                 ^
  ```

  After the fix:
  ```
  ➜  bitcoin git:(fix-fuzzer-macos) ./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ --disable-asm && make clean && make -j5
  ...
  ...
    CXXLD    test/fuzz/uint256_deserialize
  Making all in doc/man
  make[1]: Nothing to be done for `all'.
  make[1]: Nothing to be done for `all-am'.
  ```

ACKs for top commit:
  fanquake:
    ACK c8992e8959 - tested that compiling works on macOS.
  MarcoFalke:
    review ACK c8992e8959

Tree-SHA512: 965cdc61b30db0e2209c91b29f0d42de927a9a5b85e1e70f22d1452e0955f876726c7a8c1d1a5f448f12bf24eec3000802071cd4ae28d8605343fd43d174ca84
2020-07-22 18:03:41 +08:00
fanquake
597d2f905e
Merge #19548: fuzz: add missing overrides to signature_checker
c0f09c2c9d fuzz: add missing overrides to signature_checker (Jon Atack)

Pull request description:

  These functions in `fuzz/signature_checker.cpp` override virtual member functions and should be marked `override` instead of `virtual`, which is for introducing a new virtual function. The overridden virtual functions are in `script/interpreter.h:151/156/161`.

  Also, per MarcoFalke suggestion, add missing parentheses in `fuzz/scriptnum_ops.cpp` and remove useless `unsigned int >= 0` conditional in `fuzz/script.cpp`.

  These changes fix 5 compile warnings in gcc 10 and 3 in clang 11/12.

ACKs for top commit:
  vasild:
    ACK c0f09c2
  MarcoFalke:
    review ACK c0f09c2c9d

Tree-SHA512: 76ce73ec577c1f23cf8646c31d44dcd6c6303732c47187d041a8921d0d24a50163989a375352ebc221abf2ac337bc0902149be46b6f9eebc071d2f364c407f71
2020-07-22 17:32:36 +08:00
MarcoFalke
c7007babb7
Merge #18907: walletdb: Don't remove database transaction logs and instead error
d0ea9bab28 walletdb: Don't remove database transaction logs and instead error (Andrew Chow)

Pull request description:

  Instead of removing the database transaction logs and retrying the
  wallet loading, just return an error message to the user. Additionally,
  speciically for DB_RUNRECOVERY, notify the user that this could be due
  to different BDB versions.

  Kind of implements the suggestion from https://github.com/bitcoin/bitcoin/pull/18870#discussion_r421647964

ACKs for top commit:
  Sjors:
    re-utACK d0ea9bab28
  ryanofsky:
    Code review ACK d0ea9bab28. Only changes since last review are rebase and expanding error and commit messages.

Tree-SHA512: f6e67dc70f58188742a5c8af7cdc63a2b58779aa0d26ae7f1e75805a239f1a342433860e5a238d6577fae5ab04b9d15e7f11c55b867065dfd13781a6a62e4958
2020-07-22 08:58:55 +02:00
Jon Atack
c0f09c2c9d
fuzz: add missing overrides to signature_checker
and also

- add missing parentheses in fuzz/scriptnum_ops.cpp

- remove useless unsigned int conditional in fuzz/script.cpp

These changes fix 5 compile warnings in gcc 10.
2020-07-22 05:27:13 +02:00
freenancial
c8992e8959 test: Fix fuzzer compilation on macOS
fixes #19557
2020-07-21 15:23:49 -07:00
MarcoFalke
2c0c3f8e8c
Merge #19217: p2p: disambiguate block-relay-only variable names from blocksonly variables
ec4c6a17e8 scripted-diff: replace MAX_BLOCKS_ONLY_CONNECTIONS with MAX_BLOCK_RELAY_ONLY_CONNECTIONS (glowang)

Pull request description:

  We have two different concepts that have similar names: `-blocksonly` and `block-relay-only`, and the similarity of names could lead to confusion. `-blocksonly` disables all local receiving & relaying of transactions (with a few exceptions), while `block-relay-only`means that bitcoind will make 2 additional outbound connections that are only used for block relay.

  In net.h and init.cpp, `MAX_BLOCKS_ONLY_CONNECTIONS` is used to represent the maximum number of `block-relay-only` outbound peers, which is 2. But this name sounds ambiguous, and I proposed a better name,  `MAX_BLOCK_RELAY_ONLY_CONNECTION`.

ACKs for top commit:
  jnewbery:
    ACK ec4c6a17e8

Tree-SHA512: cfa592a7ff936f14d10cfc1e926a51b82bc0feaf104885a41ca8111b906cb3d1ec5536bab143a3cfca70aa49e9575c6995941eb6d3d7f4018d4535712342f155
2020-07-21 16:04:33 +02:00