Commit Graph

30181 Commits

Author SHA1 Message Date
John Newbery
2837a9f1ea [mempool] Only add a transaction to the unbroadcast set when it's added to the mempool
Currently, if BroadcastTransaction() is called to rebroadcast a
transaction (e.g. by ResendWalletTransactions()), then we add the
transaction to the unbroadcast set. That transaction has already been
broadcast in the past, so peers are unlikely to request it again,
meaning RemoveUnbroadcastTx() won't be called and it won't be removed
from m_unbroadcast_txids.

Net processing will therefore continue to attempt rebroadcast for the
transaction every 10-15 minutes. This will most likely continue until
the node connects to a new peer which hasn't yet seen the transaction
(or perhaps indefinitely).

Fix by only adding the transaction to the broadcast set when it's added
to the mempool.
2021-07-09 17:24:08 +01:00
W. J. van der Laan
8ab0c77299
Merge bitcoin/bitcoin#22253: validation: distinguish between same tx and same-nonwitness-data tx in mempool
b7a8cd9963 [test] submit same txid different wtxid as mempool tx (glozow)
fdb48163bf [validation] distinguish same txid different wtxid in mempool (glozow)

Pull request description:

  On master, if you submit a transaction with the same txid but different witness to the mempool, it thinks the transactions are the same. Users submitting through `BroadcastTransaction()` (i.e. `sendrawtransaction` or the wallet) don't get notified that there's a different transaction in the mempool, although it doesn't crash. Users submitting through `testmempoolaccept()` will get a "txn-already-in-mempool" error.

  This PR simply distinguishes between `txn-already-in-mempool` and `txn-same-nonwitness-data-in-mempool`, without handling them differently: `sendrawtransaction` still will not throw, but `testmempoolaccept` will give you a different error.

  I believe the intention of #19645 is to allow full swaps of transactions that have different witnesses but identical nonwitness data. Returning a different error message + adding a test was suggested: https://github.com/bitcoin/bitcoin/pull/19645#issuecomment-705109193 so this is that PR.

ACKs for top commit:
  naumenkogs:
    ACK b7a8cd9963
  jnewbery:
    Code review ACK b7a8cd9963
  theStack:
    Code-review ACK b7a8cd9963
  darosior:
    re-utACK b7a8cd9963

Tree-SHA512: 9c6591edaf8727ba5b4675977adb8cbdef7288584003b6cd659828032dc92d2ae915800a8ef8b6fdffe112c1b660df72297a3dcf2e2e3e1f959c6cb3678c63ee
2021-07-09 17:34:46 +02:00
Vasil Dimov
4101ec9d2e
doc: mention that we enforce port=0 in I2P
Co-authored-by: Jon Atack <jon@atack.com>
2021-07-09 11:19:38 +02:00
Vasil Dimov
e0a2b390c1
addrman: reset I2P ports to 0 when loading from disk
This is a temporary change to convert I2P addresses that have propagated
with port 8333 to ones with port 0.

It would cause a problem some day if indeed some bitcoin software is
listening on port 8333 only and rejects connections to port 0 and we are
still using SAM 3.1 which only supports port 0. In this case we would
replace 8333 with 0 and try to connect to such nodes.

This commit should be included in 22.0 and be reverted before 23.0 is
released.
2021-07-09 11:19:38 +02:00
Vasil Dimov
41cda9d075
test: ensure I2P ports are handled as expected 2021-07-09 11:19:37 +02:00
Vasil Dimov
4f432bd738
net: do not connect to I2P hosts on port!=0
When connecting to an I2P host we don't specify destination port and it
is being forced to 0 by the SAM 3.1 proxy, so if we connect to the same
host on two different ports, that would be actually two connections to
the same service (listening on port 0).

Fixes https://github.com/bitcoin/bitcoin/issues/21389
2021-07-09 11:19:37 +02:00
Vasil Dimov
1f096f091e
net: distinguish default port per network
Change `CChainParams::GetDefaultPort()` to return 0 if the network is
I2P.
2021-07-09 11:19:36 +02:00
Vasil Dimov
aeac3bce3e
net: change I2P seeds' ports to 0 2021-07-09 11:19:35 +02:00
Vasil Dimov
38f900290c
net: change assumed I2P port to 0
* When accepting an I2P connection, assume the peer has port 0 instead
  of the default 8333 (for mainnet). It is not being sent to us, so we
  must assume something.
* When deriving our own I2P listen CService use port 0 instead of the
  default 8333 (for mainnet). So that we later advertise it to peers
  with port 0.

In the I2P protocol SAM 3.1 and older (we use 3.1) ports are not used,
so they are irrelevant. However in SAM 3.2 and newer ports are used and
from the point of view of SAM 3.2, a peer using SAM 3.1 seems to have
specified port=0.
2021-07-09 11:19:35 +02:00
MarcoFalke
d1e4c56309
Merge bitcoin/bitcoin#22363: test: refactor: use script_util helpers for creating P2{PKH,SH,WPKH,WSH} scripts
905d672b74 test: use script_util helpers for creating P2W{PKH,SH} scripts (Sebastian Falbesoner)
285a65ccfd test: use script_util helpers for creating P2SH scripts (Sebastian Falbesoner)
b57b633b94 test: use script_util helpers for creating P2PKH scripts (Sebastian Falbesoner)
61b6a017a9 test: wallet util: fix multisig P2SH-P2WSH script creation (Sebastian Falbesoner)

Pull request description:

  PR #18788 (commit 08067aebfd) introduced functions to generate output scripts for various types. This PR replaces all manual CScript creations in the P2PKH, P2SH, P2WPKH, P2WSH formats with those helpers in order to increase readability and maintainability over the functional test codebase. The first commit fixes a bug in the wallet_util helper module w.r.t. to P2SH-P2WSH script creation (the result is not used in any test so far, hence it can still be seen as refactoring).

  The following table shows a summary of the output script patterns tackled in this PR:

  | Type | master branch  | PR branch |
  | ---------- | ------------- | ------------- |
  | P2PKH | `CScript([OP_DUP, OP_HASH160, hash160(key), OP_EQUALVERIFY, OP_CHECKSIG])` | `key_to_p2pkh_script(key)`  |
  |             | `CScript([OP_DUP, OP_HASH160, keyhash, OP_EQUALVERIFY, OP_CHECKSIG])` | `keyhash_to_p2pkh_script(keyhash)` |
  | P2SH  | `CScript([OP_HASH160, hash160(script), OP_EQUAL])`  | `script_to_p2sh_script(script)` |
  | P2WPKH | `CScript([OP_0, hash160(key)])` | `key_to_p2wpkh_script(key)` |
  | P2WSH | `CScript([OP_0, sha256(script)])` | `script_to_p2wsh_script(script)` |

  Note that the `key_to_...` helpers can't be used if an invalid key size (not 33 or 65 bytes) is passed, which is the case in some rare instances where the scripts still have to be created manually.

  Possible follow-up ideas:
  * further simplify by identifying P2SH-wrapped scripts and using `key_to_p2sh_p2wpkh_script()` and `script_to_p2sh_p2wsh_script()` helpers
  * introduce and use `key_to_p2pk_script()` helper for P2PK scripts

ACKs for top commit:
  rajarshimaitra:
    tACK 905d672b74
  LarryRuane:
    tACK 905d672b74
  0xB10C:
    ACK 905d672b74
  MarcoFalke:
    review ACK 905d672b74 🕹

Tree-SHA512: 7ccfe69699bc81168ac122b03536720013355c1b2fbb088355b616015318644c4d1cd27e20c4f56c89ad083ae609add4bc838cf6316794d0edb0ce9cf7fa0fd8
2021-07-09 11:17:58 +02:00
fanquake
efff9c3494
Merge bitcoin/bitcoin#22176: test: Correct outstanding -Werror=sign-compare errors
4e44f5bac4 test: Correct outstanding -Werror=sign-compare errors (Ben Woosley)

Pull request description:

  I'm unclear on why these aren't failing on CI, but they failed for me locally, e.g.:

  ```
  In file included from /usr/local/include/boost/test/test_tools.hpp:46:
  /usr/local/include/boost/test/tools/old/impl.hpp:107:17: error: comparison of integers of different signs: 'const unsigned int' and 'const int' [-Werror,-Wsign-compare]
      return left == right;
             ~~~~ ^  ~~~~~
  /usr/local/include/boost/test/tools/old/impl.hpp:130:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl<unsigned int, int>' requested here
          return equal_impl( left, right );
                 ^
  /usr/local/include/boost/test/tools/old/impl.hpp:145:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::call_impl<unsigned int, int>' requested here
          return call_impl( left, right, left_is_array() );
                 ^
  /usr/local/include/boost/test/tools/old/impl.hpp:92:50: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::operator()<unsigned int, int>' requested here
  BOOST_PP_REPEAT( BOOST_TEST_MAX_PREDICATE_ARITY, IMPL_FRWD, _ )
                                                   ^
  /usr/local/include/boost/preprocessor/repetition/repeat.hpp:30:26: note: expanded from macro 'BOOST_PP_REPEAT'
                           ^
  /usr/local/include/boost/preprocessor/cat.hpp:22:32: note: expanded from macro 'BOOST_PP_CAT'
                                 ^
  /usr/local/include/boost/preprocessor/cat.hpp:29:34: note: expanded from macro 'BOOST_PP_CAT_I'
                                   ^
  <scratch space>:153:1: note: expanded from here
  BOOST_PP_REPEAT_1
  ^
  test/streams_tests.cpp:122:5: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::check_frwd<boost::test_tools::tt_detail::equal_impl_frwd, unsigned int, int>' requested here
      BOOST_CHECK_EQUAL(varint, 54321);
      ^

  /usr/local/include/boost/test/tools/old/impl.hpp:107:17: error: comparison of integers of different signs: 'const unsigned long long' and 'const long' [-Werror,-Wsign-compare]
      return left == right;
             ~~~~ ^  ~~~~~
  /usr/local/include/boost/test/tools/old/impl.hpp:130:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl<unsigned long long, long>' requested here
          return equal_impl( left, right );
                 ^
  /usr/local/include/boost/test/tools/old/impl.hpp:145:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::call_impl<unsigned long long, long>' requested here
          return call_impl( left, right, left_is_array() );
                 ^
  /usr/local/include/boost/test/tools/old/impl.hpp:92:50: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::operator()<unsigned long long, long>' requested here
  BOOST_PP_REPEAT( BOOST_TEST_MAX_PREDICATE_ARITY, IMPL_FRWD, _ )
                                                   ^
  /usr/local/include/boost/preprocessor/repetition/repeat.hpp:30:26: note: expanded from macro 'BOOST_PP_REPEAT'
                           ^
  /usr/local/include/boost/preprocessor/cat.hpp:22:32: note: expanded from macro 'BOOST_PP_CAT'
                                 ^
  /usr/local/include/boost/preprocessor/cat.hpp:29:34: note: expanded from macro 'BOOST_PP_CAT_I'
                                   ^
  <scratch space>:161:1: note: expanded from here
  BOOST_PP_REPEAT_1
  ^
  test/serfloat_tests.cpp:41:5: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::check_frwd<boost::test_tools::tt_detail::equal_impl_frwd, unsigned long long, long>' requested here
      BOOST_CHECK_EQUAL(TestDouble(std::numeric_limits<double>::infinity()), 0x7ff0000000000000);
      ^

ACKs for top commit:
  theStack:
    ACK 4e44f5bac4

Tree-SHA512: 8d9e5245676c61207ceacdf78c78a78ccc9fd2a2551d4d8df023513795591334aa2f5e1f4a2a8ed2bfeb381f1e226b6ba84c07e0de29a1f3f00da71f3a257bc1
2021-07-09 10:22:43 +08:00
fanquake
34d1d6a112
Merge bitcoin/bitcoin#22381: guix: Test security-check sanity before performing them (with macOS)
5b4703c6a7 guix: Test security-check sanity before performing them (Carl Dong)
6cf3345297 scripts: adjust test-symbol-check for guix release environment (fanquake)
1946b5f77c scripts: more robustly test macOS symbol checks (fanquake)
a8127b34bc build: Use and test PE binutils with --reloc-section (Carl Dong)
678348db51 guix: Patch binutils to add security-related disable flags (Carl Dong)
9fdc8afe11 devtools: Improve *-check.py tool detection (Carl Dong)
bda62eab38 ci: skip running the Linux test-security-check target for now (fanquake)
d6ef3543ae lint: Run mypy with --show-error-codes (Carl Dong)

Pull request description:

  This is #20980 rebased (to include the Boost Process fix), and with an additional commit (892d6897f1e613084aa0517a660eab2412308e6e) to fix running the `test-security-check` target for the macOS build. It should pass inside Guix, as well as when cross-compiling on Ubuntu, or building natively on macOS.

  Note that the `test-security-check` may output some warnings (similar too):
  ```bash
  ld: warning: passed two min versions (10.14, 11.4) for platform macOS. Using 11.4.
  ld: warning: passed two min versions (10.14, 11.4) for platform macOS. Using 11.4.
  ld: warning: passed two min versions (10.14, 10.14) for platform macOS. Using 10.14.
  ```
  but those can be ignored, and come about due to us passing `-platform_version` when `-mmacosx-version-min` is already part of `CC`.

  Guix builds:
  ```bash
  71ed0c7a13a4726300779ffc87f7d271086a2744c36896fe6dc51fe3dc33df2e  guix-build-5b4703c6a70d/output/aarch64-linux-gnu/SHA256SUMS.part
  9273980a17052c8ec45b77579781c14ab5d189fa25aa29907d5115513dd302b1  guix-build-5b4703c6a70d/output/aarch64-linux-gnu/bitcoin-5b4703c6a70d-aarch64-linux-gnu-debug.tar.gz
  9c042179af43c8896eb95a34294df15d4910308dcdba40b2010cd36e192938b8  guix-build-5b4703c6a70d/output/aarch64-linux-gnu/bitcoin-5b4703c6a70d-aarch64-linux-gnu.tar.gz
  1ceddecac113f50a952ba6a201cdcdb722e3dc804e663f219bfac8268ce42bf0  guix-build-5b4703c6a70d/output/arm-linux-gnueabihf/SHA256SUMS.part
  759597c4e925e75db4a2381c06cda9b9f4e4674c23436148676b31c9be05c7aa  guix-build-5b4703c6a70d/output/arm-linux-gnueabihf/bitcoin-5b4703c6a70d-arm-linux-gnueabihf-debug.tar.gz
  34e3b6beabaf8c95d7c2ca0d2c3ac4411766694ef43e00bd9783badbbaf045a7  guix-build-5b4703c6a70d/output/arm-linux-gnueabihf/bitcoin-5b4703c6a70d-arm-linux-gnueabihf.tar.gz
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  guix-build-5b4703c6a70d/output/dist-archive/SKIPATTEST.TAG
  3664f6ceee7898caa374281fd877a7597fe491fa2e9f0c174c28d889d60b559c  guix-build-5b4703c6a70d/output/dist-archive/bitcoin-5b4703c6a70d.tar.gz
  d6bc35ba0750c1440bb32831b8c12cddee62f6dce10fec2650897444c2bf4748  guix-build-5b4703c6a70d/output/powerpc64-linux-gnu/SHA256SUMS.part
  a836edf6474ba0c16c19bb217549bac7936c1b44306ed512df58f607ee5568f2  guix-build-5b4703c6a70d/output/powerpc64-linux-gnu/bitcoin-5b4703c6a70d-powerpc64-linux-gnu-debug.tar.gz
  7cc91c6805d5069ca3bd1771e77d95f83eb184b137198cbf84d1d11d0a5c5afe  guix-build-5b4703c6a70d/output/powerpc64-linux-gnu/bitcoin-5b4703c6a70d-powerpc64-linux-gnu.tar.gz
  93b4cb7b83c4975120ad5de5a92f050f5760a2a3f2c37c204c647f5a581c924a  guix-build-5b4703c6a70d/output/powerpc64le-linux-gnu/SHA256SUMS.part
  2266e2c5d0dafa28c6c057ccfc1c439baeab1d714d8c3f64a83015d2827116d2  guix-build-5b4703c6a70d/output/powerpc64le-linux-gnu/bitcoin-5b4703c6a70d-powerpc64le-linux-gnu-debug.tar.gz
  85f41f42c319b83d049d6fd2e2278c07b40a1e28a2eac596427822c0eef9dc3f  guix-build-5b4703c6a70d/output/powerpc64le-linux-gnu/bitcoin-5b4703c6a70d-powerpc64le-linux-gnu.tar.gz
  1499ca9119926083d8c3714ca10d8d4c8d864cbeee8848fd8445b7a1d081222d  guix-build-5b4703c6a70d/output/riscv64-linux-gnu/SHA256SUMS.part
  1995fc1a2e45c49d4b0718aff5dcdac931917e8ae9e762fd23f1126abcecc248  guix-build-5b4703c6a70d/output/riscv64-linux-gnu/bitcoin-5b4703c6a70d-riscv64-linux-gnu-debug.tar.gz
  266889eb58429a470f0fd7bb123f2ae09b0aef86c47b0390938b3634a8f748a9  guix-build-5b4703c6a70d/output/riscv64-linux-gnu/bitcoin-5b4703c6a70d-riscv64-linux-gnu.tar.gz
  cdc3a0dcf80b110443dac5ddf8bc951001a776a651c898c5ea49bb2d487bfe29  guix-build-5b4703c6a70d/output/x86_64-apple-darwin18/SHA256SUMS.part
  8538d1eab96c97866b24546c453d95822f24cf9c6638b42ba523eb7aa441cb26  guix-build-5b4703c6a70d/output/x86_64-apple-darwin18/bitcoin-5b4703c6a70d-osx-unsigned.dmg
  d1b73133f1da68586b07292a8425f7f851e93f599c016376f23728c041cf39cc  guix-build-5b4703c6a70d/output/x86_64-apple-darwin18/bitcoin-5b4703c6a70d-osx-unsigned.tar.gz
  5ad94c5f8a5f29405955ff3ab35d137de1acc04398d6c8298fb187b57a6e316a  guix-build-5b4703c6a70d/output/x86_64-apple-darwin18/bitcoin-5b4703c6a70d-osx64.tar.gz
  8c6d7b3f847faa7b4d16ceecf228f26f146ea982615c1d7a00c57f9230a0c484  guix-build-5b4703c6a70d/output/x86_64-linux-gnu/SHA256SUMS.part
  d0a8c99750319ad8046cfa132a54e5c13a08351f94439ae9af0f8e5486c2c2ea  guix-build-5b4703c6a70d/output/x86_64-linux-gnu/bitcoin-5b4703c6a70d-x86_64-linux-gnu-debug.tar.gz
  d816bb26dd4b0e309f2f576b1cccc6d78743fb2f357daad2da09bb1177330971  guix-build-5b4703c6a70d/output/x86_64-linux-gnu/bitcoin-5b4703c6a70d-x86_64-linux-gnu.tar.gz
  65caaa7f648c7eab1eb82c3331a2ca25b8cd4fe41439de55604501e02571de55  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/SHA256SUMS.part
  5bf6f7328cbceb0db22a2d7babb07b60cb6dcc19a6db84a1698589b7f5173a06  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/bitcoin-5b4703c6a70d-win-unsigned.tar.gz
  7aabcb56115decef78d3797840b6e49dbc9b202d56f892490e92616fb06fec9e  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/bitcoin-5b4703c6a70d-win64-debug.zip
  2f369694648ff9dc5ca1261a1e5874b1c7408ccf2802f9caef56c1334e8a5b7c  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/bitcoin-5b4703c6a70d-win64-setup-unsigned.exe
  1c1f92513c4aad38419ff49a7b80bf10e6b1eca01ee8c5e3b2acd1768cf1e3d5  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/bitcoin-5b4703c6a70d-win64.zip
  ```

ACKs for top commit:
  hebasto:
    Approach ACK 5b4703c6a7.

Tree-SHA512: 2cd92a245ea64ef7176cf402a1fa5348a9421c30a4d30d01c950c48f6dcc15cf22ce69ffe1657be97e5fccc14bd933d64683c4439b695528ce3dc34d72dda927
2021-07-09 10:20:16 +08:00
Russell Yanofsky
1ee6d0b01a gui: Prompt to reset settings when settings.json cannot be read
Fixes bitcoin/bitcoin#21340

Co-authored-by: Jarol Rodriguez <jarolrod@tutanota.com>
2021-07-08 14:57:03 -04:00
MarcoFalke
df2b5dadfb
Merge bitcoin/bitcoin#22408: test: add tests for bad-txns-prevout-null reject reason
1f449586a9 test: add `bad-txns-prevout-null` test to mempool_accept.py (Sebastian Falbesoner)
aa0a5bb70d test: add `bad-txns-prevout-null` test case to invalid_txs.py (Sebastian Falbesoner)

Pull request description:

  This simple PR adds missing tests for the reject reason `bad-txns-prevout-null`, which is thrown in the function `CheckTransaction()`: a62fc35a15/src/consensus/tx_check.cpp (L52-L54)

  Basically this condition is met for non-coinbase transactions (the code snippet above only hits if `!tx.IsCoinBase()`) with coinbase-like outpoints, i.e. hash=0, n=0xffffffff.

  Can be tested by running the functional tests `feature_block.py`, `p2p_invalid_tx.py` and `mempool_accept.py`. Not sure if the redundancy in the tests is desired (I guess it would make sense if the mempool acceptance test also makes use of the invalid_txs templates?).

ACKs for top commit:
  rajarshimaitra:
    tACK 1f449586a9
  brunoerg:
    tACK 1f449586a9
  kristapsk:
    ACK 1f449586a9, code looks correct and all tests pass.

Tree-SHA512: 2d4f940a6ac8e0d80d2670c9e1111cbf43ae6ac62809a2ccf17cffee9a41d387ea4d889ee300eb4a407c055b13bfa5d37102a32ed59964a9b6950bd907ba7204
2021-07-08 17:43:14 +02:00
Carl Dong
46ce6ce378 tree-wide: Rename gitian-keys to builder-keys 2021-07-08 11:29:54 -04:00
Carl Dong
fc4f8449f3 guix: Update various check_tools lists 2021-07-08 11:29:26 -04:00
Carl Dong
263220a85c guix: Check for a sane services database
On bare systems, it is possible to be lacking a services database. Check
for basic entries before attempting a build.

See the error message in the diff for more context.
2021-07-08 11:28:07 -04:00
W. J. van der Laan
d968616422
Merge bitcoin/bitcoin#22179: Torv2 removal followups
00b875ba94 addrman: remove invalid addresses when unserializing (Vasil Dimov)
bdb62096f0 fuzz: reduce possible networks check (Vasil Dimov)
a164cd3ba6 net: simplify CNetAddr::IsRoutable() (Vasil Dimov)

Pull request description:

  * Simplify some code, now that we know `CNetAddr::IsRFC4193()` and `CNetAddr::IsTor()` cannot be `true` at the same time.
  * Drop Tor v2 addresses when loading addrman from `peers.dat` - they would have been loaded as dummy-all-zeros IPv6 addresses and linger in addrman, wasting space.

ACKs for top commit:
  sipa:
    ACK 00b875ba94. Reviewed the code, and tested with -DDEBUG_ADDRMAN (unit tests + mainnet run with peers.dat that contained v2 onions).
  laanwj:
    Code review and lightly tested ACK 00b875ba94
  jonatack:
    ACK 00b875ba94 reviewed, debug-built with -DEBUG_ADDRMAN rebased to current master, restarted node on mainnet/signet/testnet and verified that on each chain -addrinfo shows no change in address counts (as expected). Added some sanity check asserts, rebuilt/re-ran test. Checked that the new test fails on master with "test/addrman_tests.cpp(824): error: in "addrman_tests/remove_invalid": check addrman.size() == 2 has failed [4 != 2]"
  jarolrod:
    ACK 00b875ba94

Tree-SHA512: 6ed8e6745134b1b94fffaba28482de909ea39483b46b7f57bda61cdbae7a51251d15cb674de3631772fbeabe153d77a19269f96e62a89102a2d5c01e48f0ba06
2021-07-08 17:20:35 +02:00
Vasil Dimov
b1d905c225
p2p: earlier continuation when no remaining eviction candidates
in ProtectEvictionCandidatesByRatio().

With this change, `if (n.count == 0) continue;` will be true
if a network had candidates protected in the first iterations
and has no candidates remaining to be protected in later iterations.

Co-authored-by: Jon Atack <jon@atack.com>
2021-07-08 12:28:40 +02:00
Jon Atack
c9e8d8f9b1
p2p: process more candidates per protection iteration
for the usual case when some of the protected networks
don't have eviction candidates, to reduce the number
of iterations in ProtectEvictionCandidatesByRatio().

Picks up an idea in ef411cd2 that I had dropped.
2021-07-08 12:28:38 +02:00
Jon Atack
02e411ec45
p2p: iterate eviction protection only on networks having candidates
in ProtectEvictionCandidatesByRatio().

Thank you to Vasil Dimov, whose suggestions during a post-merge
discussion about PR 21261 reminded me that I had done this in
earlier versions of the PR, e.g. commits like ef411cd2.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-07-08 12:28:35 +02:00
Jon Atack
5adb064574
bench: add peer eviction protection benchmarks
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-07-08 12:28:23 +02:00
glozow
b7a8cd9963 [test] submit same txid different wtxid as mempool tx
Co-authored-by: Antoine Riard <ariard@student.42.fr>
Co-authored-by: Antoine Riard <antoine.riard@gmail.com>
2021-07-08 09:31:45 +01:00
glozow
fdb48163bf [validation] distinguish same txid different wtxid in mempool
Changes behavior.
2021-07-08 09:31:45 +01:00
Dhruv Mehta
a806647d26 [validation] Always include merkle root in coinbase commitment 2021-07-07 22:13:01 -07:00
Dhruv Mehta
189128c220 [validation] Set witness script flag with p2sh for blocks 2021-07-07 22:13:01 -07:00
Dhruv Mehta
ac82b99db7 [p2p] remove redundant NODE_WITNESS checks
nLocalServices defaults to NODE_WITNESS and these checks are obsolete
2021-07-07 22:13:01 -07:00
Dhruv Mehta
6f8b198b82 [p2p] remove unused segwitheight=-1 option
This also lets us default to NODE_WITNESS in nLocalServices
2021-07-07 22:13:01 -07:00
Dhruv Mehta
eba5b1cd64 [test] remove or move tests using -segwitheight=-1 2021-07-07 22:12:04 -07:00
Vasil Dimov
2feec3ce31
net: don't bind on 0.0.0.0 if binds are restricted to Tor
The semantic of `-bind` is to restrict the binding only to some address.
If not specified, then the user does not care and we bind to `0.0.0.0`.
If specified then we should honor the restriction and bind only to the
specified address.

Before this change, if no `-bind` is given then we would bind to
`0.0.0.0:8333` and to `127.0.0.1:8334` (incoming Tor) which is ok -
the user does not care to restrict the binding.

However, if only `-bind=addr:port=onion` is given (without ordinary
`-bind=`) then we would bind to `addr:port` _and_ to `0.0.0.0:8333` in
addition.

Change the above to not do the additional bind: if only
`-bind=addr:port=onion` is given (without ordinary `-bind=`) then bind
to `addr:port` (only) and consider incoming connections to that as Tor
and do not advertise it. I.e. a Tor-only node.
2021-07-07 15:46:38 +02:00
fanquake
4129134e84
Merge bitcoin/bitcoin#22402: doc: Install Rosetta on M1-macOS for qt in depends
cdb41d5573 doc: Install Rosetta on M1-macOS for qt in depends (Hennadii Stepanov)

Pull request description:

  On master (c609e10545) `make -C depends qt` on Apple Silicon based macOS 11.4 ends with an error:
  ```
  /bin/sh: /Users/hebasto/bitcoin/depends/work/build/aarch64-apple-darwin20.5.0/qt/5.12.11-6c4d47a8f8f/qtbase/bin/moc: Bad CPU type in executable
  ```

  Installing Rosetta 2 fixes it.

  Explanation. On Apple Silicon macOS the `qt` package in depends actually is cross compiled. All native tools (including `moc`) are x86_64 binaries, that require Rosetta 2 to run.

ACKs for top commit:
  promag:
    ACK cdb41d5573.
  fanquake:
    ACK cdb41d5573 - I have not tested after installing Rosetta 2, but I saw the same issue during my first cross-compile on an M1 box.
  Zero-1729:
    ACK cdb41d5573

Tree-SHA512: fb06a32d6fb40f405ce856b44f5d3af0c51089886f3be79e509e5c325614d7af58ce4480c064c17e0efb695a1f69f68d533c417f9631d46d8a630aba60ce4433
2021-07-07 20:59:35 +08:00
Carl Dong
5b4703c6a7
guix: Test security-check sanity before performing them 2021-07-07 20:29:05 +08:00
fanquake
6cf3345297
scripts: adjust test-symbol-check for guix release environment
Now that our release binaries are build in a glibc 2.24 and 2.27
environment, we can't use a symbol from glibc 2.28 to test our checks.
Replace renameat2() with nextup(), which was introduced in 2.24.

Note that this also means re-disabling the test for RISC-V, however
RISC-V is built in a glibc 2.27 environment, and our minimum required
glibc for that binary is 2.27.
2021-07-07 20:29:01 +08:00
fanquake
1946b5f77c
scripts: more robustly test macOS symbol checks 2021-07-07 19:31:37 +08:00
Carl Dong
a8127b34bc
build: Use and test PE binutils with --reloc-section
Also fix test-security-check.py to account for new PE PIE failure
indication.
2021-07-07 19:31:37 +08:00
Carl Dong
678348db51
guix: Patch binutils to add security-related disable flags
We use these flags in our test-security-check make target, but they are
only available because debian patches them in.

We can patch them in for our Guix builds so that we can check the sanity
of our security/symbol checking suite before running them.
2021-07-07 19:31:37 +08:00
Carl Dong
9fdc8afe11
devtools: Improve *-check.py tool detection
This is important to make sure that we're not testing tools different
from the one we're building with.

Introduce determine_wellknown_cmd, which encapsulates how we
should handle well-known tools specification (IFS splitting, env
override, etc.).
2021-07-07 19:31:37 +08:00
fanquake
bda62eab38
ci: skip running the Linux test-security-check target for now
The CI environment is a moving target, and these tests are somewhat
fragile, so for now, disable them.
2021-07-07 19:31:37 +08:00
Carl Dong
d6ef3543ae
lint: Run mypy with --show-error-codes
When using mypy ignore directives, the error code needs to be specified.
Somehow mypy doesn't print it by default...
2021-07-07 19:30:47 +08:00
fanquake
137f3b9ba3
Merge bitcoin/bitcoin#22405: build: remove --enable-glibc-back-compat from Guix build
797b3ed909 script: remove gitian reference from symbol-check.py (fanquake)
15fc9a0299 guix: add additional documentation to patches (fanquake)
4516e5ec92 lint: exclude Guix patches from spell-checking (fanquake)
de6ca41a52 guix: no-longer pass --enable-glibc-back-compat to Guix (fanquake)
84dd81fb5b build: remove glibc backcompat requirement for Linux symbol checks (fanquake)

Pull request description:

  Now that our Guix toolchains are based on glibc 2.24 and 2.27 (RISCV), we don't need to use the `--enable-glibc-back-compat` option to produce binaries that don't use any symbols from glibc 2.17 and 2.27 or later.

  This also adds additional documentation to some Guix patches (pointed out in #22365) and removes Guix patches from the spelling linter, because that isn't our spelling.

  Symbol usage: https://gist.github.com/fanquake/d15604fc580718444c5aa4b3c3c75fdc.

  Guix Builds:
  ```bash
  bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  ed54e6a6cf4fab328557c0c72eb08c73f2a58c6c70959544cf4b1882e75ea69e  guix-build-797b3ed90900/output/aarch64-linux-gnu/SHA256SUMS.part
  83bd9dadc59f89f848d143fa4fc3964f16fe0b4bdf35e5093b577ff2c4bd1f43  guix-build-797b3ed90900/output/aarch64-linux-gnu/bitcoin-797b3ed90900-aarch64-linux-gnu-debug.tar.gz
  94cb8c35281f12dec6ea5b390b66cad5e27ac8c45a30c42c8d38c438695d54c0  guix-build-797b3ed90900/output/aarch64-linux-gnu/bitcoin-797b3ed90900-aarch64-linux-gnu.tar.gz
  7318b63d65c0aa52d2446de8e1f40658d2e47ab8fb0268820c3b7585d140fb23  guix-build-797b3ed90900/output/arm-linux-gnueabihf/SHA256SUMS.part
  95e1ffb372964b73f539653ca703b70cf0c018801a9c4c0ffc46a0b63539253c  guix-build-797b3ed90900/output/arm-linux-gnueabihf/bitcoin-797b3ed90900-arm-linux-gnueabihf-debug.tar.gz
  039d3842e6499626cf955ae0a7590dd6b3d0935cdc217c98aaf9d156b0ebd3b4  guix-build-797b3ed90900/output/arm-linux-gnueabihf/bitcoin-797b3ed90900-arm-linux-gnueabihf.tar.gz
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  guix-build-797b3ed90900/output/dist-archive/SKIPATTEST.TAG
  2c4e7b6e7aff63ba811e5bf59362d16866c3a358f8844fba8739a61192870622  guix-build-797b3ed90900/output/dist-archive/bitcoin-797b3ed90900.tar.gz
  955029b949c368eabd517dd33040d2f01e2ac6a55e7b4f9107907a7c6e0c6060  guix-build-797b3ed90900/output/powerpc64-linux-gnu/SHA256SUMS.part
  fd6d6b137f8efedf58a879d11205b1d4649e1f97d7f91e193239ef206fcc285d  guix-build-797b3ed90900/output/powerpc64-linux-gnu/bitcoin-797b3ed90900-powerpc64-linux-gnu-debug.tar.gz
  51736ac8e77737999f1b5bd4c381b0016f19a8d5e40e786fe941ff04e84c11c9  guix-build-797b3ed90900/output/powerpc64-linux-gnu/bitcoin-797b3ed90900-powerpc64-linux-gnu.tar.gz
  8c244c16bfa46c1efdb120e1d91fdd14d3f14eefee8d7e1fbb0a9b4664a5c315  guix-build-797b3ed90900/output/powerpc64le-linux-gnu/SHA256SUMS.part
  704ee593251a1b1c65a5bebeef93b23f266af4e8cbf8ae556150c3b2e8f06a6c  guix-build-797b3ed90900/output/powerpc64le-linux-gnu/bitcoin-797b3ed90900-powerpc64le-linux-gnu-debug.tar.gz
  0ec06ae7d344de20d61e3965d8b383747ef20b0e9d93a3165733ea23bdf2ead8  guix-build-797b3ed90900/output/powerpc64le-linux-gnu/bitcoin-797b3ed90900-powerpc64le-linux-gnu.tar.gz
  2dd6c6ecc67b0ea40ca9c43f92efca81ccd054b8db8c197ad84ad9674d510a25  guix-build-797b3ed90900/output/riscv64-linux-gnu/SHA256SUMS.part
  5ebb27a855a677f7a188d83995be6b2a3ea8606be152abb7fc7832713fb0677a  guix-build-797b3ed90900/output/riscv64-linux-gnu/bitcoin-797b3ed90900-riscv64-linux-gnu-debug.tar.gz
  bdaf1783f5e1861597afa37c1880364e118d9a7a7af8017302d82202791019f6  guix-build-797b3ed90900/output/riscv64-linux-gnu/bitcoin-797b3ed90900-riscv64-linux-gnu.tar.gz
  726c9092b60ac2e7d7e14b2c24467fcf276a6f89170a871ddab9dce6ac230699  guix-build-797b3ed90900/output/x86_64-apple-darwin18/SHA256SUMS.part
  2af4d709b44952654f3c08c86593bf2ccc9a44ed422783a1b95b8a199a894db2  guix-build-797b3ed90900/output/x86_64-apple-darwin18/bitcoin-797b3ed90900-osx-unsigned.dmg
  fd49ba445aa6cf3d8c47019a05e9e5740cb0f53349344dd80671297127f49f1a  guix-build-797b3ed90900/output/x86_64-apple-darwin18/bitcoin-797b3ed90900-osx-unsigned.tar.gz
  3f51cbf8cf18420d4be70e656aa993675cf5e828a255c2030047ae2e059ed5b7  guix-build-797b3ed90900/output/x86_64-apple-darwin18/bitcoin-797b3ed90900-osx64.tar.gz
  afd1edee1447bb88d81e972abfae4c4e065b5b1827769f033cff9472084c7c1b  guix-build-797b3ed90900/output/x86_64-linux-gnu/SHA256SUMS.part
  ec468ef886d25e685f4f7a18b4f7d497dedf757495e0d5beb72c23cc32ab69b5  guix-build-797b3ed90900/output/x86_64-linux-gnu/bitcoin-797b3ed90900-x86_64-linux-gnu-debug.tar.gz
  1934d7294f0c9e083d38a3f68d4a61cd679defa79ce0a89f77386978692b9b18  guix-build-797b3ed90900/output/x86_64-linux-gnu/bitcoin-797b3ed90900-x86_64-linux-gnu.tar.gz
  94c11c328a628052eb6f50e9816aa768f87ea7acfbbbafdab60f6928da766811  guix-build-797b3ed90900/output/x86_64-w64-mingw32/SHA256SUMS.part
  fd371922ba93d81bd4a2b711d617af6756f9f0494db6d83aa0e5f491a24168ef  guix-build-797b3ed90900/output/x86_64-w64-mingw32/bitcoin-797b3ed90900-win-unsigned.tar.gz
  4e4ad976bc029bbbf9596ad8493accaaba8b0d5c598dd342f8da330609bbdf21  guix-build-797b3ed90900/output/x86_64-w64-mingw32/bitcoin-797b3ed90900-win64-debug.zip
  3a89a16b9101e9a17d98efb9234b5bdd264c0bba2c6326511017730e1a08311f  guix-build-797b3ed90900/output/x86_64-w64-mingw32/bitcoin-797b3ed90900-win64-setup-unsigned.exe
  e285ab737e3c843fd3f1c26c2f053e421a3c39b33995747ce48281884d3f28d1  guix-build-797b3ed90900/output/x86_64-w64-mingw32/bitcoin-797b3ed90900-win64.zip
  ```

ACKs for top commit:
  sipa:
    utACK 797b3ed909
  hebasto:
    ACK 797b3ed909

Tree-SHA512: 3a569702d8832c155c5ce8d2f6d823f7f12603885576078bc5192bc9038a48261ecb541800f79d1e9bc86d71fa640265c5b8b89df9d8bb680b3bb05d9d78a666
2021-07-07 19:21:57 +08:00
MarcoFalke
914c0cad97
Merge bitcoin/bitcoin#22399: fuzz: Rework CTxDestination fuzzing
fabf17056c fuzz: Move CTxDestination fuzzing to script fuzz target (MarcoFalke)
fa42800a51 fuzz: Simplify CTxDestination fuzzing in the script target (MarcoFalke)
fab99865c0 fuzz: Improve ConsumeTxDestination (MarcoFalke)
fa40c0964b fuzz: Move ConsumeTxDestination to cpp file (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    cr ACK fabf17056c

Tree-SHA512: afd2cf384d04a810c0c462c6d80849bd0fefd017d7acac877f64f2bffae3fc8d687701bc479e67a727a05f43431a17cb4ccaf09c6b3c68106562c94b7ed19250
2021-07-07 12:03:49 +02:00
fanquake
692b78793b
Merge bitcoin/bitcoin#22360: doc: Remove unused section from release process
fa051d3e5e doc: Remove unused section from release process (MarcoFalke)

Pull request description:

  The section does not apply to us, because it is being done by the maintainers of the `bitcoin.org` website, see https://github.com/bitcoin-dot-org/Bitcoin.org/issues/3641#issuecomment-819586576 . Documentation about the process is in their repo: https://github.com/bitcoin-dot-org/bitcoin.org/blob/master/docs/adding-events-release-notes-and-alerts.md#release-notes

ACKs for top commit:
  jarolrod:
    ACK fa051d3e5e
  Zero-1729:
    ACK fa051d3e5e
  fanquake:
    ACK fa051d3e5e

Tree-SHA512: eadc192958bb0d1009ad94c2ec79087b87514e3d19e2e6d759440392fa4abc432c512ba60692018e789c822a98a515a5f9a42966de4f8fe6b0a8b99470a76b5c
2021-07-07 14:45:41 +08:00
Pieter Wuille
1edddf5de4 Avoid GCC 7.1 ABI change warning in guix build 2021-07-06 10:41:43 -07:00
Jon Atack
566357f8f7
refactor: move GetRandomNodeEvictionCandidates() to test utilities
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-07-06 17:28:15 +02:00
Brian Liotti
fa84caebc7
doc: added info to bitcoin.conf doc 2021-07-06 09:32:37 -04:00
fanquake
797b3ed909
script: remove gitian reference from symbol-check.py 2021-07-06 20:50:47 +08:00
fanquake
15fc9a0299
guix: add additional documentation to patches 2021-07-06 20:50:47 +08:00
fanquake
4516e5ec92
lint: exclude Guix patches from spell-checking
Co-authored-by: Carl Dong <contact@carldong.me>
2021-07-06 20:50:46 +08:00
fanquake
de6ca41a52
guix: no-longer pass --enable-glibc-back-compat to Guix
Now that our Guix builds are performed on glibc 2.24 and 2.27 (RISCV),
we no-longer need to pass the --enable-glibc-back-compat option.

Replace it with --disable-threadlocal, to prevent the usage of symbols
from glibc 2.18.

None of the binaries produced required symbols later than 2.17, and 2.27
(RISCV).
2021-07-06 20:50:40 +08:00
fanquake
84dd81fb5b
build: remove glibc backcompat requirement for Linux symbol checks 2021-07-06 15:21:17 +08:00