Commit Graph

25298 Commits

Author SHA1 Message Date
Jonas Schnelli
386ec192a5 Reduce cs_main lock accumulation during GUI startup 2020-08-12 16:44:09 +02:00
Jonas Schnelli
d42cb79068 Optionally populate BlockAndHeaderTipInfo during AppInitMain 2020-08-12 16:44:09 +02:00
Jonas Schnelli
b354a1480a Add BlockAndHeaderTipInfo to the node interface/appInit 2020-08-12 16:44:06 +02:00
Jonas Schnelli
25e1d0bf41 RPCConsole, take initial chaintip data as parameter 2020-08-12 16:31:23 +02:00
fanquake
ce3bdd0ed1
Merge #19316: [net] Cleanup logic around connection types
01e283068b [net] Remove unnecessary default args on CNode constructor (Amiti Uttarwar)
bc5d65b3ca [refactor] Remove IsOutboundDisconnectionCandidate (Amiti Uttarwar)
2f2e13b6c2 [net/refactor] Simplify multiple-connection checks (Amiti Uttarwar)
7f7b83deb2 [net/refactor] Rework ThreadOpenConnections logic (Amiti Uttarwar)
35839e963b [net] Fix bug where AddrFetch connections would be counted as outbound full relay (Amiti Uttarwar)
4972c21b67 [net/refactor] Clarify logic for selecting connections in ThreadOpenConnections (Amiti Uttarwar)
60156f5fc4 [net/refactor] Remove fInbound flag from CNode (Amiti Uttarwar)
7b322df629 [net/refactor] Remove m_addr_fetch member var from CNode (Amiti Uttarwar)
14923422b0 [net/refactor] Remove fFeeler flag from CNode (Amiti Uttarwar)
49efac5cae [net/refactor] Remove m_manual_connection flag from CNode (Amiti Uttarwar)
d3698b5ee3 [net/refactor] Add connection type as a member var to CNode (Amiti Uttarwar)
46578c03e9 [doc] Describe different connection types (Amiti Uttarwar)
442abae2ba [net/refactor] Add AddrFetch connections to ConnectionType enum (Amiti Uttarwar)
af59feb052 [net/refactor] Extract m_addr_known logic from initializer list (Amiti Uttarwar)
e1bc29812d [net/refactor] Add block relay only connections to ConnectionType enum (Amiti Uttarwar)
0e52a659a2 [net/refactor] Add feeler connections to ConnectionType enum (Amiti Uttarwar)
1521c47438 [net/refactor] Add manual connections to ConnectionType enum (Amiti Uttarwar)
26304b4100 [net/refactor] Introduce an enum to distinguish type of connection (Amiti Uttarwar)
3f1b7140e9 scripted-diff: Rename OneShot to AddrFetch (Amiti Uttarwar)

Pull request description:

  **This is part 1 of #19315, which enables the ability to test `outbound` and `block-relay-only` connections from the functional tests.** Please see that PR for more information of overall functionality.

  **This PR simplifies how we manage different connection types.** It introduces an enum with the various types of connections so we can explicitly define the connection type. The existing system relies on a series of independent flags, then has asserts scattered around to ensure that conflicting flags are not enabled at the same time. I find this approach to be both brittle and confusing. While making these changes, I found a small bug due to the silent assumptions.

  This PR also proposes a rename from `OneShot` to `AddrFetch`. I find the name `OneShot` to be very confusing, especially when we also have `onetry` manual connections. Everyone I've talked to offline has agreed that the name is confusing, so I propose a potential alternative. I think this is a good opportunity for a rename since I'm creating an enum to explicitly define the connection types.
  (some context for the unfamiliar: `oneshot` or `addrfetch` connections are short-lived connections created on startup. They connect to the seed peers, send a `getaddr` to solicit addresses, then close the connection.)

  Overview of this PR:
  * rename `oneshot` to `addrfetch`
  * introduce `ConnectionType` enum
  * one by one, add different connection types to the enum
  * expose the `conn_type` on CNode, and use this to reduce reliance on flags (& asserts)
  * fix the bug in counting different type of connections
  * some additional cleanup to simplify logic and make expectations explicit/inclusive rather than implicit/exclusive.

ACKs for top commit:
  jnewbery:
    utACK 01e283068b
  laanwj:
    Code review ACK 01e283068b, the commits are pretty straightforward to follow, and I think this is a move in the right direction overall
  vasild:
    ACK 01e283068
  sdaftuar:
    ACK 01e283068b.
  fanquake:
    ACK 01e283068b - I don't have as much experience with the networking code but these changes look fairly straight forward, the new code seems more robust/understandable and the additional documentation is great. I'm glad that a followup branch is already underway. There might be some more review comments here later today, so keep an eye on the discussion, however I'm going to merge this now.
  jb55:
    wow this code was messy before... ACK 01e283068b

Tree-SHA512: 7bb644a6ed5849913d777ebc2ff89133ca0fbef680355a9a344e07496a979e6f9ff21a958e8eea93dcd7d5c343682b0c7174b1a3de380a4247eaae73da436e15
2020-08-12 10:01:44 +08:00
fanquake
cb1ee1551c
Merge #19674: refactor: test: use throwaway _ variable for unused loop counters
dac7a111bd refactor: test: use _ variable for unused loop counters (Sebastian Falbesoner)

Pull request description:

  This tiny PR substitutes Python loops in the form of `for x in range(N): ...` by `for _ in range(N): ...` where applicable. The idea is indicating to the reader that a block (or statement, in list comprehensions) is just repeated N times, and that the loop counter is not used in the body, hence using the throwaway variable. This is already done quite often in the current tests (see e.g. `$ git grep "for _ in range("`). Another alternative would be using `itertools.repeat` (according to Python core developer Raymond Hettinger it's [even faster](https://twitter.com/raymondh/status/1144527183341375488)), but that doesn't seem to be widespread in use and I'm not sure about a readability increase.

  The only drawback I see is that whenever one wants to debug loop iterations, one would need to introduce a loop variable again. Reviewing this is basically a no-brainer, since tests would fail immediately if a a substitution has taken place on a loop where the variable is used.

  Instances to replace were found by `$ git grep "for.*in range("` and manually checked.

ACKs for top commit:
  darosior:
    ACK dac7a111bd
  instagibbs:
    manual inspection ACK dac7a111bd
  practicalswift:
    ACK dac7a111bd -- the updated code is easier to reason about since the throwaway nature of a variable is expressed explicitly (using the Pythonic `_` idiom) instead of implicitly. Explicit is better than implicit was we all know by now :)

Tree-SHA512: 5f43ded9ce14e5e00b3876ec445b90acda1842f813149ae7bafa93f3ac3d510bb778e2c701187fd2c73585e6b87797bb2d2987139bd1a9ba7d58775a59392406
2020-08-11 09:24:50 +08:00
Wladimir J. van der Laan
85fa648c85
Merge #19596: Deduplicate parent txid loop of requested transactions and missing parents of orphan transactions
4c0731f9c5 Deduplicate missing parents of orphan transactions (Suhas Daftuar)
8196176243 Rewrite parent txid loop of requested transactions (Suhas Daftuar)

Pull request description:

  I noticed a couple of places recently where we loop over all inputs of a transaction in order to do some processing on the txids we find in those inputs.  There may be thousands of inputs in a transaction, and the same txid may appear many times.  In a couple of places in particular, we loop over those txids and add them to a rolling bloom filter; doing that multiple times for the same txid wastes entries in that filter.

  This PR fixes that in two places relating to transaction relay: one on the server side, where we look for parent transactions of a tx that we are delivering to a peer to ensure that getdata requests for those parents will succeed; and the other on the client side, where when we process an orphan tx we want to loop over the parent txids and ensure that all are eventually requested from the peer who provided the orphan.

  This addresses a couple of [related](https://github.com/bitcoin/bitcoin/pull/19109#discussion_r455197217) [comments](https://github.com/bitcoin/bitcoin/pull/19109#discussion_r456820373) left in #19109.

ACKs for top commit:
  laanwj:
    Code review ACK 4c0731f9c5
  jonatack:
    ACK 4c0731f9c5
  ajtowns:
    ACK 4c0731f9c5

Tree-SHA512: 8af9df7f56c6e54b5915519d7d5465e081473ceb1bcc89bbebf83e78722cf51ff58145e588cf57126bce17071a8053273f4bcef0ad8166bec83ba14352e40f5d
2020-08-10 20:38:19 +02:00
fanquake
6d5a9fec2d
Merge #19605: doc: set CC_FOR_BUILD when building on OpenBSD
01cd24c226 doc: set CC_FOR_BUILD when building on OpenBSD (fanquake)

Pull request description:

  Closes: #19559

  While #19559 has been fixed upstream, it makes sense to not only
  recommend using `CC_FOR_BUILD`here  until the fix is pulled in as
  part of our next libsecp update, but after discussing with Cory,
  he suggested we should be setting this on OpenBSD (which still has
  the an ancient GCC) regardless.

ACKs for top commit:
  real-or-random:
    ACK 01cd24c226 I looked at the diff (but can't test the instructions on OpenBSD)
  laanwj:
    Code review ACK 01cd24c226

Tree-SHA512: 322802b9303771f1be2ad9628f268dfa71dc7ee77948fa2a34f21eceb19b2d8efdd8876c8f0778adbfcde48fa0f88cd4e698ae425428159abca38e8c7980da1d
2020-08-10 22:39:11 +08:00
fanquake
04a0676b3f
Merge #17563: lib: fix a compiler warning: unused GetDevURandom()
ca2e474372 Fix a compiler warning: unused GetDevURandom() (Vasil Dimov)

Pull request description:

  ~~Only define GetDevURandom() if it is going to be used.~~

  Silence by planting a dummy reference to the `GetDevURandom` symbol
  in the places where we don't call the function.

ACKs for top commit:
  practicalswift:
    ACK ca2e474372 -- increased signal to noise in compiler diagnostics is good
  sipa:
    utACK ca2e474372
  hebasto:
    re-ACK ca2e474372, tested on macOS 10.15.6 + llvm clang 10.0.0

Tree-SHA512: 03c98f00dad5d9a3c5c9f68553d72ad5489ec02f18b9769108a22003ec7be7819a731b1eab6a9f64dafb5be0efddccf6980de7e3bb90cd20d4f4d72f74124675
2020-08-10 21:30:42 +08:00
fanquake
f306384f5a
Merge #19622: build: Drop ancient hack in gitian-linux descriptor
33a84e8f40 build: Update and sort package list in gitian-linux.yml (Hennadii Stepanov)
95051682be build: Drop old hack which is unneeded now (Hennadii Stepanov)

Pull request description:

  The hack was aimed to fix an issue in Ubuntu Trusty 14.04 (see #8188).
  The current hack implementation was added in #8315.

  On master (8db23349fe) this hack is effectively noop, and it is no longer needed.

  I see this PR as a step to removing `libfaketime` from gitian builds.

ACKs for top commit:
  dongcarl:
    tACK 33a84e8f40
  laanwj:
    Code review ACK 33a84e8f40

Tree-SHA512: 90036c555a500649ccc3d108bf11f09a9cfd2c92c0b598f7e0c0df63a713ae7abaf78f350b68c025470619c967223f45f6a235ad37a6ce1d1a0341ed34963ba0
2020-08-10 20:15:09 +08:00
Wladimir J. van der Laan
be11f94e95
Merge #19631: test: Wait for 'cmpctblock' in p2p_compactblocks when it is expected
9e165d0de4 test: Wait for 'cmpctblock' in p2p_compactblocks when it is expected (Ben Woosley)

Pull request description:

  This is a more narrowly-construed wait which eliminates the possibility of the
  wait being triggered by other messages.

  Note `received_block_announcement` reflect three possible messages:
  edec7f7c25/test/functional/p2p_compactblocks.py (L34-L53)

  Prompted by looking into: #19449

ACKs for top commit:
  laanwj:
    Code review ACK 9e165d0de4
  theStack:
    ACK 9e165d0de4

Tree-SHA512: bc4a9c8bf031c8a7efb40d9625feaa3fd1f56f3b75da7034944af71ccea44328a6c708ab0c13fea85fb7cf4fd9043fe90eb94a25e95b2d42be44c2962b4904ce
2020-08-09 18:54:42 +02:00
Wladimir J. van der Laan
5e4fed9e58
Merge #19657: test: Wait until is_connected in add_p2p_connection
fa4dfd215f test: Wait until is_connected in add_p2p_connection (MarcoFalke)

Pull request description:

  Moving the wait_until from the individual test scripts to the test framework simplifies two tests

ACKs for top commit:
  jnewbery:
    Code review ACK fa4dfd215f
  theStack:
    ACK fa4dfd215f ☕

Tree-SHA512: 36eda7eb323614a4c4f9215f1d7b40b9f9c4036d1c08eb701ea705f3e2986fdabd2fc558965a6aadabeed861034aeaeef3c00f968ca17ed7a27e42e506cda87d
2020-08-09 18:35:31 +02:00
Wladimir J. van der Laan
214e6655c3
Merge #19649: Restore test case for p2p transaction blinding
566aada386 Test that wtxid relay peers add wtxid to reject filter (Gregory Sanders)
0fea6ede1b Restore test case for p2p transaction blinding (Gregory Sanders)

Pull request description:

  Introduced in ca10a03add then erroneously removed in 8d8099e97a. The restored line is how we are
  checking that the node will still re-request a specific txid given a witness-related failure.

ACKs for top commit:
  fjahr:
    tACK 566aada386

Tree-SHA512: be2b75b5eddb88019b79cc798f9922ca7347ccbb2210b8d4eae93fdde62e2cbb614b5247cb2fbd7ee3577dbe053875a9b62c5747aace8617f12790b8fccdeab4
2020-08-09 18:30:25 +02:00
Wladimir J. van der Laan
b75f2ad72d
Merge #19660: refactor: Make HexStr take a span
0a8aa626dd refactor: Make HexStr take a span (Wladimir J. van der Laan)

Pull request description:

  Make `HexSt`r take a span of bytes, instead of an awkward pair of templated iterators. This simplifies most of the uses.

ACKs for top commit:
  elichai:
    Code review ACK 0a8aa626dd
  hebasto:
    re-ACK 0a8aa626dd
  jonatack:
    re-ACK 0a8aa626dd

Tree-SHA512: 6e178ece5cbac62119c857a10299b1e85422938084c3f03063e17119a5129e0c28016e05a6fabaa4c271a7e0a37c7cd89fa47c435ee19b38a5acfe80d00de992
2020-08-09 15:35:58 +02:00
Wladimir J. van der Laan
6ea7348122
Merge #19638: Replace hidden service with onion service
1e72b68ab3 Replace `hidden service` with `onion service` (Riccardo Masutti)

Pull request description:

  For a couple of years, Tor has made the term `hidden service` obsolete, in favor of `onion service`: [Tor Project | Onion Services](https://community.torproject.org/onion-services/)

  This PR updates all the references.

ACKs for top commit:
  laanwj:
    Code review ACK 1e72b68ab3
  hebasto:
    ACK 1e72b68ab3, tested on Linux Mint 20 (x86_64).

Tree-SHA512: 6a29e828e1c5e1ec934b5666f67326dbd84d77c8b2641f6740abac6d3d5923b7729763b9ff2230390b0bb23359a5f3731ccd9a30011ca69004f7c820aed17262
2020-08-09 15:08:20 +02:00
MarcoFalke
e349eeeb2c
Merge #19672: build: make clean removes .gcda and .gcno files from fuzz directory
90bd476ea6 build: make clean removes .gcda and .gcno files from fuzz directory (eugene)

Pull request description:

  I believe these should also be deleted upon invoking `make clean`. It also garbles the coverage file if you try to fuzz the same harness again.

ACKs for top commit:
  practicalswift:
    ACK 90bd476ea6 -- patch looks correct
  hebasto:
    ACK 90bd476ea6, tested with hints from #12602 and #18107.
  darosior:
    ACK 90bd476ea6

Tree-SHA512: 4b2eb664f64d18bc0385c5a0040b0b9fa6fe470c941ae39c7cb4544c4283427a8d4985517475fe0295c3ab2794b9a2ad4f76b6a443c05d846c97c966add87ca9
2020-08-08 16:43:03 +02:00
Amiti Uttarwar
01e283068b [net] Remove unnecessary default args on CNode constructor 2020-08-07 17:18:17 -07:00
Amiti Uttarwar
bc5d65b3ca [refactor] Remove IsOutboundDisconnectionCandidate 2020-08-07 17:18:17 -07:00
Amiti Uttarwar
2f2e13b6c2 [net/refactor] Simplify multiple-connection checks
Extract logic that check multiple connection types into interface functions &
structure as switch statements. This makes it very clear what touch points are
for accessing `m_conn_type` & using the switch statements enables the compiler
to warn if a new connection type is introduced but not handled for these cases.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
7f7b83deb2 [net/refactor] Rework ThreadOpenConnections logic
Make the connection counts explicit and extract into interface functions around
m_conn_type. Using explicit counting and switch statements where possible
should help prevent counting bugs in the future.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
35839e963b [net] Fix bug where AddrFetch connections would be counted as outbound full relay
The desired logic is for us to only open feeler connections after we have hit
the max count for outbound full relay connections.  A short lived AddrFetch
connection (previously called oneshot) could cause ThreadOpenConnections to
miscount and mistakenly open a feeler instead of full relay.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
4972c21b67 [net/refactor] Clarify logic for selecting connections in ThreadOpenConnections 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
60156f5fc4 [net/refactor] Remove fInbound flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
7b322df629 [net/refactor] Remove m_addr_fetch member var from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
14923422b0 [net/refactor] Remove fFeeler flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
49efac5cae [net/refactor] Remove m_manual_connection flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
d3698b5ee3 [net/refactor] Add connection type as a member var to CNode
- Directly maintaining the connection type prevents having to deduce it from
  several flags.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
46578c03e9 [doc] Describe different connection types 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
442abae2ba [net/refactor] Add AddrFetch connections to ConnectionType enum
- AddrFetch connections are short lived connections used to getaddr from a peer
- previously called "one shot" connections
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
af59feb052 [net/refactor] Extract m_addr_known logic from initializer list 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
e1bc29812d [net/refactor] Add block relay only connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
0e52a659a2 [net/refactor] Add feeler connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
1521c47438 [net/refactor] Add manual connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
26304b4100 [net/refactor] Introduce an enum to distinguish type of connection
- extract inbound & outbound types
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
3f1b7140e9 scripted-diff: Rename OneShot to AddrFetch
-BEGIN VERIFY SCRIPT-
sed -i 's/a oneshot/an addrfetch/g' src/chainparams.cpp #comment
sed -i 's/oneshot/addrfetch/g' src/net.cpp #comment
sed -i 's/AddOneShot/AddAddrFetch/g' src/net.h src/net.cpp
sed -i 's/cs_vOneShots/m_addr_fetches_mutex/g' src/net.h src/net.cpp
sed -i 's/vOneShots/m_addr_fetches/g' src/net.h src/net.cpp
sed -i 's/fOneShot/m_addr_fetch/g' src/net.h src/net.cpp src/net_processing.cpp
sed -i 's/ProcessOneShot/ProcessAddrFetch/g' src/net.h src/net.cpp
-END VERIFY SCRIPT-
2020-08-07 17:18:12 -07:00
Riccardo Masutti
1e72b68ab3 Replace hidden service with onion service
For a couple of years, Tor documentation has made
the term hidden service obsolete, in favor of onion
service.

This PR updates all the references in the code base.
2020-08-07 14:55:02 +02:00
MarcoFalke
4b705b1c98
Merge #19098: test: Remove duplicate NodeContext hacks
edc316020e test: Remove duplicate NodeContext hacks (Russell Yanofsky)

Pull request description:

  Qt tests currently are currently using two NodeContext structs at the same time, one in interfaces::NodeImpl::m_context, and the other in BasicTestingSetup::m_node, and the tests have hacks transferring state between them.

  Fix this by getting rid of the NodeImpl::m_context struct and making it a pointer. This way a common BitcoinApplication object can be used for all qt tests, but they can still have their own testing setups.

  Non-test code is changing but non-test behavior is still the same as before.

  Motivation for this PR is to be able to remove the "std::move(test.m_node.connman)" and mempool hacks for swapping individual NodeContext members in Qt tests, because followup PR #19099 adds yet another member (wallet_client) that needs to be swapped. After this change, the whole NodeContext struct can be swapped instead of individual members, so the workarounds are less fragile and invasive.

ACKs for top commit:
  MarcoFalke:
    crACK edc316020e 🌮
  promag:
    ACK edc316020e.

Tree-SHA512: c1650e4127f43a4020304ca7c13b5d9122fb5723aacd8fa1cf855d03c6052fcfb7685810aa2a5ef708561015f0022fecaacbad479295104ca45d2c17579466a4
2020-08-07 08:07:37 +02:00
fanquake
6d8543504d
Merge #19620: Add txids with non-standard inputs to reject filter
9f88ded82b test addition of unknown segwit spends to txid reject filter (Gregory Sanders)
7989901c7e Add txids with non-standard inputs to reject filter (Suhas Daftuar)

Pull request description:

  Our policy checks for non-standard inputs depend only on the non-witness
  portion of a transaction: we look up the scriptPubKey of the input being
  spent from our UTXO set (which is covered by the input txid), and the p2sh
  checks only rely on the scriptSig portion of the input.

  Consequently it's safe to add txids of transactions that fail these checks to
  the reject filter, as the witness is irrelevant to the failure. This is helpful
  for any situation where we might request the transaction again via txid (either
  from txid-relay peers, or if we might fetch the transaction via txid due to
  parent-fetching of orphans).

  Further, in preparation for future witness versions being deployed on the
  network, ensure that WITNESS_UNKNOWN transactions are rejected in
  AreInputsStandard(), so that transactions spending v1 (or greater) witness
  outputs will fall into this category of having their txid added to the reject
  filter.

ACKs for top commit:
  ajtowns:
    ACK 9f88ded82b - code review
  jnewbery:
    Code review ACK 9f88ded82b
  ariard:
    Code Review/Tested ACK 9f88ded
  naumenkogs:
    utACK 9f88ded82b
  jonatack:
    ACK 9f88ded82b

Tree-SHA512: 1e93c0a5b68cb432524780ffc0093db893911fdfed9e2ed17f888e59114cc75d2a07062aefad4e5ce2e87c9270886117a8abb3c78fb889c9b9f31967f1777148
2020-08-07 07:34:27 +08:00
Wladimir J. van der Laan
0a8aa626dd refactor: Make HexStr take a span
Make HexStr take a span of bytes, instead of an awkward pair of
templated iterators.
2020-08-06 19:41:43 +02:00
Sebastian Falbesoner
dac7a111bd refactor: test: use _ variable for unused loop counters
substitutes "for x in range(N):" by "for _ in range(N):"
indicates to the reader that a block is just repeated N times, and
that the loop counter is not used in the body
2020-08-06 18:39:33 +02:00
fanquake
82127d27c9
Merge #19667: build: set minimum required Boost to 1.58.0
70452a070b build: set minimum required Boost to 1.58 (fanquake)

Pull request description:

  Any systems which only have an older installable Boost can use depends.
  1.58.0 retains compatibility with the packages [installable on Ubuntu 16.04](https://packages.ubuntu.com/xenial/libboost-dev).

  The projects usage of Boost wont be going away any time soon, if ever (i.e #15382), and our usage of the test framework.

  Fixes: #19506

ACKs for top commit:
  practicalswift:
    ACK 70452a070b -- patch looks correct
  laanwj:
    ACK 70452a070b
  hebasto:
    ACK 70452a070b, tested on Linux Mint 20 (x86_64).

Tree-SHA512: d290415e3c70a394b3d7659c0480a35b4082bdce8d48b1c64a0025f7ad6e21567b4dc85813869513ad246d27f950706930410587c11c1aa3693ae6245084765c
2020-08-06 19:29:09 +08:00
eugene
90bd476ea6 build: make clean removes .gcda and .gcno files from fuzz directory
With this commit, make clean now removes coverage files from the
fuzzing directory. Without this, subsequent fuzzing runs would have
garbled coverage signals for files in the fuzz directory as
they were never deleted with make clean.
2020-08-06 00:44:03 -04:00
fanquake
19f812c57b
Merge #19669: contrib: Fixup valgrind suppressions file
fa5288cf5c contrib: Fixup valgrind suppressions file (MarcoFalke)

Pull request description:

  I am observing this one on bionic with system boost::fs:

  ```
  {
     <insert_a_suppression_name_here>
     Memcheck:Cond
     fun:__wcsnlen_avx2
     fun:wcsnrtombs
     fun:_ZNKSt7codecvtIwc11__mbstate_tE6do_outERS0_PKwS4_RS4_PcS6_RS6_
     fun:_ZN5boost10filesystem11path_traits7convertEPKwS3_RNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt7codecvtIwc11__mbstate_tE
     fun:_ZN5boost10filesystem6detail11unique_pathERKNS0_4pathEPNS_6system10error_codeE
  ...

ACKs for top commit:
  practicalswift:
    ACK fa5288cf5c -- patch looks correct

Tree-SHA512: 067e10e932a7f5b13e516134e0cfd3030265c1b582cdfde1cea97042e31399aa40c4590710a39429854c68ad703a0ae9f0b06e9af4cdd81e3cacb042939a84b6
2020-08-06 10:14:44 +08:00
Wladimir J. van der Laan
e3272ff290
Merge #19614: util: use HAVE_FDATASYNC to determine fdatasync() use
1d8338d6b7 util: use HAVE_FDATASYNC to determine fdatasync() use (fanquake)

Pull request description:

  Rather than just using on Linux and NetBSD, use `fdatasync()` based
  on whether it's available. i.e `fdatasync` is available in newer versions of FreeBSD.

  This also aligns more closely with what is being done in leveldb.

  Was pointed out by Luke in #19430.

ACKs for top commit:
  practicalswift:
    ACK 1d8338d6b7 -- patch looks correct
  laanwj:
    ACK 1d8338d6b7
  hebasto:
    ACK 1d8338d6b7

Tree-SHA512: 7dd6d87f5dc0c0ba21ae42f96b63fc12b34806cd536457fc4284f14bb8c235765344be228b000c6adf4cd1e8c4e6a03a18ca18ab22599c42cc3b706e0bcd1a17
2020-08-05 17:33:42 +02:00
Hennadii Stepanov
33a84e8f40
build: Update and sort package list in gitian-linux.yml
Virtual package 'binutils-gold' replaced with 'binutils'.
Explicitly added 'patch' package.
2020-08-05 18:08:47 +03:00
MarcoFalke
fa5288cf5c
contrib: Fixup valgrind suppressions file 2020-08-05 16:43:30 +02:00
Wladimir J. van der Laan
4644b13b44
Merge #19632: test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli
82fc4017b7 test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli (Ben Woosley)

Pull request description:

  `decimal.InvalidOperation` is a special case of a float parsing error, which
  presumably should be handled in the same way as a general parsing error,
  rather than blow up.

  Alternatives include: logging the error, or re-raising with more information.

  Example log output:
  ```
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
        self.sync_blocks(nodes)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
        return self.cli.send_cli(self.command, *args, **kwargs)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
        return json.loads(cli_stdout, parse_float=decimal.Decimal)
      File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
        return cls(**kw).decode(s)
      File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
        obj, end = self.scan_once(s, idx)
    decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
  ```
  See: https://travis-ci.org/github/bitcoin/bitcoin/jobs/713502326

ACKs for top commit:
  laanwj:
    ACK 82fc4017b7

Tree-SHA512: 8c102b8bf831b05c5ca4b2e1feb5574dcbaed8cab0b2f22b013c5dfcb81788a38839a163dd1e2c6470ccbe5874214663b84485f45467738fd850ca38d539ae25
2020-08-05 16:27:54 +02:00
Samuel Dobson
e4df534c60
Merge #15382: util: add RunCommandParseJSON
31cf68a3ad [util] add RunCommandParseJSON (Sjors Provoost)
c17f54ee53 [ci] use boost::process (Sjors Provoost)
32128ba682 [doc] include Doxygen comments for HAVE_BOOST_PROCESS (Sjors Provoost)
3c84d85f7d [build] msvc: add boost::process (Sjors Provoost)
c47e4bbf0b [build] make boost-process opt-in (Sjors Provoost)
929cda5470 configure: add ax_boost_process (Sjors Provoost)
8314c23d7b [depends] boost: patch unused variable in boost_process (Sjors Provoost)

Pull request description:

  Prerequisite for external signer support in #16546. Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).

  This adds a new dependency [boost process](https://github.com/boostorg/process/tree/boost-1.64.0). This is part of Boost since 1.64 which is part of `depends`. Because the minimum Boost version is 1.47, this functionality is skipped for older versions of Boost.

  Use `./configure --with-boost-process` to opt in, which checks for the presence of Boost::Process.

  We add `UniValue runCommandParseJSON(const std::string& strCommand)` to `system.{h,cpp}` which calls an arbitrary command and processes the JSON returned by it. This is currently only called by the test suite.

  ~For testing purposes this adds a new regtest-only RPC method `runcommand`, as well as `test/mocks/command.py` used by functional tests.~ (this is no longer the case)

  TODO:
  - [ ] review boost process in #15440

ACKs for top commit:
  achow101:
    ACK 31cf68a3ad
  hebasto:
    re-ACK 31cf68a3ad, only rebased (verified with `git range-diff`) and removed an unintentional tab character since the [previous](https://github.com/bitcoin/bitcoin/pull/15382#pullrequestreview-458371035) review.
  meshcollider:
    Very light utACK 31cf68a3ad, although I am not very confident with build stuff.
  promag:
    Code review ACK 31cf68a3ad, don't mind the nit.
  ryanofsky:
    Code review ACK 31cf68a3ad. I left some comments below that could be ignored or followed up later. The current change is clean and comprehensive.

Tree-SHA512: c506e747014b263606e1f538ed4624a8ad7bcf4e025cb700c12cc5739964e254dc04a2bbb848996b170e2ccec3fbfa4fe9e2b3976b191222cfb82fc3e6ab182d
2020-08-05 23:43:43 +12:00
fanquake
70452a070b
build: set minimum required Boost to 1.58
Any systems which only have an older install-able Boost can use depends.

Fixes: #19506
2020-08-05 17:13:45 +08:00
MarcoFalke
65e4ecabd5
Merge #19654: lint: Don't use TRAVIS_COMMIT_RANGE in commit message linter
72351784b3 lint: Remove travis env var from commit linter (Fabian Jahr)

Pull request description:

  #19439 was recently merged and seemed to work fine but I now noticed strange behavior when it was running in Travis, which I could not reproduce locally. It turns out `TRAVIS_COMMIT_RANGE` which is used in Travis to get the commits for the linter, uses all the commits that were in a push, which includes all rebase commits for example. This means that the linter can fail on a commit that the developer has never even seen before, which can be very confusing. See an example here which caused me to look into this: https://travis-ci.org/github/bitcoin/bitcoin/jobs/714296381 The commit that is reported as failing in my PR is not part of my PR.

  I think we rather want to use something like `git merge-base` to get the commit range by default and in Travis. I am leaving the env variable functionality in place with a different name but this is not a variable that can be expected to be present in the CI environments so the `merge-base` range should be used there by default.

ACKs for top commit:
  hebasto:
    ACK 72351784b3, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: afb27bb386855cb8d5cf84fd3a6c11ef1160b25af6175ed0aa146bf04b9a26eb77298df70df0a855f8c46f19f08b3f62c49872c12974fcfa5526a15ee05b3c10
2020-08-05 10:45:33 +02:00