bitcoin/src
fanquake c73bd004ae
Merge #18861: Do not answer GETDATA for to-be-announced tx
2896c412fa Do not answer GETDATA for to-be-announced tx (Pieter Wuille)
f2f32a3dee Push down use of cs_main into FindTxForGetData (Pieter Wuille)
c6131bf407 Abstract logic to determine whether to answer tx GETDATA (Pieter Wuille)

Pull request description:

  This PR intends to improve transaction-origin privacy.

  In general, we should try to not leak information about what transactions we have (recently) learned about before deciding to announce them to our peers. There is a controlled transaction dissemination process that reveals our transactions to peers that has various safeguards for privacy (it's rate-limited, delayed & batched, deterministically sorted, ...), and ideally there is no way to test which transactions we have before that controlled process reveals them. The handling of the `mempool` BIP35 message has protections in this regard as well, as it would be an obvious way to bypass these protections (handled asynchronously after a delay, also deterministically sorted).

  However, currently, if we receive a GETDATA for a transaction that we have not yet announced to the requester, we will still respond to it if it was announced to *some* other peer already (because it needs to be in `mapRelay`, which only happens on the first announcement). This is a slight privacy leak.

  Thankfully, this seems easy to solve: `setInventontoryTxToSend` keeps track of the txids we have yet to announce to a peer - which almost(*) exactly corresponds to the transactions we know of that we haven't revealed to that peer. By checking whether a txid is in that set before responding to a GETDATA, we can filter these out.

  (*) Locally resubmitted or rebroadcasted transactions may end up in setInventoryTxToSend while the peer already knows we have them, which could result in us incorrectly claiming we don't have such transactions if coincidentally requested right after we schedule reannouncing them, but before they're actually INVed. This is made even harder by the fact that filterInventoryKnown will generally keep known reannouncements out of setInventoryTxToSend unless it overflows (which needs 50000 INVs in either direction before it happens).

  The condition for responding now becomes:

  ```
    (not in setInventoryTxToSend) AND
    (
      (in relay map) OR
      (
        (in mempool) AND
        (old enough that it could have expired from relay map) AND
        (older than our last getmempool response)
      )
    )
  ```

ACKs for top commit:
  naumenkogs:
    utACK 2896c41
  ajtowns:
    ACK 2896c412fa
  amitiuttarwar:
    code review ACK 2896c412fa
  jonatack:
    ACK 2896c412fa per `git diff 2b3f101 2896c41` only change since previous review is moving the recency check up to be verified first in `FindTxForGetData`, as it was originally in 353a391 (good catch), before looking up the transaction in the relay pool.
  jnewbery:
    code review ACK 2896c412fa

Tree-SHA512: e7d5bc006e626f60a2c108a9334f3bbb67205ace04a7450a1e4d4db1d85922a7589e0524500b7b4953762cf70554c4a08eec62c7b38b486cbca3d86321600868
2020-05-19 15:18:06 +08:00
..
bench Remove CCoinsViewCache::GetValueIn(...) 2020-05-03 18:42:14 +00:00
compat test: remove glibc fdelt sanity check 2020-05-07 15:45:09 +08:00
config
consensus scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
crc32c Import crc32c using subtree merge as as 'src/crc32c' 2020-01-28 17:00:01 +01:00
crypto Avoid non-trivial global constants in SHA-NI code 2020-04-07 13:57:34 -07:00
index Make ThreadSafe{MessageBox|Question} bilingual 2020-05-05 04:45:59 +03:00
interfaces Merge #18922: gui: Do not translate InitWarning messages in debug.log 2020-05-13 20:30:39 +02:00
leveldb Update to leveldb upstream using subtree merge 2020-01-28 16:59:07 +01:00
logging Remove use of non-standard zero variadic macros 2020-04-30 18:02:04 +08:00
node Merge #18038: P2P: Mempool tracks locally submitted transactions to improve wallet privacy 2020-04-29 16:32:37 +08:00
policy scripted-diff: Bump copyright of files changed in 2020 2020-01-15 02:18:00 +07:00
primitives Remove CCoinsViewCache::GetValueIn(...) 2020-05-03 18:42:14 +00:00
qt Merge #18922: gui: Do not translate InitWarning messages in debug.log 2020-05-13 20:30:39 +02:00
rpc Merge #18742: miner: Avoid stack-use-after-return in validationinterface 2020-05-14 20:40:55 +08:00
script refactor: Remove override for final overriders 2020-05-08 10:37:05 +03:00
secp256k1 Update the secp256k1 subtree to the latest upstream version 2019-03-31 11:41:05 -07:00
support Merge #18443: lockedpool: avoid sensitive data in core files (FreeBSD) 2020-05-04 16:31:07 +02:00
test Merge #18938: tests: Fill fuzzing coverage gaps for functions in consensus/validation.h, primitives/block.h and util/translation.h 2020-05-17 08:19:39 -04:00
univalue Update univalue subtree 2020-02-09 07:44:29 -08:00
util Merge #18922: gui: Do not translate InitWarning messages in debug.log 2020-05-13 20:30:39 +02:00
wallet Merge #18946: rpcwallet: Replace boost::optional<T>::emplace with simple assignment of T{} 2020-05-14 19:26:17 +02:00
zmq scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
.clang-format util: Add AllowShortCaseLabelsOnASingleLine option 2019-10-14 13:24:54 +03:00
addrdb.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
addrdb.h Convert addrdb/addrman to new serialization 2020-01-02 11:05:36 -08:00
addrman.cpp Merge #18512: Improve asmap checks and add sanity check 2020-05-06 14:59:28 +02:00
addrman.h logging: asmap logging and #include fixups 2020-03-04 14:24:19 +01:00
amount.h Update copyright headers to 2018 2018-12-29 10:15:01 +01:00
arith_uint256.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
arith_uint256.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
attributes.h Add NODISCARD to all {Decode,Parse}[...](...) functions returning bool. Sort includes. 2018-11-05 17:03:11 +01:00
banman.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
banman.h refactor: move DUMP_BANS_INTERVAL to banman.h 2020-03-10 09:52:53 -04:00
base58.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
base58.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
bech32.cpp Add some general std::vector utility functions 2019-10-16 08:56:57 -07:00
bech32.h Assert that the HRP is lowercase in Bech32::Encode 2019-09-05 13:25:11 +12:00
bitcoin-cli-res.rc
bitcoin-cli.cpp build: fix ASLR for bitcoin-cli on Windows 2020-04-19 10:05:29 +08:00
bitcoin-tx-res.rc
bitcoin-tx.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
bitcoin-wallet-res.rc [tools] Add wallet inspection and modification tool 2019-01-30 16:26:52 -05:00
bitcoin-wallet.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
bitcoind-res.rc
bitcoind.cpp Make InitError bilingual 2020-05-05 04:46:04 +03:00
blockencodings.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
blockencodings.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
blockfilter.cpp tests: Add fuzzing harness for Golomb-Rice coding (GolombRiceEncode/GolombRiceDecode) 2020-04-20 14:57:48 +00:00
blockfilter.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
bloom.cpp net: remove is{Empty,Full} flags from CBloomFilter, clarify CVE fix 2020-04-28 19:27:22 +02:00
bloom.h net: remove is{Empty,Full} flags from CBloomFilter, clarify CVE fix 2020-04-28 19:27:22 +02:00
chain.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
chain.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
chainparams.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
chainparams.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
chainparamsbase.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
chainparamsbase.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
chainparamsseeds.h net: Hardcoded seeds update for 0.20 2020-04-03 16:29:26 +02:00
checkqueue.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
clientversion.cpp Drop unused GIT_COMMIT_DATE macro 2020-05-04 19:53:58 +03:00
clientversion.h
coins.cpp Remove CCoinsViewCache::GetValueIn(...) 2020-05-03 18:42:14 +00:00
coins.h Merge #18854: doc: Fix typo in Coin doxygen comment 2020-05-06 13:15:28 +02:00
compat.h windows: Set _WIN32_WINNT to 0x0601 (Windows 7) 2019-01-23 16:28:27 +08:00
compressor.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
compressor.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
core_io.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
core_memusage.h
core_read.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
core_write.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
cuckoocache.h tests: Add fuzzing harness for classes/functions in cuckoocache.h 2020-04-08 14:45:27 +00:00
dbwrapper.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
dbwrapper.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
dummywallet.cpp wallet: Avoid translating RPC errors when loading wallets 2020-05-01 07:39:00 -04:00
flatfile.cpp Style cleanup. 2019-02-22 17:38:45 -08:00
flatfile.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
fs.cpp scripted-diff: Add missed copyright headers 2020-01-04 20:18:28 +02:00
fs.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
hash.cpp Extract CSipHasher to it's own file in crypto/ directory. 2018-11-05 09:25:15 -08:00
hash.h Squashed 'src/secp256k1/' changes from 0b70241850..b19c000063 2019-03-31 11:41:05 -07:00
httprpc.cpp Make ThreadSafe{MessageBox|Question} bilingual 2020-05-05 04:45:59 +03:00
httprpc.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
httpserver.cpp Make ThreadSafe{MessageBox|Question} bilingual 2020-05-05 04:45:59 +03:00
httpserver.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
indirectmap.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
init.cpp Merge #18922: gui: Do not translate InitWarning messages in debug.log 2020-05-13 20:30:39 +02:00
init.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
key.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
key.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
key_io.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
key_io.h
limitedmap.h
logging.cpp log: Use Join() helper when listing log categories 2020-04-16 11:08:46 -04:00
logging.h log: Use Join() helper when listing log categories 2020-04-16 11:08:46 -04:00
Makefile.am test: remove glibc fdelt sanity check 2020-05-07 15:45:09 +08:00
Makefile.bench.include bench: add CAddrMan benchmarks 2020-04-21 15:06:59 +02:00
Makefile.crc32c.include build: CRC32C build system integration 2020-01-28 17:01:48 +01:00
Makefile.leveldb.include build: Enable -Wsuggest-override 2020-05-12 18:03:39 +03:00
Makefile.qt.include Revert "Merge #16367: Multiprocess build support" 2020-04-10 19:38:21 -04:00
Makefile.qt_locale.include qt: Translations update pre-branch 2020-04-01 12:49:15 +02:00
Makefile.qttest.include build: Create test utility library from src/test/util/ 2019-11-21 21:13:08 +01:00
Makefile.test.include Merge #18512: Improve asmap checks and add sanity check 2020-05-06 14:59:28 +02:00
Makefile.test_fuzz.include build: create test_fuzz library from src/test/fuzz/fuzz.cpp 2020-04-05 01:01:13 +02:00
Makefile.test_util.include fuzz: Add process_messages harness 2020-04-05 10:46:24 +08:00
memusage.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
merkleblock.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
merkleblock.h
miner.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
miner.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
net.cpp net: use CMessageHeader::HEADER_SIZE, add missing include 2020-05-10 15:58:42 +02:00
net.h p2p: add MAX_FEELER_CONNECTIONS constant 2020-05-12 17:30:33 +08:00
net_permissions.cpp net: Avoid using C-style NUL-terminated strings as arguments in the netbase interface 2020-01-08 12:35:59 +00:00
net_permissions.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
net_processing.cpp Merge #18861: Do not answer GETDATA for to-be-announced tx 2020-05-19 15:18:06 +08:00
net_processing.h [init] Add -peerblockfilters option 2020-05-08 16:36:18 -04:00
net_types.h refactor: Remove addrdb.h dependency from node.h 2019-10-29 11:30:12 +02:00
netaddress.cpp Merge #18512: Improve asmap checks and add sanity check 2020-05-06 14:59:28 +02:00
netaddress.h Merge #18512: Improve asmap checks and add sanity check 2020-05-06 14:59:28 +02:00
netbase.cpp Merge #17754: net: Don't allow resolving of std::string with embedded NUL characters. Add tests. 2020-01-22 20:20:45 +01:00
netbase.h net: Avoid using C-style NUL-terminated strings as arguments in the netbase interface 2020-01-08 12:35:59 +00:00
netmessagemaker.h
noui.cpp Make ThreadSafe{MessageBox|Question} bilingual 2020-05-05 04:45:59 +03:00
noui.h Make ThreadSafe{MessageBox|Question} bilingual 2020-05-05 04:45:59 +03:00
optional.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
outputtype.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
outputtype.h List output types in an array in order to be iterated over 2020-01-23 16:35:06 -05:00
pow.cpp
pow.h
prevector.h prevector: Avoid unnamed struct, which is a GNU extension 2020-04-30 18:02:03 +08:00
protocol.cpp [net processing] Message handling for getcfcheckpt. 2020-05-08 16:36:19 -04:00
protocol.h [net processing] Message handling for getcfcheckpt. 2020-05-08 16:36:19 -04:00
psbt.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
psbt.h Merge #17509: gui: save and load PSBT 2020-04-23 13:16:23 +12:00
pubkey.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
pubkey.h Implement TopUp in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
random.cpp Merge #18781: Add templated GetRandDuration<> 2020-05-15 08:58:49 -04:00
random.h Add templated GetRandomDuration<> 2020-04-30 09:19:14 -04:00
randomenv.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
randomenv.h [MOVEONLY] Move perfmon data gathering to new randomenv module 2019-11-12 14:50:44 -08:00
rest.cpp Merge #18810: doc: update rest info on block size and json 2020-04-30 11:45:06 +02:00
reverse_iterator.h
scheduler.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
scheduler.h scheduler: Make schedule* methods type safe 2020-03-10 09:47:32 -04:00
serialize.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
shutdown.cpp
shutdown.h
span.h Merge #18591: Add C++17 build to Travis 2020-04-30 11:16:56 +02:00
streams.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
sync.cpp util: Remove unused itostr 2020-03-27 08:59:06 -04:00
sync.h refactor: Add interfaces::FoundBlock class to selectively return block data 2020-03-31 08:36:02 -05:00
threadinterrupt.cpp Merge #11640: Make LOCK, LOCK2, TRY_LOCK work with CWaitableCriticalSection 2018-08-31 16:00:38 +02:00
threadinterrupt.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
threadsafety.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
timedata.cpp Make ThreadSafe{MessageBox|Question} bilingual 2020-05-05 04:45:59 +03:00
timedata.h
tinyformat.h util: Update tinyformat to upstream 2019-12-06 10:02:08 +01:00
torcontrol.cpp net: Avoid using C-style NUL-terminated strings as arguments in the netbase interface 2020-01-08 12:35:59 +00:00
torcontrol.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
txdb.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
txdb.h scripted-diff: Bump copyright of files changed in 2020 2020-01-15 02:18:00 +07:00
txmempool.cpp Merge #18038: P2P: Mempool tracks locally submitted transactions to improve wallet privacy 2020-04-29 16:32:37 +08:00
txmempool.h [mempool] Track "unbroadcast" transactions 2020-04-23 14:42:25 -07:00
ui_interface.cpp gui: Do not translate InitWarning messages in debug.log 2020-05-10 18:01:28 +03:00
ui_interface.h gui: Do not translate InitWarning messages in debug.log 2020-05-10 18:01:28 +03:00
uint256.cpp refactor: define a UINT256_ONE global constant 2020-01-23 16:35:08 -05:00
uint256.h refactor: define a UINT256_ONE global constant 2020-01-23 16:35:08 -05:00
undo.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
validation.cpp Merge #16224: gui: Bilingual GUI error messages 2020-05-08 12:17:55 -04:00
validation.h Merge #18786: init: Remove boost from ThreadImport 2020-05-04 16:06:42 +02:00
validationinterface.cpp validationinterface: Rework documentation, Rename pwalletIn to callbacks 2020-05-13 19:57:55 -04:00
validationinterface.h validationinterface: Rework documentation, Rename pwalletIn to callbacks 2020-05-13 19:57:55 -04:00
version.h
versionbits.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
versionbits.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
versionbitsinfo.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
versionbitsinfo.h MOVEONLY: Move versionbits info out of versionbits.o 2018-09-23 22:55:11 +02:00
walletinitinterface.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
warnings.cpp scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00
warnings.h scripted-diff: Bump copyright of files changed in 2019 2019-12-30 10:42:20 +13:00