bitcoin/src
Andrew Chow d232e36abd
Merge bitcoin/bitcoin#28207: mempool: Persist with XOR
fa6b053b5c mempool: persist with XOR (MarcoFalke)

Pull request description:

  Currently the `mempool.dat` file stores data received from remote peers as-is. This may be problematic when a program other than Bitcoin Core tries to interpret them by accident. For example, an anti-virus program or other program may scan the file and move it into quarantine, or delete it, or corrupt it.

  While the local wallet is expected to re-submit any pending transactions, unrelated transactions may be missing from the mempool after a restart. This may cause fee estimates to be off, or may cause block relay to be slower.

  Fix this, similar to https://github.com/bitcoin/bitcoin/pull/6650, by rolling a random XOR pattern over the dat file when writing or reading it.

  Obviously this can only protect against programs that accidentally and unintentionally are trying to mess with the dat file. Any program that intentionally wants to mess with the dat file can still trivially do so.

ACKs for top commit:
  achow101:
    re-ACK fa6b053b5c
  glozow:
    reACK fa6b053b5c
  ismaelsadeeq:
    ACK fa6b053b5c

Tree-SHA512: ded2ce3d81bc944b828263534e3178a1e45a914fe8e024f4a14c6561a73e301820944ecc75dd704b3d4221a7a3a5c0597ccab79546250c1197609ee981fe324e
2023-11-13 11:28:15 -05:00
..
bench Merge bitcoin/bitcoin#28530: tests, bug fix: DisconnectedBlockTransactions rewrite followups 2023-11-02 11:12:17 +00:00
common Merge bitcoin/bitcoin#28076: util: Replace std::filesystem with util/fs.h 2023-11-13 14:10:54 +00:00
compat Sort includes in compat.h 2023-08-08 17:50:41 +02:00
config
consensus [validation] add TxValidationResult::TX_RECONSIDERABLE, TX_UNKNOWN 2023-11-06 14:41:56 +00:00
crc32c Update crc32c subtree to latest upstream master 2022-08-13 13:55:38 +01:00
crypto Add ability to specify SHA256 implementation for benchmark purposes 2023-09-20 21:11:55 +01:00
index coinstats: Fix hash_serialized2 calculation 2023-10-20 22:53:05 +02:00
init init: remove config option names from translated -loglevel strings 2023-06-15 10:27:56 -06:00
interfaces interfaces: Rename CalculateBumpFees methods to be compatible with capn'proto 2023-10-20 10:30:16 -04:00
ipc refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
kernel Merge bitcoin/bitcoin#28207: mempool: Persist with XOR 2023-11-13 11:28:15 -05:00
leveldb Update leveldb-subtree subtree to latest upstream 2022-09-29 16:43:30 +01:00
logging Use steady clock for logging timer 2023-01-31 18:48:50 +01:00
minisketch Update minisketch subtree to latest upstream 2022-10-23 15:03:04 +01:00
node Merge bitcoin/bitcoin#28207: mempool: Persist with XOR 2023-11-13 11:28:15 -05:00
policy Merge bitcoin/bitcoin#28076: util: Replace std::filesystem with util/fs.h 2023-11-13 14:10:54 +00:00
primitives refactor: Add LIFETIMEBOUND to all (w)txid getters 2023-10-27 13:01:42 +02:00
qt Merge bitcoin/bitcoin#28721: multiprocess compatibility updates 2023-11-13 12:32:55 +00:00
rpc Merge bitcoin/bitcoin#28721: multiprocess compatibility updates 2023-11-13 12:32:55 +00:00
script iwyu: Export prevector.h from script.h 2023-10-25 11:55:50 +02:00
secp256k1 Update secp256k1 subtree to upstream release 0.4.0 2023-09-04 12:51:20 -04:00
support Add make_secure_unique helper 2023-09-27 15:05:15 -04:00
test Merge bitcoin/bitcoin#28721: multiprocess compatibility updates 2023-11-13 12:32:55 +00:00
univalue Remove unused raw-pointer read helper from univalue 2023-07-27 14:24:52 +02:00
util Merge bitcoin/bitcoin#28076: util: Replace std::filesystem with util/fs.h 2023-11-13 14:10:54 +00:00
wallet Merge bitcoin/bitcoin#28721: multiprocess compatibility updates 2023-11-13 12:32:55 +00:00
zmq validationinterface: only send zmq notifications for active 2023-09-30 06:38:47 -04:00
.bear-tidy-config tidy: modernize-use-emplace 2023-10-12 11:27:19 +02:00
.clang-format
.clang-tidy refactor: Fix bugprone-string-constructor warning 2023-10-30 14:59:17 +01:00
addrdb.cpp build: Bump minimum supported Clang to clang-13 2023-10-24 18:52:00 +02:00
addrdb.h Use serialization parameters for CAddress serialization 2023-09-05 10:13:25 +02:00
addresstype.cpp Add PubKeyDestination for P2PK scripts 2023-09-12 12:14:31 -04:00
addresstype.h bugfix: Mark CNoDestination and PubKeyDestination constructor explicit 2023-10-25 22:46:55 +02:00
addrman.cpp addrman: log AS only when using asmap 2023-10-30 18:46:06 -03:00
addrman.h rpc: getrawaddrman for addrman entries 2023-10-02 15:34:28 +02:00
addrman_impl.h rpc: getrawaddrman for addrman entries 2023-10-02 15:34:28 +02:00
arith_uint256.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
arith_uint256.h Fix clang-tidy readability-const-return-type violations 2023-02-01 11:33:35 +01:00
attributes.h Introduce platform-agnostic ALWAYS_INLINE macro 2023-05-04 20:57:51 +01:00
banman.cpp Merge bitcoin/bitcoin#24097: Replace RecursiveMutex m_cs_banned with Mutex, and rename it 2023-11-02 14:09:27 -04:00
banman.h Merge bitcoin/bitcoin#24097: Replace RecursiveMutex m_cs_banned with Mutex, and rename it 2023-11-02 14:09:27 -04:00
base58.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
base58.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
bech32.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
bech32.h
bip324.cpp net: merge V2Transport constructors, move key gen 2023-09-10 16:11:52 -04:00
bip324.h net: merge V2Transport constructors, move key gen 2023-09-10 16:11:52 -04:00
bitcoin-chainstate.cpp refactor: Replace <filesystem> with <util/fs.h> 2023-09-14 18:58:37 +02:00
bitcoin-cli-res.rc
bitcoin-cli.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
bitcoin-tx-res.rc
bitcoin-tx.cpp kernel: Remove Univalue from kernel library 2023-07-25 17:40:07 +02:00
bitcoin-util-res.rc
bitcoin-util.cpp [refactor] Add missing includes for next commit 2023-09-12 22:51:42 +02:00
bitcoin-wallet-res.rc
bitcoin-wallet.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
bitcoind-res.rc
bitcoind.cpp Remove the syscall sandbox 2023-06-16 10:38:19 +01:00
blockencodings.cpp scripted-diff: rename vTxHashes to txns_randomized 2023-11-10 16:44:44 +01:00
blockencodings.h script: remove out-of-date snprintf TODO 2023-02-15 14:42:28 -08:00
blockfilter.cpp Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader 2023-09-19 14:19:57 +00:00
blockfilter.h Remove unused includes from blockfilter.h 2023-08-17 18:28:15 +02:00
chain.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
chain.h refactor: Remove CBlockFileInfo::SetNull 2023-10-20 16:29:02 +02:00
chainparams.cpp test: Throw error when -signetchallenge is non-hex 2023-05-25 19:24:05 +02:00
chainparams.h test: Throw error when -signetchallenge is non-hex 2023-05-25 19:24:05 +02:00
chainparamsbase.cpp refactor: Use ChainType enum exhaustively 2023-05-10 10:39:58 +02:00
chainparamsbase.h refactor: Use ChainType enum exhaustively 2023-05-10 10:39:58 +02:00
chainparamsseeds.h p2p: update hardcoded mainnet seeds for 25.x 2023-04-20 06:08:22 -07:00
checkqueue.h Remove the syscall sandbox 2023-06-16 10:38:19 +01:00
clientversion.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
clientversion.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
coins.cpp Merge bitcoin/bitcoin#26331: Implement CCoinsViewErrorCatcher::HaveCoin and check disk space periodically 2023-10-09 14:52:00 -04:00
coins.h Merge bitcoin/bitcoin#26331: Implement CCoinsViewErrorCatcher::HaveCoin and check disk space periodically 2023-10-09 14:52:00 -04:00
compressor.cpp Clean up things that include script/standard.h 2023-08-14 17:38:27 -04:00
compressor.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
core_io.h kernel: Remove Univalue from kernel library 2023-07-25 17:40:07 +02:00
core_memusage.h
core_read.cpp kernel: Remove Univalue from kernel library 2023-07-25 17:40:07 +02:00
core_write.cpp Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
cuckoocache.h clang-tidy: Fix modernize-use-default-member-init in headers 2023-01-31 11:50:10 +00:00
dbwrapper.cpp dbwrapper: Use DataStream for batch operations 2023-09-12 12:07:39 +02:00
dbwrapper.h dbwrapper: Use DataStream for batch operations 2023-09-12 12:07:39 +02:00
deploymentinfo.cpp Decouple RegTestChainParams from ArgsManager 2023-03-15 16:42:42 +01:00
deploymentinfo.h Decouple RegTestChainParams from ArgsManager 2023-03-15 16:42:42 +01:00
deploymentstatus.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
deploymentstatus.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
dummywallet.cpp move-only: Extract common/args and common/config.cpp from util/system 2023-04-19 10:48:30 +02:00
external_signer.cpp tidy: modernize-use-emplace 2023-10-12 11:27:19 +02:00
external_signer.h refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
flatfile.cpp refactor: Extract util/fs_helpers from util/system 2023-03-23 12:52:00 +01:00
flatfile.h Merge bitcoin/bitcoin#27254: refactor: Extract util/fs from util/system 2023-04-03 14:41:22 +01:00
hash.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
hash.h Remove CHashWriter type 2023-09-19 16:38:08 +02:00
headerssync.cpp tidy: modernize-use-emplace 2023-10-12 11:27:19 +02:00
headerssync.h [headerssync] Make m_commit_offset protected 2022-12-12 21:06:04 +00:00
httprpc.cpp refactor: drop protocol.h include header in rpc/util.h 2023-09-19 15:54:17 -06:00
httprpc.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
httpserver.cpp Merge bitcoin/bitcoin#26078: p2p: return CSubNet in LookupSubNet 2023-10-26 14:29:47 -04:00
httpserver.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
i2p.cpp Merge bitcoin/bitcoin#28695: net: Sanity check private keys received from SAM proxy 2023-10-30 14:44:40 +01:00
i2p.h i2p: destroy the session if we get an unexpected error from the I2P router 2023-10-05 14:11:13 +02:00
indirectmap.h
init.cpp mempool: persist with XOR 2023-11-09 19:44:50 +01:00
init.h refactor: index, decouple 'Init' from 'Start' 2023-07-10 10:47:17 -03:00
key.cpp key: don't allocate secure mem for null (invalid) key 2023-09-27 15:05:26 -04:00
key.h key: don't allocate secure mem for null (invalid) key 2023-09-27 15:05:26 -04:00
key_io.cpp Merge bitcoin/bitcoin#28246: wallet: Use CTxDestination in CRecipient instead of just scriptPubKey 2023-09-19 16:48:43 +00:00
key_io.h Clean up things that include script/standard.h 2023-08-14 17:38:27 -04:00
logging.cpp [log] add category TXPACKAGES for orphanage and package relay 2023-08-29 16:41:22 +01:00
logging.h [log] add category TXPACKAGES for orphanage and package relay 2023-08-29 16:41:22 +01:00
Makefile.am Merge bitcoin/bitcoin#28530: tests, bug fix: DisconnectedBlockTransactions rewrite followups 2023-11-02 11:12:17 +00:00
Makefile.bench.include [bench] DisconnectedBlockTransactions 2023-09-13 11:37:13 +01:00
Makefile.crc32c.include build: Create .la library for crc32c 2022-04-26 16:25:38 -04:00
Makefile.leveldb.include Use more specific path when including memenv.h header 2022-06-23 15:33:01 +02:00
Makefile.minisketch.include build: Build minisketch test in make check, not in make 2022-07-21 10:42:09 +01:00
Makefile.qt.include Merge bitcoin/bitcoin#22764: build: Include qt sources for parsing with extract_strings.py 2023-10-19 13:25:49 +01:00
Makefile.qt_locale.include qt: Translation updates from Transifex 2023-09-01 07:39:00 +01:00
Makefile.qttest.include build: extract $(BOOST_CPPFLAGS) from $(BITCOIN_INCLUDES) 2022-09-13 17:15:17 +01:00
Makefile.test.include Merge bitcoin/bitcoin#28721: multiprocess compatibility updates 2023-11-13 12:32:55 +00:00
Makefile.test_fuzz.include [fuzz] Move ConsumeNetAddr to fuzz/util/net.h 2022-11-17 14:52:45 +00:00
Makefile.test_util.include Merge bitcoin/bitcoin#27425: test: move remaining rand code from util/setup_common to util/random 2023-07-19 10:26:11 +01:00
Makefile.univalue.include
mapport.cpp Remove the syscall sandbox 2023-06-16 10:38:19 +01:00
mapport.h Remove configure-time setting of DEFAULT_UPNP 2023-01-28 15:24:13 +00:00
memusage.h add std::list to memusage 2023-09-13 11:37:45 +01:00
merkleblock.cpp
merkleblock.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
net.cpp Merge bitcoin/bitcoin#28155: net: improves addnode / m_added_nodes logic 2023-11-08 11:31:36 +00:00
net.h Merge bitcoin/bitcoin#28155: net: improves addnode / m_added_nodes logic 2023-11-08 11:31:36 +00:00
net_permissions.cpp Merge bitcoin/bitcoin#26078: p2p: return CSubNet in LookupSubNet 2023-10-26 14:29:47 -04:00
net_permissions.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
net_processing.cpp [validation] add TxValidationResult::TX_RECONSIDERABLE, TX_UNKNOWN 2023-11-06 14:41:56 +00:00
net_processing.h [net processing] PeerManager holds a FastRandomContext 2023-10-03 11:23:24 +01:00
net_types.cpp p2p: return CSubNet in LookupSubNet 2023-05-30 16:27:21 -03:00
net_types.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
netaddress.cpp Inline short, often-called, rarely-changed basic CNetAddr getters 2023-07-19 12:43:05 -06:00
netaddress.h net: put CJDNS prefix byte in a constant 2023-10-05 15:10:32 +02:00
netbase.cpp Merge bitcoin/bitcoin#28649: Do the SOCKS5 handshake reliably 2023-11-07 14:11:58 -05:00
netbase.h Merge bitcoin/bitcoin#28649: Do the SOCKS5 handshake reliably 2023-11-07 14:11:58 -05:00
netgroup.cpp net: put CJDNS prefix byte in a constant 2023-10-05 15:10:32 +02:00
netgroup.h [netgroupman] Remove NetGroupManager::GetAsmap() 2022-04-20 14:35:53 +01:00
netmessagemaker.h Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader 2023-09-19 14:19:57 +00:00
noui.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
noui.h
outputtype.cpp Clean up things that include script/standard.h 2023-08-14 17:38:27 -04:00
outputtype.h Move CTxDestination to its own file 2023-08-14 17:38:27 -04:00
pow.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
pow.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
prevector.h util: implement prevector's move ctor & move assignment 2023-03-26 15:49:52 +02:00
protocol.cpp net: advertise NODE_P2P_V2 if CLI arg -v2transport is on 2023-10-02 18:09:53 -04:00
protocol.h scripted-diff: Use ser params operator 2023-10-13 10:30:49 +02:00
psbt.cpp script/sign: Miniscript support in Tapscript 2023-10-08 02:43:24 +02:00
psbt.h tidy: modernize-use-emplace 2023-10-12 11:27:19 +02:00
pubkey.cpp pubkey: introduce a GetEvenCorrespondingCPubKey helper 2023-10-08 02:43:19 +02:00
pubkey.h pubkey: introduce a GetEvenCorrespondingCPubKey helper 2023-10-08 02:43:19 +02:00
random.cpp Merge bitcoin/bitcoin#26839: Add support for RNDR/RNDRRS for AArch64 on Linux 2023-11-07 15:00:38 -05:00
random.h crypto: refactor ChaCha20 classes to use Span<std::byte> interface 2023-08-17 15:26:34 -04:00
randomenv.cpp [refactor] Remove compat.h from kernel headers 2023-09-12 22:51:48 +02:00
randomenv.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
rest.cpp Merge bitcoin/bitcoin#28136: refactor: move GetServicesNames from rpc/util.{h,cpp} to rpc/net.cpp 2023-11-07 14:19:09 -05:00
rest.h Handle query string when parsing data format 2022-03-10 12:01:53 +01:00
reverse_iterator.h doc: remove usages of C++11 2023-01-12 13:42:44 +00:00
scheduler.cpp Remove the syscall sandbox 2023-06-16 10:38:19 +01:00
scheduler.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
serialize.h Remove WithParams serialization helper 2023-10-30 13:54:52 +01:00
shutdown.cpp kernel: Add fatalError method to notifications 2023-06-28 09:52:33 +02:00
shutdown.h kernel: Add fatalError method to notifications 2023-06-28 09:52:33 +02:00
signet.cpp Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader 2023-09-19 14:19:57 +00:00
signet.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
span.h span: Make Span template deduction guides work in SFINAE context 2023-10-20 10:30:16 -04:00
streams.cpp streams: Teach AutoFile how to XOR 2023-07-19 18:12:42 +02:00
streams.h Merge bitcoin/bitcoin#28207: mempool: Persist with XOR 2023-11-13 11:28:15 -05:00
sync.cpp sync: unpublish LocksHeld() which is used only in sync.cpp 2023-02-17 11:42:41 +01:00
sync.h Merge bitcoin/bitcoin#27116: doc: clarify that LOCK() internally checks whether the mutex is held 2023-10-26 15:02:13 -04:00
threadsafety.h
timedata.cpp move-only: Extract common/args and common/config.cpp from util/system 2023-04-19 10:48:30 +02:00
timedata.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
tinyformat.h clang-tidy: Fix modernize-use-default-member-init in headers 2023-01-31 11:50:10 +00:00
torcontrol.cpp net: move IsReachable() code to netbase and encapsulate it 2023-10-05 15:10:34 +02:00
torcontrol.h doc, refactor: Changing -torcontrol help to specify that a default port is used 2023-08-17 23:58:47 -05:00
txdb.cpp Merge bitcoin/bitcoin#28195: blockstorage: Drop legacy -txindex check 2023-09-05 11:37:35 +01:00
txdb.h move-only: Move CBlockTreeDB to node/blockstorage 2023-08-01 15:27:33 +02:00
txmempool.cpp Merge bitcoin/bitcoin#28207: mempool: Persist with XOR 2023-11-13 11:28:15 -05:00
txmempool.h Merge bitcoin/bitcoin#28207: mempool: Persist with XOR 2023-11-13 11:28:15 -05:00
txorphanage.cpp Use type-safe txid types in orphanage 2023-10-12 11:56:37 +01:00
txorphanage.h Use type-safe txid types in orphanage 2023-10-12 11:56:37 +01:00
txrequest.cpp refactor: Remove unused gcc-9 workaround in txrequest 2023-10-30 15:18:40 +01:00
txrequest.h
uint256.cpp refactor: modernize the implementation of uint256.* 2022-12-10 14:34:44 -06:00
uint256.h Merge bitcoin/bitcoin#27927: util: Allow std::byte and char Span serialization 2023-06-28 15:12:12 -04:00
undo.h
validation.cpp Merge bitcoin/bitcoin#28391: refactor: Simplify CTxMempool/BlockAssembler fields, remove some external mapTx access 2023-11-13 10:51:41 +00:00
validation.h [validation] change package-fee-too-low, return wtxid(s) and effective feerate 2023-11-07 11:26:17 +00:00
validationinterface.cpp validation: pass ChainstateRole for validationinterface calls 2023-09-30 06:38:47 -04:00
validationinterface.h validation: pass ChainstateRole for validationinterface calls 2023-09-30 06:38:47 -04:00
version.h Use serialization parameters for CAddress serialization 2023-09-05 10:13:25 +02:00
versionbits.cpp refactor: use braced init for integer constants instead of c style casts 2023-01-03 19:31:29 -06:00
versionbits.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
walletinitinterface.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
warnings.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
warnings.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00