Commit Graph

1868 Commits

Author SHA1 Message Date
Sebastian Falbesoner
cb0b7125c1 doc: libbitcoinconsensus: add missing error code description, fix NBitcoin link 2020-12-05 13:37:00 +01:00
MarcoFalke
dca80ffb45
Merge #20255: util: Add Assume() identity function
faa05854f8 util: Remove probably misleading TODO (MarcoFalke)
fac5efe730 util: Add Assume() identity function (MarcoFalke)
fa861569dc util: Allow Assert(...) to be used in all contexts (practicalswift)

Pull request description:

  This is needed for #20138. Please refer to the added documentation for motivation.

ACKs for top commit:
  practicalswift:
    cr ACK faa05854f8
  jnewbery:
    utACK faa05854f8
  hebasto:
    ACK faa05854f8, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 72165fbd898b92ab9a79b070993fa1faa86c2e3545b6645e72c652bda295d5107bc298d0482bf3aaf0926fc0c3e6418a445c0e073b08568c44231f547f76a688
2020-12-04 11:07:28 +01:00
MarcoFalke
283f22cabb
Merge #20461: rpc: Validate -rpcauth arguments
053b4fbad8 doc: Release note regarding -rpcauth validation (João Barbosa)
46001323b1 rpc: Validate -rpcauth arguments (João Barbosa)
d37c813a43 rpc: Refactor to process -rpcauth once (João Barbosa)

Pull request description:

  Invalid `-rpcauth` arguments are currently silently ignored. This make server initialization fail if any `-rpcauth` is invalid.

ACKs for top commit:
  MarcoFalke:
    review ACK 053b4fbad8
  jonatack:
    ACK 053b4fbad8
  ryanofsky:
    Code review ACK 053b4fbad8. Only changes since last review are moving a variable declaration and adding a comment, release notes, and a `const`.

Tree-SHA512: c99923d4a121f0c9f882b07f5402ea53e9b2d9455ad34468a094ffab1d64df26c82e1279734c0d42bc2e113eae7b581fbc3be52f3ed4a2d7450d11793afcf406
2020-12-02 09:37:37 +01:00
Emil Engler
1d578c078f
doc: Add bash as an OpenBSD dependency
If we require Python for the test framework, we should also require
bash. It is required for the linters and other scripts and does not
comes in a default OpenBSD installation.
2020-11-26 21:34:25 +01:00
Wladimir J. van der Laan
0918eb49d5 doc: Document current boost dependency as 1.71.0
This was forgotten in #19764.
2020-11-24 10:36:07 +01:00
MarcoFalke
fac5efe730
util: Add Assume() identity function 2020-11-24 09:47:29 +01:00
João Barbosa
053b4fbad8 doc: Release note regarding -rpcauth validation 2020-11-23 21:02:54 +00:00
Wladimir J. van der Laan
555b5d1bf9
Merge #20419: build: set minimum supported macOS to 10.14
a52ecc936a build: set minimum supported macOS to 10.14 (fanquake)

Pull request description:

  This is a requirement for C++17 support. See my comments [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643722538):

  > You cannot use std::get with std::variant on macOS < 10.14, because Apples libc++ doesn't support the std::bad_variant_access exception. [Relevant comment](https://github.com/bitcoin/bitcoin/pull/19183#discussion_r439794318) in #19183.

  > While we could work around this in our own code, using std::get_if, this would still be a problem for 3rd-party dependencies.

  > I've been testing Qt 5.15LTS (we'll have to enable C++17 in qt, and may upgrade to a newer version at the same time), and you can't enable -std c++17, while targeting a macOS deployment version < 10.14, configuring will fail. They are making use of std::get with std::variant throughout their cocoa code.

  We would have to had to have bumped to at least 10.13 in any case, as Qt 5.15 (#19716) [requires 10.13+](https://doc.qt.io/qt-5/supported-platforms.html).

ACKs for top commit:
  hebasto:
    ACK a52ecc936a, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: f669b2fc777aeea1e9afdbbc7bd9afe3997418211db6ba53c934cae0e62a9b999603da539518c229f34961d275c9e2f315c7b022cf5fb97bd201a69c85d470cc
2020-11-23 14:24:58 +01:00
Wladimir J. van der Laan
86bf3ae3b5
Merge #20202: wallet: Make BDB support optional
d52f502b1e Fix mock SQLiteDatabases (Andrew Chow)
99309ab3e9 Allow disabling BDB in configure with --without-bdb (Andrew Chow)
ee47f11f73 GUI: Force descriptor wallets when BDB is not compiled (Andrew Chow)
71e40b33bd RPC: Require descriptors=True for createwallet when BDB is not compiled (Andrew Chow)
6ebc41bf9c Enforce salvage is only for BDB wallets (Andrew Chow)
a58b719cf7 Do not compile BDB things when USE_BDB is defined (Andrew Chow)
b33af48210 Include wallet/bdb.h where it is actually being used (Andrew Chow)

Pull request description:

  Adds a `--without-bdb` option to `configure` which disables the compilation of the BDB stuff. Legacy wallets will not be created when BDB is not compiled. A legacy-sqlite wallet can be loaded, but we will not create them.

  Based on #20156 to resolve the situation where both `--without-sqlite` and `--without-bdb` are provided. In that case, the wallet is disabled and `--disable-wallet` is effectively set.

ACKs for top commit:
  laanwj:
    Code review ACK d52f502b1e

Tree-SHA512: 5a92ba7a542acc2e27003e9d4e5940e0d02d5c1f110db06cdcab831372bfd83e8d89c269caff31dd5bff062c1cf5f04683becff12bd23a33be731676f346553d
2020-11-23 10:30:01 +01:00
MarcoFalke
d4159984c3
Merge #20223: build: Drop the leading 0 from the version number
8f7b930475 Drop the leading 0 from the version number (Andrew Chow)

Pull request description:

  Removes the leading 0 from the version number. The minor version, which we had been using as the major version, is now the major version. The revision, which we had been using as the minor version, is now the minor version. The revision number is dropped. The build number is promoted to being part of the version number. This also avoids issues where it was accidentally not included in the version number.

  The CLIENT_VERSION remains the same format as previous as previously, as the Major version was 0 so it never actually got included in it.

  The user agent string formatter is updated to follow this new versioning.

  ***

  Honestly I'm just tired of all of the people asking for "1.0" that maybe this'll shut them up. Skip the whole 1.0 thing and go straight to version 22.0!

  Also, this means that the terminology we commonly use lines up with how the variables are named. So major versions are actually bumping the major version number, etc.

ACKs for top commit:
  jnewbery:
    Code review ACK 8f7b930475
  MarcoFalke:
    review ACK 8f7b930475 🎻

Tree-SHA512: b5c3fae14d4c0a9c0ab3b1db7c949ecc0ac3537646306b13d98dd0efc17c489cdd16d43f0a24aaa28e9c4a92ea360500e05480a335b03f9fb308010cdd93a436
2020-11-20 15:42:07 +01:00
MarcoFalke
db58b857f7
Merge #20329: docs/descriptors.md: Remove hardened marker in the path after xpub
dc80a7d0b0 docs/descriptors.md: Remove hardened marker in the path after xpub (Dmitry Petukhov)

Pull request description:

  As described in "Key origin identification" section, a descriptor
  that has hardened derivation after xpub does not let you compute scripts
  without access to the corresponding private keys. Such a descriptor is
  practically useless.

  The text after the descriptor said "with child key *1'/2* of the
  specified xpub", and clearly an xpub cannot have "child key" with
  hardened derivation. Therefore it makes sense to fix this inconsistency
  to not confuse the reader of the doc

Top commit has no ACKs.

Tree-SHA512: 9f35951d90868585303681c27ea2ef9d36d4036181e337cfbd48730de0c346320b08dd089350b116d4c8542f3b506868cf318796bb713e5f80600ccbd96f9970
2020-11-19 17:59:20 +01:00
fanquake
ea7926527c
Merge #20413: build: Require C++17 compiler
fac7198728 Use std::make_unique (MarcoFalke)
faaee810e6 build: Require C++17 compiler (MarcoFalke)

Pull request description:

  Developers have been compiling with C++17 for a few months now (fuzz tests and the msvc build have it even enabled by default). According to #16684, the 22.0 release shall be compiled with C++17 enabled.

  This only sets the build flag, any other changes need more discussion and can be done later.

ACKs for top commit:
  elichai:
    utACK fac7198728
  hebasto:
    ACK fac7198728, I've locally compiled on ARM 32bit SBC without GUI.
  fanquake:
    ACK fac7198728

Tree-SHA512: 53eb40ba5d496376a2d2cf16e2000bef36616cc2a3696c3ec59a5366e41fa8b872817a7ca21751f030f9c1efb339dfa63cc655eaa5199b9a3d2e52c2de0ccb29
2020-11-19 10:29:05 +08:00
Andrew Chow
8f7b930475 Drop the leading 0 from the version number
Removes the leading 0 from the version number. The minor version, which
we had been using as the major version, is now the major version. The
revision, which we had been using as the minor version, is now the minor
version. The revision number is dropped. The build number is promoted to
being part of the version number. This also avoids issues where it was
accidentally not included in the version number.

The CLIENT_VERSION remains the same format as previous as previously,
the Major version was 0 so that was never a factor in CLIENT_VERSION.
2020-11-18 12:00:57 -05:00
Andrew Chow
99309ab3e9 Allow disabling BDB in configure with --without-bdb 2020-11-18 11:56:12 -05:00
MarcoFalke
faaee810e6
build: Require C++17 compiler 2020-11-18 15:15:04 +01:00
fanquake
a52ecc936a
build: set minimum supported macOS to 10.14 2020-11-18 21:46:09 +08:00
MarcoFalke
50e019a97a
Merge #20414: doc: Remove generated manual pages from master branch
daf1ebf0b1 doc: Remove generated manual pages from master branch (Wladimir J. van der Laan)

Pull request description:

  Replace the generated manual pages on the master branch with placeholders with instructions. The master branch is too much in flux for anything generated from command output to be kept up to date, and having pages from versions ago looks silly.

  We can't remove them completely because `make dist` relies on the files being present.

  Resolves #20062.

ACKs for top commit:
  MarcoFalke:
    review ACK daf1ebf0b1
  RiccardoMasutti:
    ACK daf1ebf

Tree-SHA512: fdd0bb777ddf66c663e4f6b00d206b3f6283ae89028e00b38d16e68a224a781e9245a25edf6f3326f9252ae2ef7c2950c591667c56d4e9d10a16fc50a6c358f4
2020-11-18 14:42:37 +01:00
fanquake
dc5a35a507
doc: clean out release notes post branch-off 2020-11-18 20:52:38 +08:00
Wladimir J. van der Laan
daf1ebf0b1 doc: Remove generated manual pages from master branch
Replace the generated manual pages on the master branch with
placeholders with instructions. The master branch is too much in flux
for anything generated from command output to be kept up to date,
and having pages from versions ago looks silly.

We can't remove them completely because `make dist` relies
on the files being present.

Resolves #20031.
2020-11-18 11:12:46 +01:00
MarcoFalke
4b24c3962f
Merge #19504: Bump minimum python version to 3.6
97c738ff1b [tests] Recommend f-strings for formatting, update feature_block to use them (Anthony Towns)
8ae9d314e9 Bump minimum python version to 3.6 (Anthony Towns)

Pull request description:

  Python 3.5 has reached [end-of-life](https://devguide.python.org/#status-of-python-branches) as of September 2020, and 3.6 has some moderately nice [features](https://docs.python.org/3/whatsnew/3.6.html):

  - `f'x = {x}'` as an alternative to `'x = {}'.format(x)` format strings (cf https://github.com/bitcoin/bitcoin/pull/13718#issuecomment-406591027)
  - underscore separators for large numbers, like `1_234_567`
  - improvements to async
  - improvements to typing module

  Note that 3.6 is not available in xenial (16.04), but is available in bionic (18.04), while focal (20.04) has 3.8. CentOS 7 and 8 have 3.6.8, Debian stable has 3.7.3, and [gentoo and arch already had 3.6 and 3.7 in 2018](https://github.com/bitcoin/bitcoin/pull/14954#issuecomment-447118707).

ACKs for top commit:
  MarcoFalke:
    re-ACK 97c738ff1b

Tree-SHA512: ec7fce68845edde4d61a42de12c065fd49e5217311a6fda1323206f091a0afd50f293645dffc27d420127e4e5deb864e953f1b67eff735a0dfbbedd7899a9d60
2020-11-18 10:24:22 +01:00
Wladimir J. van der Laan
0bd4929cd0
Merge #20284: addrman: ensure old versions don't parse peers.dat
38ada892ed addrman: ensure old versions don't parse peers.dat (Vasil Dimov)

Pull request description:

  Even though the format of `peers.dat` was changed in a backwards
  incompatible way, it is not guaranteed that old versions will fail to
  parse it. There is a chance that old versions parse its contents as
  garbage and use it.

  Old versions expect the "key size" field to be 32 and fail the parsing
  if it is not. Thus, we put something other than 32 in it. This will make
  versions between 0.11.0 and 0.20.1 deterministically fail on the new
  format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
  will still parse it as garbage.

  Also, introduce a way to increment the `peers.dat` format in a way that
  does not necessary make older versions refuse to read it.

ACKs for top commit:
  jnewbery:
    ACK 38ada892ed
  laanwj:
    Code review ACK 38ada892ed
  MarcoFalke:
    re-ACK 38ada892ed 🥐

Tree-SHA512: 550bd660c5019dba0f9c334aca8a11c4a0463cfddf11efe7a4a5585ffb05549c82b95066fba5d073ae37893e0eccc158a7ffea9b33ea031d9be4a39e44f6face
2020-11-12 17:05:51 +01:00
Vasil Dimov
38ada892ed
addrman: ensure old versions don't parse peers.dat
Even though the format of `peers.dat` was changed in an incompatible
way (old software versions <0.21 cannot understand the new file format),
it is not guaranteed that old versions will fail to parse it. There is a
chance that old versions parse its contents as garbage and use it.

Old versions expect the "key size" field to be 32 and fail the parsing
if it is not. Thus, we put something other than 32 in it. This will make
versions between 0.11.0 and 0.20.1 deterministically fail on the new
format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
(<0.11.0) will still parse it as garbage.

Also, introduce a way to increment the `peers.dat` format in a way that
does not necessary make older versions refuse to read it.
2020-11-11 16:05:15 +01:00
Anthony Towns
8ae9d314e9 Bump minimum python version to 3.6 2020-11-09 17:53:47 +10:00
Dmitry Petukhov
dc80a7d0b0
docs/descriptors.md: Remove hardened marker in the path after xpub
As described in "Key origin identification" section, a descriptor
that has hardened derivation after xpub does not let you compute scripts
without access to the corresponding private keys. Such a descriptor is
practically useless.

The text after the descriptor said "with child key *1'/2* of the
specified xpub", and clearly an xpub cannot have "child key" with
hardened derivation. Therefore it makes sense to fix this inconsistency
to not confuse the reader of the doc
2020-11-06 14:27:47 +05:00
Jon Atack
e5f3e95a8e
doc: fix getchaintxstats fields in release-process.md
also:

- use the getblockheader (and getblockhash) RPCs instead of getblockchaininfo
  for updating the nMinimumChainWork and defaultAssumeValid consensus params

- use "RPC" consistently

- update the example PR from 17002 to 20263

- improve a link with a named anchor tag
2020-11-01 22:19:24 +01:00
MarcoFalke
42b66a6b81
Merge #20186: wallet: Make -wallet setting not create wallets
01476a88a6 wallet: Make -wallet setting not create wallets (Russell Yanofsky)

Pull request description:

  This changes `-wallet` setting to only load existing wallets, not create new ones.

  - Fixes settings.json corner cases reported by sjors & promag: https://github.com/bitcoin-core/gui/issues/95, https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578, https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578

  - Prevents accidental creation of wallets reported most recently by jb55 http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-14.html#l-355

  - Simplifies behavior after #15454. #15454 took the big step of disabling creation of the default wallet. This PR extends it to avoid creating other wallets as well. With this change, new wallets just aren't created on startup, instead of sometimes being created, sometimes not. #15454 release notes are updated here and are simpler.

  This change should be targeted for 0.21.0. It's a bug fix and simplifies behavior of the #15937 / #19754 / #15454 features added in 0.21.0.

  ---

  This PR is implementing the simplest, most basic alternative listed in https://github.com/bitcoin-core/gui/issues/95#issuecomment-694236940. Other improvements mentioned there can build on top of this.

ACKs for top commit:
  achow101:
    ACK 01476a88a6
  hebasto:
    re-ACK 01476a88a6
  MarcoFalke:
    review ACK 01476a88a6 🏂

Tree-SHA512: 0d50f4e5dfbd04a2efd9fd66c02085a0ed705807bdec1cf5770d0ae8cb6af07080fb81306349937bf66acdb713d03fb35636f6442b650d0820e66cbae09c2f87
2020-10-29 15:01:39 +01:00
Wladimir J. van der Laan
f3727fd735
Merge #20156: build: Make sqlite support optional (compile-time)
bbb42a6896 RPC: createwallet: Nicer error message if descriptor wallet requested and sqlite support not compiled in (Luke Dashjr)
6608fec332 GUI: Create Wallet: Nicely disable descriptor wallet checkbox if sqlite support not compiled in (Luke Dashjr)
7b54d768e1 Make sqlite support optional (compile-time) (Luke Dashjr)

Pull request description:

  As a new requirement, sqlite support should be optional. This PR aims to be only minimum/blocker changes for 0.21.

  Potential follow-up PRs after this:
  * Make BDB support optional
  * Nicer error messages when user tries to load an unsupported wallet
  * Don't compile descriptor wallet code if sqlite disabled

ACKs for top commit:
  jonasschnelli:
    Tested ACK bbb42a6896
  achow101:
    ACK bbb42a6896
  Sjors:
    re-utACK bbb42a6896
  hebasto:
    ACK bbb42a6896, tested on Linux Mint 20 (x86_64, Qt 5.12.8).

Tree-SHA512: 500209dd1971310fab8ae51543343ce0ba91f088ccccff6109b4cc27547cd5532289dca6cb7dac2a7d7c59cdf3c8f5aacc31e9b0f912e38cea52ec26b97100bd
2020-10-29 12:03:36 +01:00
fanquake
67d4643a1a
Merge #20152: doc: Update wallet files in files.md
defe48a51f doc: Update wallet files in files.md (Hennadii Stepanov)

Pull request description:

  This PR is a #19077 follow up, and it addresses the [comment](https://github.com/bitcoin/bitcoin/pull/19077#discussion_r504805234):

  > If need to update, there are two corrections that could be made:
  >
  >  * Line 69 "Wallets are Berkeley DB (BDB) databases" is no longer true
  >
  >  * Line 76 "Wallet lock file" should say "BDB wallet lock file"

ACKs for top commit:
  RiccardoMasutti:
    ACK defe48a
  meshcollider:
    ACK defe48a51f

Tree-SHA512: 39939f86a9c7842bf06913998305dcbd6209585f1da0fe9c274bac0572eb8464e59176884dd9e2b91312f34efad40cdeb4085ec72c2a2c1b33d16b6ab505140c
2020-10-27 15:43:53 +08:00
Russell Yanofsky
01476a88a6 wallet: Make -wallet setting not create wallets
This changes -wallet setting to only load existing wallets, not create new ones.

- Fixes settings.json corner cases reported by sjors & promag:
  https://github.com/bitcoin-core/gui/issues/95,
  https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578,
  https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578

- Prevents accidental creation of wallets reported most recently by jb55
  http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-14.html#l-355

- Simplifies behavior after #15454. #15454 took the big step of disabling
  creation of the default wallet. This PR extends it to avoid creating other
  wallets as well. With this change, new wallets just aren't created on
  startup, instead of sometimes being created, sometimes not. #15454 release
  notes are updated here and are simpler.

This change should be targeted for 0.21.0. It's a bug fix and simplifies
behavior of the #15937 / #19754 / #15454 features added in 0.21.0.
2020-10-21 08:48:43 -04:00
Luke Dashjr
7b54d768e1 Make sqlite support optional (compile-time) 2020-10-20 13:44:43 +00:00
MarcoFalke
fa38093bee
doc: Merge release notes 2020-10-19 12:00:16 +02:00
Pieter Wuille
5669642a0b docs: mention BIPs 340-342 in doc/bips.md 2020-10-15 14:20:20 -07:00
Wladimir J. van der Laan
9855422e65
Merge #17428: p2p: Try to preserve outbound block-relay-only connections during restart
a490d074b3 doc: Add anchors.dat to files.md (Hennadii Stepanov)
0a85e5a7bc p2p: Try to connect to anchors once (Hennadii Stepanov)
5543c7ab28 p2p: Fix off-by-one error in fetching address loop (Hennadii Stepanov)
4170b46544 p2p: Integrate DumpAnchors() and ReadAnchors() into CConnman (Hennadii Stepanov)
bad16aff49 p2p: Add CConnman::GetCurrentBlockRelayOnlyConns() (Hennadii Stepanov)
c29272a157 p2p: Add ReadAnchors() (Hennadii Stepanov)
567008d2a0 p2p: Add DumpAnchors() (Hennadii Stepanov)

Pull request description:

  This is an implementation of #17326:
  - all (currently 2) outbound block-relay-only connections (#15759) are dumped to `anchors.dat` file
  - on restart a node tries to connect to the addresses from `anchors.dat`

  This PR prevents a type of eclipse attack when an attacker exploits a victim node restart to force it to connect to new, probably adversarial, peers.

ACKs for top commit:
  jnewbery:
    code review ACK a490d074b3
  laanwj:
    Code review ACK a490d074b3

Tree-SHA512: 0f5098a3882f2814be1aa21de308cd09e6654f4e7054b79f3cfeaf26bc02b814ca271497ed00018d199ee596a8cb9b126acee8b666a29e225b08eb2a49b02ddd
2020-10-15 20:19:55 +02:00
Hennadii Stepanov
defe48a51f
doc: Update wallet files in files.md 2020-10-15 11:32:26 +03:00
Samuel Dobson
8ed37f6c84
Merge #19077: wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets
c4a29d0a90 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky)
310b0fde04 Run dumpwallet for legacy wallets only in  wallet_backup.py (Andrew Chow)
6c6639ac9f Include sqlite3 in documentation (Andrew Chow)
f023b7cac0 wallet: Enforce sqlite serialized threading mode (Andrew Chow)
6173269866 Set and check the sqlite user version (Andrew Chow)
9d3d2d263c Use network magic as sqlite wallet application ID (Andrew Chow)
9af5de3798 Use SQLite for descriptor wallets (Andrew Chow)
9b78f3ce8e walletutil: Wallets can also be sqlite (Andrew Chow)
ac38a87225 Determine wallet file type based on file magic (Andrew Chow)
6045f77003 Implement SQLiteDatabase::MakeBatch (Andrew Chow)
727e6b2a4e Implement SQLiteDatabase::Verify (Andrew Chow)
b4df8fdb19 Implement SQLiteDatabase::Rewrite (Andrew Chow)
010e365906 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow)
ac5c1617e7 Implement SQLiteDatabase::Backup (Andrew Chow)
f6f9cd6a64 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow)
bf90e033f4 Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow)
7aa45620e2 Add SetupSQLStatements (Andrew Chow)
6636a2608a Implement SQLiteBatch::Close (Andrew Chow)
93825352a3 Implement SQLiteDatabase::Close (Andrew Chow)
a0de83372b Implement SQLiteDatabase::Open (Andrew Chow)
3bfa0fe125 Initialize and Shutdown sqlite3 globals (Andrew Chow)
5a488b3d77 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow)
ca8b7e04ab Implement SQLiteDatabaseVersion (Andrew Chow)
7577b6e1c8 Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow)
e87df82580 Add sqlite to travis and depends (Andrew Chow)
54729f3f4e Add libsqlite3 (Andrew Chow)

Pull request description:

  This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`.

  For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite.

  We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use.

  I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier.

ACKs for top commit:
  Sjors:
    re-utACK c4a29d0a90
  promag:
    Tested ACK c4a29d0a90.
  fjahr:
    reACK c4a29d0a90
  S3RK:
    Re-review ACK c4a29d0a90
  meshcollider:
    re-utACK c4a29d0a90
  hebasto:
    re-ACK c4a29d0a90, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`.
  ryanofsky:
    Code review ACK c4a29d0a90. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction.
  jonatack:
    ACK c4a29d0a90, debug builds and test runs after rebase to latest master @ c2c4dbaebd, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns.

Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
2020-10-15 20:12:29 +13:00
Wladimir J. van der Laan
c2c4dbaebd
Merge #19988: Overhaul transaction request logic
fd9a0060f0 Report and verify expirations (Pieter Wuille)
86f50ed10f Delete limitedmap as it is unused now (Pieter Wuille)
cc16fff3e4 Make txid delay penalty also apply to fetches of orphan's parents (Pieter Wuille)
173a1d2d3f Expedite removal of tx requests that are no longer needed (Pieter Wuille)
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers (Pieter Wuille)
242d16477d Change transaction request logic to use txrequest (Pieter Wuille)
5b03121d60 Add txrequest fuzz tests (Pieter Wuille)
3c7fe0e5a0 Add txrequest unit tests (Pieter Wuille)
da3b8fde03 Add txrequest module (Pieter Wuille)

Pull request description:

  This replaces the transaction request logic with an encapsulated class that maintains all the state surrounding it. By keeping it stand alone, it can be easily tested (using included unit tests and fuzz tests).

  The major changes are:

  * Announcements from outbound (and whitelisted) peers are now always preferred over those from inbound peers. This used to be the case for the first request (by delaying the first request from inbound peers), and a bias afters. The 2s delay for requests from inbound peers still exists, but after that, if viable outbound peers remain for any given transaction, they will always be tried first.
  * No more hard cap of 100 in flight transactions per peer, as there is less need for it (memory usage is linear in the number of announcements, but independent from the number in flight, and CPU usage isn't affected by it). Furthermore, if only one peer announces a transaction, and it has over 100 in flight already, we still want to request it from them. The cap is replaced with a rule that announcements from such overloaded peers get an additional 2s delay (possibly combined with the existing 2s delays for inbound connections, and for txid peers when wtxid peers are available).
  * The limit of 100000 tracked announcements is reduced to 5000; this was excessive. This can be bypassed using the PF_RELAY permission (to accommodate locally dumping a batch of many transactions).

  This replaces #19184, rebased on #18044 and with many small changes.

ACKs for top commit:
  ariard:
    Code Review ACK fd9a006. I've reviewed the new TxRequestTracker, its integration in net_processing, unit/functional/fuzzing test coverage. I looked more for soundness of new specification rather than functional consistency with old transaction request logic.
  MarcoFalke:
    Approach ACK fd9a0060f0 🏹
  naumenkogs:
    Code Review ACK fd9a006. I've reviewed everything, mostly to see how this stuff works at the lower level (less documentation-wise, more implementation-wise), and to try breaking it with unexpected sequences of events.
  jnewbery:
    utACK fd9a0060f0
  jonatack:
    WIP light ACK fd9a0060f0 have read the code, verified that each commit is hygienic, e.g. debug build clean and tests green, and have been running a node on and off with this branch and grepping the net debug log. Am still unpacking the discussion hidden by GitHub by fetching it via the API and connecting the dots, storing notes and suggestions in a local branch; at this point none are blockers.
  ryanofsky:
    Light code review ACK fd9a0060f0, looking at txrequest implementation, unit test implementation, and net_processing integration, just trying to understand how it works and looking for anything potentially confusing in the implementation. Didn't look at functional tests or catch up on review discussion. Just a sanity check review focused on:

Tree-SHA512: ea7b52710371498b59d9c9cfb5230dd544fe9c6cb699e69178dea641646104f38a0b5ec7f5f0dbf1eb579b7ec25a31ea420593eff3b7556433daf92d4b0f0dd7
2020-10-14 18:36:59 +02:00
Andrew Chow
6c6639ac9f Include sqlite3 in documentation 2020-10-14 11:28:18 -04:00
fanquake
e21b824386
Merge #20109: Release notes and followups from 19339
88197b0769 [doc] release notes for max fee checking (gzhao408)
c201d73df3 style and nits for fee-checking in BroadcastTransaction (gzhao408)

Pull request description:

  Pretty trivial... addresses some tiny comments from #19339. Also fixes a docs typo from #19940 and adds a release note about the error message change for testmempoolaccept.

ACKs for top commit:
  achow101:
    ACK 88197b0769
  MarcoFalke:
    cr re-ACK 88197b0769

Tree-SHA512: fff16d731426b9b4db5222df02633983402f4c7241551eec98bb1554145dbdc132f40ed8ca4abd5edcebe1f4d1e879fb6d11bd91730604f6552c10cdf65706a1
2020-10-14 12:19:42 +08:00
Pieter Wuille
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers
Maintaining up to 100000 INVs per peer is excessive, as that is far more
than fits in a typical mempool.

Also disable the "overload" penalty for PF_RELAY peers.
2020-10-12 12:14:53 -07:00
Pieter Wuille
79f3d9b932 Mention BIP155 in doc/bips.md 2020-10-11 11:29:07 -07:00
fanquake
0b2abaa666
Merge #19954: Complete the BIP155 implementation and upgrade to TORv3
dcf0cb4776 tor: make a TORv3 hidden service instead of TORv2 (Vasil Dimov)
353a3fdaad net: advertise support for ADDRv2 via new message (Vasil Dimov)
201a4596d9 net: CAddress & CAddrMan: (un)serialize as ADDRv2 (Vasil Dimov)
1d3ec2a1fd Support bypassing range check in ReadCompactSize (Pieter Wuille)

Pull request description:

  This PR contains the two remaining commits from #19031 to complete the [BIP155](https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki) implementation:

  `net: CAddress & CAddrMan: (un)serialize as ADDRv2`
  `net: advertise support for ADDRv2 via new message`

  plus one more commit:

  `tor: make a TORv3 hidden service instead of TORv2`

ACKs for top commit:
  jonatack:
    re-ACK dcf0cb4776 per `git diff 9b56a68 dcf0cb4` only change since last review is an update to the release notes which partially picked up the suggested text. Running a node on this branch and addnode-ing to 6 other Tor v3 nodes, I see "addrv2" and "sendaddrv2" messages in getpeerinfo in both the "bytesrecv_per_msg" and "bytessent_per_msg" JSON objects.
  sipa:
    ACK dcf0cb4776
  hebasto:
    re-ACK dcf0cb4776, the node works flawlessly in all of the modes: Tor-only, clearnet-only, mixed.
  laanwj:
    Edit: I have to retract this ACK for now, I'm having some problems with this PR on a FreeBSD node. It drops all outgoing connections with this dcf0cb4776 merged on master (12a1c3ad1a).
  ariard:
    Code Review ACK dcf0cb4

Tree-SHA512: 28d4d0d817b8664d2f4b18c0e0f31579b2f0f2d23310ed213f1f436a4242afea14dfbf99e07e15889bc5c5c71ad50056797e9307ff8a90e96704f588a6171308
2020-10-11 08:51:57 +08:00
gzhao408
88197b0769 [doc] release notes for max fee checking 2020-10-09 08:58:47 -07:00
Vasil Dimov
dcf0cb4776
tor: make a TORv3 hidden service instead of TORv2
TORv2 is deprecated [1], thus whenever we create the hidden service
ourselves create a TORv3 one instead.

[1] https://blog.torproject.org/v2-deprecation-timeline
2020-10-09 16:42:50 +02:00
Vasil Dimov
201a4596d9
net: CAddress & CAddrMan: (un)serialize as ADDRv2
Change the serialization of `CAddrMan` to serialize its addresses
in ADDRv2/BIP155 format by default. Introduce a new `CAddrMan` format
version (3).

Add support for ADDRv2 format in `CAddress` (un)serialization.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-10-09 16:42:49 +02:00
Hennadii Stepanov
a490d074b3
doc: Add anchors.dat to files.md 2020-10-09 14:30:09 +03:00
fanquake
12a1c3ad1a
Merge #20107: doc: Collect release-notes snippets
faa0847dec doc: Add release notes for #20101 (MarcoFalke)
99992e7832 doc: Collect release-notes snippets (MarcoFalke)

Pull request description:

  Also add a note for #20101

ACKs for top commit:
  fanquake:
    ACK faa0847dec - no need to bike-shed here as these will all get massaged to death in the wiki anyways.

Tree-SHA512: 63d3597e2bbd422ec182e76112110477d22e3afedb479114ddec958405bcdd63492df9477267aac65605612af49c0aff6246b1bc3d41dd606d6d61c30117c109
2020-10-09 18:08:14 +08:00
MarcoFalke
faa0847dec
doc: Add release notes for #20101 2020-10-08 15:11:25 +02:00
Hennadii Stepanov
2dc79c4264
doc: Update and improve files.md
Added the `signet` subdirectory and the `ip_asn.map` file.
2020-10-06 11:05:03 +03:00
Wladimir J. van der Laan
df2129a234
Merge #19991: net: Use alternative port for incoming Tor connections
96571b3d4c doc: Update onion service target port numbers in tor.md (Hennadii Stepanov)
bb145c9050 net: Extend -bind config option with optional network type (Hennadii Stepanov)
92bd3c1da4 net, refactor: Move AddLocal call one level up (Hennadii Stepanov)
57f17e57c8 net: Pass onion service target to Tor controller (Hennadii Stepanov)
e3f07851f0 refactor: Rename TorController::target to m_tor_control_center (Hennadii Stepanov)
fdd3ae4d26 net, refactor: Refactor CBaseChainParams::RPCPort function (Hennadii Stepanov)
a5266d4546 net: Add alternative port for onion service (Hennadii Stepanov)
b3273cf403 net: Use network byte order for in_addr.s_addr (Hennadii Stepanov)

Pull request description:

  This PR adds ability to label incoming Tor connections as different from normal localhost connections.

  Closes #8973.
  Closes #16693.

  Default onion service target ports are:
  - 8334 on mainnnet
  - 18334 on testnet
  - 38334 on signet
  - 18445 on regtest

  To set the onion service target socket manually the extended `-bind` config option could be used:

  ```
  $ src/bitcoind -help | grep -A 6 -e '-bind'
    -bind=<addr>[:<port>][=onion]
         Bind to given address and always listen on it (default: 0.0.0.0). Use
         [host]:port notation for IPv6. Append =onion to tag any incoming
         connections to that address and port as incoming Tor connections
         (default: 127.0.0.1:8334=onion, testnet: 127.0.0.1:18334=onion,
         signet: 127.0.0.1:38334=onion, regtest: 127.0.0.1:18445=onion)

  ```

  Since [pr19991.02 update](https://github.com/bitcoin/bitcoin/pull/19991#issuecomment-698882284) this PR is an alternative to #19043.

ACKs for top commit:
  Sjors:
    re-utACK 96571b3d4c
  vasild:
    ACK 96571b3d4
  laanwj:
    Re-ACK 96571b3d4c

Tree-SHA512: cb0eade80f4b3395f405f775e1b89c086a1f09d5a4464df6cb4faf808d9c2245474e1720b2b538f203f6c1996507f69b09f5a6e35ea42633c10e22bd733d4438
2020-10-02 13:37:23 +02:00
MarcoFalke
99992e7832
doc: Collect release-notes snippets 2020-10-02 11:24:46 +02:00