mirror of
https://github.com/bitcoinj/bitcoinj.git
synced 2025-03-13 11:36:15 +01:00
Merge 6949b480f9 into 1090649211
This commit is contained in:
Andreas Schildbach
GitHub
commit
26c47c3492
1 changed files with 38 additions and 57 deletions
|
|
@ -1,9 +1,15 @@
|
|||
Design of deterministic wallets support
|
||||
---------------------------------------
|
||||
Design of hierarchical deterministic (HD) wallets support
|
||||
---------------------------------------------------------
|
||||
|
||||
Goals:
|
||||
- Wallets to derive new keys deterministically using the BIP32 algorithm.
|
||||
- Seamless/silent upgrade of old wallets to use the new scheme
|
||||
- Wallets to derive new keys deterministically using the BIP32 algorithm
|
||||
- Support different wallet structures:
|
||||
- The default BIP32 structure, described in the later part of the document
|
||||
- BIP43/44/84/86
|
||||
- Custom, by implementing the KeyChainGroupStructure interface
|
||||
- Support multiple output script types (P2PKH, P2WPKH)
|
||||
- One active keychain per script type, so that compatible addresses can be generated
|
||||
- Seamless upgrade to new script types
|
||||
- Support watching a key tree without knowing the private key
|
||||
- Integrate with existing features like key rotation, encryption and bloom filtering
|
||||
- Clean up and reduce the size of the Wallet class
|
||||
|
|
@ -12,52 +18,52 @@ Goals:
|
|||
|
||||
Non-goals:
|
||||
- Expose multiple accounts or "wallets within a wallet" via the API.
|
||||
- Optional support for HD: all wallets after the change will be HD by default, even though they may have
|
||||
- Optional support for HD: all wallets are HD by default, even though they may have
|
||||
non-HD keys imported into them.
|
||||
- Actual support for hardware wallets
|
||||
|
||||
|
||||
API design
|
||||
-----------
|
||||
----------
|
||||
|
||||
Create a new KeyChain interface and provide BasicKeyChain, DeterministicKeyChain implementations.
|
||||
|
||||
Wallets may contain multiple key chains. However only the last one is "active" in the sense that it will be used to
|
||||
create new keys. There's no way to change that.
|
||||
Wallets may contain multiple keychains per script type. However only the last one for each script type is "active"
|
||||
in the sense that it will be used to create new keys. There's no way to change that.
|
||||
|
||||
The Wallet class has most key handling code refactored out into KeyChainGroup, which handles multiplexing a
|
||||
BasicKeyChain (for random keys, if any), and zero or more DeterministicKeyChain. Wallet ends up just forwarding method
|
||||
calls to this class most of the time. Thus in this section where the Wallet API is discussed, it can be assumed that
|
||||
KeyChainGroup has the same API. Although individual key chain objects have their own locks and are expected to be thread
|
||||
KeyChainGroup has the same API. Although individual keychain objects have their own locks and are expected to be thread
|
||||
safe, KeyChainGroup itself is not and is not exposed directly by Wallet: it's an implementation detail, and locked under
|
||||
the Wallet lock.
|
||||
|
||||
The Wallet API changes to have an importKey method that works like addKey does today, and forwards to the BasicKeyChain.
|
||||
There's also a freshKey method that forwards to the active HD chain and requests a key for a specific purpose,
|
||||
There's also a freshKey method that forwards to the active HD keychain and requests a key for a specific purpose,
|
||||
specified by an enum parameter. The freshKey method supports requesting keys for the following purposes:
|
||||
|
||||
- CHANGE
|
||||
- RECEIVE_FUNDS
|
||||
|
||||
and may in future also have additional purposes like for micropayment channels, etc. These map to the notion of
|
||||
and may in future also have additional purposes like for layer 2 protocols, etc. These map to the notion of
|
||||
"accounts" as defined in the BIP32 spec, but otherwise should not be exposed in any user interfaces. freshKey is
|
||||
guaranteed to return a freshly generated key: it will not return the same key repeatedly. There is also a currentKey
|
||||
method that returns a stable key suitable for display in the user interface: it will be changed automatically when
|
||||
it's observed being used in a transaction.
|
||||
|
||||
There can be multiple key chains. There is always:
|
||||
There can be multiple keychains. There is always:
|
||||
|
||||
* 1 basic key chain, though it may be empty.
|
||||
* >=0 deterministic key chains
|
||||
* 1 basic keychain, though it may be empty.
|
||||
* >=0 deterministic keychains
|
||||
|
||||
Thus it's possible to have more than one deterministic key chain, but not more than one basic key chain.
|
||||
Thus it's possible to have more than one deterministic keychain, but not more than one basic keychain.
|
||||
|
||||
Multiple deterministic key chains become relevant when key rotation happens. Individual keys in a deterministic
|
||||
hierarchy do not rotate. Instead the rotation time is applied only to the seed. Either the whole key chain rotates or
|
||||
Multiple deterministic keychains become relevant when key rotation happens. Individual keys in a deterministic
|
||||
hierarchy do not rotate. Instead the rotation time is applied only to the seed. Either the whole keychain rotates or
|
||||
none of it does. This reflects the fact that a hierarchy only has one real secret and in the case of a wallet
|
||||
compromise, all keys derived from that secret must also be rotated.
|
||||
|
||||
Multiple key chains within a wallet are NOT intended to support the following use cases:
|
||||
Multiple keychains within a wallet are NOT intended to support the following use cases:
|
||||
|
||||
- Watching wallets
|
||||
- Hardware wallets
|
||||
|
|
@ -65,15 +71,15 @@ Multiple key chains within a wallet are NOT intended to support the following us
|
|||
From a user interface design perspective, it does not make much sense to have a wallet that can have multiple
|
||||
"subwallets", as it would rapidly get confusing especially when trying to spend money from multiple kinds of
|
||||
hierarchy at once. Key rotation is a special case because it is expected that (a) the private key material is always
|
||||
available on a rotating chain and (b) the only spends crafted for keys in these chains will be created internally by
|
||||
available on a rotating keychain and (b) the only spends crafted for keys in these keychains will be created internally by
|
||||
bitcoinj and thus there is virtually no added complexity to the API or UI to handle it.
|
||||
|
||||
When chains are encrypted, they must all be encrypted under exactly the same key. This is also true of rotating chains.
|
||||
When keychains are encrypted, they must all be encrypted under exactly the same key. This is also true of rotating keychains.
|
||||
The API will not support the case of multiple keychains in the same wallet that have a different password, as this would
|
||||
make the API and wallet apps too confusing.
|
||||
|
||||
For hardware/watching wallets, where the private key/seed material is not available, the correct approach is to create
|
||||
an entirely separate Wallet object with a single deterministic key chain that does not have access to the seed. Special
|
||||
an entirely separate Wallet object with a single deterministic keychain that does not have access to the seed. Special
|
||||
support can be added to the wallet code later to handle "external signing" which would be a separate/new feature.
|
||||
External signing can be done today already of course by manually building a transaction and using hashForSignature, but
|
||||
it's less convenient than an integrated solution would be.
|
||||
|
|
@ -81,15 +87,13 @@ it's less convenient than an integrated solution would be.
|
|||
Chain structure
|
||||
---------------
|
||||
|
||||
We follow the suggested chain structure outlined in BIP32 in which there is the notion of a top-level account, though
|
||||
in bitcoinj this will always be zero as it won't be exposed in the API for now. Under the account are two keys, one
|
||||
for receiving of funds (i.e. exposed to the use) and one which is used for "internal purposes", typically change keys
|
||||
though it may also be used for things like micropayment channels and, in future, de/refragmentation.
|
||||
We follow the suggested keychain structure outlined in BIP32, or BIP 43/44/84/86. Custom structures can be implemented
|
||||
via the KeyChainGroupStructure interface.
|
||||
|
||||
In the most obvious implementation, a standard key chain would have all private or all pubkey only nodes. However,
|
||||
In the most obvious implementation, a standard keychain would have all private or all pubkey only nodes. However,
|
||||
this will not be the case in bitcoinj. Instead private keys will largely be rederived on the fly, for a few reasons:
|
||||
|
||||
1) When a chain is encrypted, the private key/seed bytes are not available, yet the chain still needs to be extended
|
||||
1) When a keychain is encrypted, the private key/seed bytes are not available, yet the keychain still needs to be extended
|
||||
on demand. In this sense an encrypted wallet is somewhat like a watching wallet.
|
||||
2) Private keys can be derived from their parent extremely fast, as it merely involves a single bigint modular addition.
|
||||
3) We would like to hold as many keys in RAM as possible, and so throwing away and rederiving private key bytes on the
|
||||
|
|
@ -101,7 +105,7 @@ due to the need to calculate public keys in the gap.
|
|||
|
||||
Because it would be complicated and confusing to have some user-exposed keys that have an encrypted private part and
|
||||
others that are missing it entirely, and because private key derivation is fast, we use the following arrangement for
|
||||
an encrypted deterministic key chain:
|
||||
an encrypted deterministic keychain:
|
||||
|
||||
- The root seed is encrypted. We keep it around even after the master private key is derived from it, because wallet
|
||||
apps may wish to show it to users so they can opt to write it down later.
|
||||
|
|
@ -115,13 +119,13 @@ an encrypted deterministic key chain:
|
|||
Bloom filtering
|
||||
---------------
|
||||
|
||||
Each key chain is responsible for implementing PeerFilterProvider, the wallet multiplexes all implementors together.
|
||||
Each keychain is responsible for implementing PeerFilterProvider, the wallet multiplexes all implementors together.
|
||||
The code that currently does this multiplexing (between Wallets) would be extracted and reused, resulting in what is
|
||||
internally a hierarchy of filter providers whose outputs are combined to result in a single Bloom filter and earliest
|
||||
key time, calculated by the PeerGroup.
|
||||
|
||||
When a wallet is informed about a transaction, it compiles a list of what pubkeys were seen and sends those lists
|
||||
to each key chain. The basic key chain ignores this and does nothing. The deterministic key chain examines each key
|
||||
to each keychain. The basic keychain ignores this and does nothing. The deterministic keychain examines each key
|
||||
to see if it's within the "gap", which is defined as a set of keys that have been pre-generated but not yet used by
|
||||
the wallet for change or returned to the user.
|
||||
|
||||
|
|
@ -130,13 +134,13 @@ wallet "in the future" are recognized and added to the Bloom filters. The gap mu
|
|||
that more than that number of keys will be consumed in a single block (or more accurately, in a single getdata run).
|
||||
For version one of this system, the gap will be manually sized to be appropriate for desktop wallets in typical use
|
||||
cases: web servers that are tracking very high traffic addresses might be at risk of exhausting the gap, and that would
|
||||
be treated as a fatal error. If the gap is exhausted before the key chain is asked to recalculate a new Bloom filter,
|
||||
be treated as a fatal error. If the gap is exhausted before the keychain is asked to recalculate a new Bloom filter,
|
||||
an exception is sent to the user-provided exception handler, which wallets should be treated as fatal and cause a
|
||||
crash (the user would not be able to sync beyond that point and would need help). In future, the gap should be resized
|
||||
on the fly and blocks re-downloaded if traffic seems to be higher than expected, but that's more complex and can be
|
||||
handled by future work.
|
||||
|
||||
When the deterministic key chain is informed that a key has been observed, it finds its offset in the gap list, and
|
||||
When the deterministic keychain is informed that a key has been observed, it finds its offset in the gap list, and
|
||||
then extends the gap by that amount of keys. Thus if a key is seen that is 10 keys in the future, the gap would be
|
||||
extended by 10 keys, where "extended" means the BIP32 algorithm is run to derive more keys and they would become
|
||||
persisted to disk/held in memory. Extension does not automatically result in recalculation of the bloom filter (see
|
||||
|
|
@ -164,7 +168,7 @@ Encryption
|
|||
----------
|
||||
|
||||
bitcoinj allows private keys to be encrypted under an AES key derived from a password using scrypt. We need to keep
|
||||
this function working for deterministic wallets. Also, to achieve the goal of silent/automatic upgrade, we need to
|
||||
this function working for deterministic wallets. Also, to achieve the goal of seamless upgrade, we need to
|
||||
perform the upgrade once the AES key is provided by the user and keep it synchronizable in the previous state until
|
||||
then.
|
||||
|
||||
|
|
@ -173,7 +177,7 @@ before"). Deterministic wallets must also support encryption, which requires enc
|
|||
private keys derived from it.
|
||||
|
||||
Because there are no seeds or private keys, watching/hardware wallets do not support encryption. An attempt to encrypt
|
||||
a wallet that contains both a basic key chain and a key hierarchy without a seed/private keys will throw an exception
|
||||
a wallet that contains both a basic keychain and a key hierarchy without a seed/private keys will throw an exception
|
||||
to avoid the case where a wallet is "half encrypted".
|
||||
|
||||
Serialization
|
||||
|
|
@ -198,27 +202,6 @@ that encode the index of each child as the tree is traversed downwards, with the
|
|||
private derivation (see the BIP32 spec for more information on this).
|
||||
|
||||
|
||||
Upgrade
|
||||
-------
|
||||
|
||||
HD wallets are strictly superior to old random wallets, thus by default all new wallets will be HD. The deterministic
|
||||
key chain will be created on demand by the KeyChainGroup, which allows the default parameters like lookahead size to
|
||||
be configured after construction of the wallet but before the DeterministicKeyChain is constructed.
|
||||
|
||||
For old wallets that contain random keys, attempts to use any methods that rely on an HD chain being present will
|
||||
either automatically upgrade the wallet to HD, or if encrypted, throw an unchecked exception until the API user invokes
|
||||
an upgrade method that takes the users encryption key. The upgrade will select the oldest non-rotating private key,
|
||||
truncate it to 128 bits and use that as the seed for the new HD chain. We truncate and thus lose entropy because
|
||||
128 bits is more than enough, and people like to write down their seeds on paper. 128 bit seeds using the BIP 39
|
||||
mnemonic code specification yields 12 words, which is a convenient size.
|
||||
|
||||
As part of migrating to deterministic wallets, if the wallet is encrypted the wallet author is expected to test
|
||||
after load whether the wallet needs an upgrade, and call the upgrade method explicitly with the password.
|
||||
Note that attempting to create a spend will fail if the wallet is not upgraded, because it will attempt to retrieve a
|
||||
change key which is done deterministically in the new version of the code: thus an non-upgraded wallet is not very
|
||||
useful for more than viewing (unless the API caller explicitly overrides the change address behaviour using the
|
||||
relevant field in SendRequest of course).
|
||||
|
||||
Test plan
|
||||
---------
|
||||
|
||||
|
|
@ -236,5 +219,3 @@ Basic:
|
|||
[ ] Decrypt the wallet. Dump: check it's the same as the first wallet, modulo a few pregenned public keys.
|
||||
[ ] Send another half coin back. Check we can send from a decrypted wallet.
|
||||
[ ] Send another half coin back. Check we can send from the change in a decrypted wallet.
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue