Commit graph

161 commits

Author SHA1 Message Date
Ben Carman
ea20f394e8 Add wallet pay to many (#1317)
* Add wallet pay to many

* Create sendToOutputs

* Add sendToAddresses Unit Test

* Send to addresses can reserve utxos
2020-04-09 15:00:26 -05:00
Ben Carman
b3250dcfba Update TxoState for transactions after they've been confirmed (#1178)
* Add onBlockHeadersReceived NodeCallback

* Add requiredConfirmations config option

* Update TxoState for transactions after they've been confirmed

* Response to review

* Fix compile issue
2020-04-08 16:51:17 -05:00
Ben Carman
29eb6c2e05 Wallet Transaction Tracking (#1197)
* Incoming and Outgoing Transaction Tables

* Remove script sigs for witness txs

* Create parent tx_table for incoming and outgoing txs

* Response to review

* Use isCloseEnough

* Fix test

* Fix rebase error

* Test that tx is tracking all sats correctly
2020-04-02 06:55:09 -05:00
Ben Carman
ff8df7065c Wallet integration for nested segwit v0 spending (#1272)
* Wallet integration for nested-segwit spending

* Move parameter
2020-03-28 09:40:05 -05:00
Chris Stewart
cd84e3483d 2020 03 06 wallet rescan test (#1218)
* Add wallet rescan tests

* Create FilterSync, which gives us an API inside of the chain project to sync filters with

Add another unit test to filter sync

Add more unit tests for ChainSync and FilterSync

Clean up some docs, remove some extra lines of code

Run scalafmt

Add filter-sync.md

Cleanup some nits

Add more information of how FilterSync.syncFilters() works

Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project

Run scalafmt, hide imports in filter-sync.md so code appears cleaner

Move implicits out of invisible block as it seems to cause errors

Make it so FilterSync processes filters in batches rather than fetching them all at once

Fix compile error

* WIP bitcoind implement ChainQueryApi

* rework fixtures to be able to support injecting ChainQueryApi implemented by bitcoind into our fixture infrastructure for creating wallets

* Fix rebase problem

* Implement getFiltersBetweenHeight() with ben's solution

* WIP Start implementing NodeApi against bitcoind

* Actually inject the bitcoind backed nodeApi into our fixture

* Get first rescan test working for rescanning the entire blockchain

* Implement test case for rescanning from a specific height

* Fix NeutrinoNodeWalletTest test case that uses a experimental version of bitcoind, for now i believe our experimental binary is on v18

* Add wallet-rescan.md, add helper method to WalletApi that allows you to clear out the wallet's utxos/addresses

* Add another log to try to debug CI

* Address code review

* Fix wrong ordering of deletion of tables in clearUtxosAndAddresses()

* reset logging level
2020-03-13 16:11:02 -05:00
Ben Carman
bc87bc64eb TxoState Life Cycle Tests 2020-02-28 17:18:28 -06:00
Ben Carman
cf3dd113b5 Reserved TxoState (#1111)
* Reserved TxoState

* Allow fund transaction handling to mark utxos as Reserved

* Add test, fix overloading default arg issue
2020-02-13 06:51:50 -06:00
Chris Stewart
f8a21201ac Run test:scalafmt, and enforce it on CI (#1023)
* Run test:scalafmt, and enforce it on CI

* Run test:scalafmt
2020-01-23 14:46:57 -06:00
Chris Stewart
b8c59b4c93 2020 1 11 accounts (#1022)
* Create 'fundRawTransaction' and 'fundRawTransactionInternal'

* Move TransactionTestUtil to testkit, begin writing tests for fundrawtransaction

* Add FundTransactionTests

* Move things to 'FundTransactionHandling' trait

* Start segregating things by HDAccount in the wallet, this gives us the ability to query for balances based on account, fund a transaction based on an account, generate a new address based on an account etc. All old api calls are now based on the default account, i.e. getBalance() returns the balance for account 0, fundRawTransaction funds the transaction from account 0 etc.

* Fix compile issue with 2.11.x

* Bump address generation Thread.sleep() to 500ms

* Address ben code review

* Address code review
2020-01-21 14:20:58 -06:00
Chris Stewart
54b2fa814e 2020 1 04 fund raw tx (#1010)
* Create 'fundRawTransaction' and 'fundRawTransactionInternal'

* Move TransactionTestUtil to testkit, begin writing tests for fundrawtransaction

* Add FundTransactionTests

* Move things to 'FundTransactionHandling' trait

* Address code review

* Run scalafmt
2020-01-13 13:45:33 -06:00
Chris Stewart
3bedcb20e0 Add more expressive txo states for the wallet (#1001)
* Add more expressive txo states for the wallet

* Rename 'PendingReceived/PendingSpent' -> 'UnconfirmedReceived/UnconfirmedSent'

* Address code review

* Fix posInt implementation to work with older scala versions

* Rename to PendingConfirmationReceived/PendingConfirmationSpent
2020-01-05 18:42:20 -06:00
Chris Stewart
55f168f920 Implement abililty to use BIP39 password. This means this password ne… (#990)
* Implement abililty to use BIP39 password. This means this password needs to be password through our various projects to be able to correctly generate the key that controls the wallet. This also renames 'CreateKeyManagerApi' -> 'BIP39CreateKeymanagerApi' as the bip39 password is needed when creating the KeyManager

* Add bip39 password to BIP39KeyManager.fromParams(), fix mdocs

* Fix bug in unit test were whe weren't specifying password

* Fix rebase issues
2020-01-03 11:03:45 -06:00
rorp
93c02f02d0 Automated wallet recovery (#985)
* Automated wallet recovery

* responded to the PR comments

* some more changes

* fix docs

* cleanup
2020-01-02 13:18:41 -06:00
Chris Stewart
c3fb7c9a67 This creates a subtype BIP39KeyManager and moves all existing KeyMana… (#988)
* This creates a subtype BIP39KeyManager and moves all existing KeyManager functionality to that subtype. We leave a empty 'KeyManager' trait for now

* Fix website docs
2019-12-27 10:09:04 -06:00
Chris Stewart
24ff908240 Move initialization of wallet entropy into the key manager (#966)
* Move initialization of wallet entropy into the key manager

* Enable key-manager in ci, add coverage minimum

* Add coverage minimum for key manager, add KeyManagerParams, start adding KeyManager test harness

* Remove success cases on ADTs, start using Either

* fix import

* Get test coverage to 90%

* Add documentation for key-manager.md, add side bar for key-manager

* Fix wallet.md

* Make seedPath default to baseDatadir, so we don't have a unique seed for every network. Add more documentation

* Address code review

* Address code review from Ben

* Add missing import

* Add more tests for coverage
2019-12-26 19:12:08 -06:00
rorp
72ead16e5c Move rescan logic from node to wallet (#974)
* Move rescan logic from node to wallet
2019-12-23 08:57:55 -08:00
rorp
2c53a39fd1 Compute confirmations dynamically (#938)
* Compute confirmations dynamically
2019-12-14 12:06:22 -08:00
rorp
4acd1fbba9 Add a reference to ChainQueryApi to the wallet (#936)
* Add reference to ChainQueryApi to wallet
2019-12-10 14:22:33 -08:00
rorp
af70acb03b ChainQueryApi (#926)
* ChainQueryApi

* cleanup
2019-12-09 09:39:35 -06:00
rorp
cbd038b514 Neutrino Wallet: OnCompactFilter handler (#905)
* Neutrino Wallet: OnCompactFilter handler
2019-12-06 13:10:33 -08:00
Chris Stewart
dd4787f2af 2019 11 30 key manager (#904)
* Segregate key manager from wallet

* More consistent naming in build.sbt, run scalafmt

* Add test case making sure we don't overwrite a mnemonic seed

* Add key-manager.md

* Fix compile issues with older scala versions

* Address code review
2019-12-04 07:43:29 -06:00
rorp
74a44d1c4d Initial BIP157 support (#695)
* WIP: Initial BIP157 support

* store block hash and hetgh along with its compact filter header

* download and parse block filters

* getcfilters/cfilter

* cfilter table

* rescan full filter chain

* improved rescan performance

* optimize compact headers download

* addressed the PR comments

* split SVP and Neutrino node implementations

* configurable filter batch sizes

* initial filter sync

* addressed comments

* chage filter table's primary key

* fix Golomb filter deserialization

* batch database inserts

* neutrino node test

* fixed node test

* addressed the PR comments

* serializers tests

* cleanup

* fix compilation errors

* fix unit tests

* increase test coverage

* enable NeutrinoNodeTest

* make scalafmt happy

* don't cache experimental binaries

* inclease test coverage

* fix unit tests

* more granular CI tests

* disable NeutrinoNodeTest

* refactor tests

* addressed comments

* test coveage

* fix formatting

* responded to the comments

* some more changes

* fix the build

* test coverage

* revert testnet3 config parameter

* minor changes

* cleanup
2019-09-25 13:18:51 -05:00
Chris Stewart
67ead0bc7e 2019 09 05 scalafmt update (#735)
* Update scalafmt to 2.0.1

* Scalafmt

In this commit we
1) Reconfigure the .scalafmt.conf file
2) Add a scalafmt binary (same version as specified in conf file)
3) Use said binary on CI to check that everything is formatted correctly

* Ran scalafmt

* Update scalafmt ignore

* Add scalafmt plugin to project/plugins.sbt, run it

* run scalafmt
2019-09-05 20:02:58 -05:00
Torkel Rogstad
e0a5646258 Deprecate generate (#728) 2019-09-01 07:46:38 -05:00
Chris Stewart
c934d8efc2 Address issue #673, also refactor fixture stuff to be in companion ob… (#676)
* Address issue #673, also refactor fixture stuff to be in companion objects rather than traits so they can be re-used without extending the trait itself. This allows for more modularity with our fixtures

* fix bugs in unit tests, address #675 to make MerkleBufferTests faster

* Refactor test timeout in UpdateBloomFitlerTest

* fix callback logic bug on matching unrelated txs/addresses in UpdateBloomFilterTest, re-order broadcasting of things to avoid async bugs hopefully

* Make  a def in NodeUnitTest, this keeps tests in the same suite from using the same config. This allows us to write multiple tests per suite, instead of just one. This also adds implicit parameters to our fixture constructors/destructors to properly create and destroy this config. The long term goal here also should be getting rid of config.initialize() we are calling everywhere as this is a anti-pattern, fixtures should take care of construction and destruction of things

* Broadcast tx earlier in test case for UpdateBloomFilterTest

* Rework NodeWithWalletTest to use the new fixtures we have in the node project, now use SpvNodeFundedWalletBitcoind
2019-08-06 11:49:17 -05:00
Torkel Rogstad
46280c9e59 Add functionality for updating SPV node bloom filter (#585)
* Add functionality for updating SPV node bloom filter

* Add SPV node shutdown to exit hook

* Clean up traits/case classes

* Change fixture in WalletBloomTest

* Fix logging bug in TransactionProcessing

* Add MerkleBuffers

In this commit we add MerkleBuffers, which is an object
that lets us aggreagate merkleblocks with their corresponding
transactions before sending them out. This is global, mutable
state (bad!) but it's a working solution for now;

* Use TestAsyncUtil

* Add MerkleBuffers test

* Send getdata if receiving single header

* Change awaitSync to use block count

* Fix UpdateBloomFilterTest

* Add more logging of chain/headers validation

* Send getdata for all blocks

* Nits: Scaladocs, comments toString
2019-08-02 09:22:20 -05:00
Torkel Rogstad
a76f61f97c Add configurable logging to data directory (#640)
* Add logging to data directory

In this commit we add the ability for the node, chain
and wallet projects (+ the server) to log to the users
data directory instead of whatever directory the
binaries was launched from. This is inherently a bit
more complicated than our previous setup, because
we need to read the user directory before we can create
loggers. As a result of this, some files/methods were
moved around, so the relevant app config could be
found in scope.

We also  introduce several logging categories that can be
tuned individually through user configuration. These logggers
are exposed both as traits that give a field `logger`, or
as methods that return the required logger.

* Add datadir configuration to AppConfig

In this commit we add support for AppConfig to pick up
the data directory configuration file. We also add
a section to the contributing guide on how to tune
logging levels.

* Pass data directories explicitly for configuration
2019-08-01 06:01:56 -05:00
Torkel Rogstad
5194fb1f8b Merge pull request #594 from torkelrogstad/2019-07-10-bloom
Add all wallet outpoints to bloom filter
2019-07-16 17:07:21 +02:00
Torkel Rogstad
45a3e93a3d Add all wallet outpoints to bloom filter
This is a workaround for Bloom filters never being
updated for SegWit. See comment in
LockedWallet#getBloomFilter for more context.
2019-07-16 14:18:29 +02:00
Torkel Rogstad
4bdc7966d0 Add notion of address types in wallet
In this commit we
1) Add the notion of address types in the wallet, and remove
    the emphasis on account types. Users now just request
    a segwit/nested-segwit/legacy address, and we take care
    of querying for the correct account
2) Fix a bug where a wallet could only get addresses for
    its default address type. This was a pretty minor bug,
    where a few values that should have been dynamic was
    hard coded.
2019-07-16 12:36:38 +02:00
Torkel Rogstad
30e6d7030f Somewhat dirty standalone server and CLI binary (#558)
* PoC bitcoin-s-cli

* Add CLI, Server sbt projects, remove Ammonite

In this commit we set up sbt configuration for
CLI, Server (in-work-name) and corresponding
test projects.

We also remove Ammonite shell from sbt, as that
isn't really being used. bloop console offers
the same functionality way more ergonimic.

* Move BitcoinSAppConfig into new server project

Server project depends on node, chain wand wallet
so this is a good time for introducing this class
into main sources. We also introduce
BitcoinSTestAppConfig in testkit, to replace the
functionality in BitcoinSAppConfig related to
tests.

* Type chain in blockchainresult

* MVP server setup for node, chain and wallet

* Extremely dirty CLI for interacting with server

* initial attempt at mimicking Bitcoin Core API

* WalletStorage: add method for checking for seed existance

* Check for seed existance on wallet startup

* Fix bug where MnemonicNotFound was not an error

* Segregate confirmed and unconfirmed balance methods

* Add error handling, improve formatting of CLI output

* Tweak build

Bump Sttp version, downgrade to uPickle 2.11 compat,
skip publish in cli-test and server-test

* Add CLI, server and picklers to root project
2019-07-10 06:33:17 -05:00
Torkel Rogstad
a71aecec52 Process outgoing transactions (#555)
* Split wallet functionality into multiple traits

In this commit we refactor LockedWallet into multiple traits
that provide functionality related to a subset of total wallet
functionality. This has the benefit of making it clear which
methods are helper methods that are only intended to be used
in a very specific setting, and which methods are part of the
internal wallet API that other parts of the wallet can use.

* Rework TransactionOutput and TransactionOutPoint to case classes

* Add extension methods for flattening lists of assertions

* Segregate confirmed and unconfirmed balance methods

* Add test for FutureUtil.sequentially

* Add trace logging of balance fetching

* Process outgoing TXOs

Move TX processing into separate trait, add internal API method

Unify DB representation of TXOs

    Prior to this commit we stored TXO information
    across diferent tables, with joins and tuples
    needed a bunch of places to keep track of
    everything we needed. In this commit we unify
    the tables, leaving us with only one table for
    TXOs.
2019-07-09 06:25:24 -05:00
Torkel Rogstad
6caaf1938a Tune down excessive logging 2019-07-02 11:22:18 +02:00
Torkel Rogstad
c2349c44c2 Rebase cleanup 2019-06-27 15:34:41 +02:00
Torkel Rogstad
6ffd505946 Add sampleSome method to scalacheck.Gen 2019-06-27 15:29:21 +02:00
Torkel Rogstad
c884b9c048 Fix bad comparison in ProcessTransactionTest 2019-06-27 15:28:08 +02:00
Torkel Rogstad
86517c0a6d Replace incoming/outgoing TXs with TXOs
Rework the wallet DB model so that instead
of dealing with incoming and outgoing transactions
we deal with incoming and outgoing transaction
outputs.
2019-06-27 15:28:08 +02:00
Torkel Rogstad
06b8375c81 Add ProcessTransactionTest 2019-06-27 15:28:08 +02:00
Torkel Rogstad
5a04260461 Update wallet integration tests 2019-06-27 15:28:08 +02:00
Torkel Rogstad
5513344373 Add findTx to IncomingTxDAO 2019-06-27 15:28:08 +02:00
Torkel Rogstad
e2278f7606 Get UTXO DB unit tests passing 2019-06-27 15:28:08 +02:00
Torkel Rogstad
88e318d485 Move WalletDAOs to testkit 2019-06-27 15:28:08 +02:00
Torkel Rogstad
b630cfbf5c Generalize DAO fixture for wallet 2019-06-27 15:28:08 +02:00
Torkel Rogstad
bdd0468383 Add tables for incoming and outgoing transactions 2019-06-27 15:28:08 +02:00
Torkel Rogstad
e4e9ec9db9 Add SPK field to AddressTable 2019-06-27 15:23:23 +02:00
Torkel Rogstad
db1987b36a Merge pull request #546 from torkelrogstad/2019-06-21-aes-cfb
Use key based encryption in AesCrypt
2019-06-27 12:16:53 +02:00
Torkel Rogstad
000fe1e2ef Merge pull request #431 from nkohen/coin-selection
Basic Coin Selection
2019-06-26 13:13:27 +02:00
Torkel Rogstad
505d3cbaa1 Fix Scala 2.11 complaints 2019-06-25 11:58:18 +02:00
Torkel Rogstad
334d410220 Fix CoinSelectorTest after rebase 2019-06-24 17:45:31 +02:00
nkohen
269d3c8f21 Implemented simple greedy coin selection
Added tests for current coin selection algorithm options

Responded to code review

Implemented fee estimation calculation in coin selection

Updated tests after rebase

Fixed CoinSelectorTest

Fixed CoinSelectorTest after rebase
2019-06-24 17:41:43 +02:00