Commit Graph

276 Commits

Author SHA1 Message Date
Chris Stewart
e802254a20
2022 01 24 rm appconfig varargs (#4011)
* Get things compiling

* Get tests passing

* Fix docs
2022-01-25 07:25:05 -06:00
benthecarman
42d6955f79
Filter dust from coin selection (#3866) 2022-01-21 13:15:12 -06:00
rorp
21c97bba12
Filter transactions for onTransactionProcessed (#3990) 2022-01-18 09:52:41 -06:00
Chris Stewart
50bec2abc6
2021 12 14 websockets (#3906)
* WIP

* Get basic websocket working (sort of)

* Push websocket fixes

* WIP: Implementing SpendingInfoDb json serializer

* Implement onreserved callbacks, add json serializers for SpendingInfoDb

* Get first unit test working for websockets when a address is generated

* Rework WalletNotification to have case classes, get unit test working for receiving addresses over the websocket

* WIP

* Add websocket callback for when a wallet processes a block

* Cleanup

* basic updates to unit test

* Fix compile

* Fix submodule

* Fix compile

* Get both unit tests passing when run by themselves

* Fix so both test cases can be run

* Implement unit tests for blockpressed and reservedutxos websockets

* Fix RoutesSpec get a block header test

* Implement configuration for wsbind and wsport via bitcoin-s.conf

* Add some nonblocking sleeps on all WebsocketTests

* Add documentation

* Properly close the server with .terminate() rather than .unbind()

* Add BitcoinSServerMainBitcoindFixture.afterAll()

* Add println

* Add more println

* Try to downgrade bitcoind version

* Fix datadir bug

* Cleanup callbacks so they don't have nested futures

* Fix SpendingInfoDb.apply() pattern match

* Add spendingInfoDb json serializer test, does not pass

* Fix SpendingInfoDb json serializer

* Make small refactors

* Fix compile

* Add maxBufferSize, change overflow strategy to OverflowStrategy.dropHead

* Address Nadav's code review

* Address ben's code review
2021-12-25 11:11:04 -06:00
rorp
41b96c4c7e
Add rescan field to walletinfo response (#3933) 2021-12-23 14:30:26 -08:00
Chris Stewart
4ca586ca46
2021 12 19 dlc callbacks (#3923)
* Implement DLC wallet callbacks

Add callbacks and tests for Claimed/RemoteClaimed

Add unit test for confirmed state

* Add some docs

* Address ben's code review

* Fix scalafmt
2021-12-20 14:28:52 -06:00
Chris Stewart
3cd57d37f5
Remove backupwallet / backup oracle (#3920)
* Remove backupwallet / backup oracle

* Add zipdatadir documentation

* Fix submodule
2021-12-20 08:19:46 -06:00
Chris Stewart
665f931721
Work around for WalletAppConfig.scheduler being blocked by AddressQueueRunnable (#3917)
Add smallest workaround for 3916
2021-12-17 11:58:40 -06:00
Chris Stewart
2d9d12816b
Add WalletCallbacks.onBlockProcessed() (#3912) 2021-12-16 12:26:48 -06:00
Chris Stewart
1969056372
Add better logs for fundTransactionInternal (#3911)
* Add better logs for fundTransactionInternal to understand the state of utxos when they are selected

* Move where log happens for unreserving utxos
2021-12-16 11:28:52 -06:00
Chris Stewart
8307fdc0d1
2021 12 12 wallet doublespend tests (#3898)
* WIP

* Fix sql queries, fix up unit tests that were implicitly using wallet.processOurTransaction()

* Remove default parameter markAsReserved=false on FundTransactionHandling.fundRawTransactionInternal()

* Fix submodule

* Add test to make sure if we fail to reserve utxos, we roll the database transaction back

* Add some clarifying comments

* address ben's code review for failures in finishSend()

* Scalafmt
2021-12-15 07:03:11 -06:00
Chris Stewart
6c67e83d3f
2021 12 10 issue 3891 (#3894)
* Add some logs to indicate how long startup time is

* Parallelize submodule configuration startup between modules dependent on tor and modules not dependent on tor

* Start tor dependent modules in parallel too

* Make Wallet.initialize() use DBIOAction

* Scalafmt

* Make Wallet.initialize() use .transasctionally
2021-12-13 14:20:51 -06:00
Chris Stewart
174c511cd3
2021 12 12 wallet optimization (#3897)
* Add logs for receiving/spending utxos in a block, modify code to use sql query

* Fix bug on join

* Turn off logging
2021-12-12 16:02:31 -06:00
Chris Stewart
d060c1c5fa
Use bigendian txids in wallet log messages (#3893) 2021-12-10 12:03:25 -06:00
Chris Stewart
085b8b1910
Implement CRUDAction.deleteAction() (#3877) 2021-12-06 07:19:31 -06:00
Chris Stewart
2d5732375f
Add CRUDAction.{updateAllAction, updateAction} (#3872)
* Add CRUDAction.{updateAllAction, updateAction}

* Move updateDLCOracleSigs into DLCActionBuilder and make it an action

* Update DLCTransactionProcessing.calculateAndSetOutcome() to use actions
2021-12-04 05:29:51 -06:00
Chris Stewart
f71d3567ed
2021 11 23 crud action (#3851)
* WIP

* Finish implementing {CRUD,CRUDAction}.createAllAction()

* Try to make database inserts more atomic

* Consolidate more database calls in DLCWallet.initDLCForAccept()

* WIP2

* More refactor and using actions

* Add DLCIdDaoUtilNoPK for querying when dlcId is not a primary key

* Add DLCDataManagement.deleteDLC()

* make DLCWalletApi.createDLCOffer() use actions

* Rework DLCWallet.createNewDLCAccept() to use DBIOAction

* Use actions in DLCWallet.registerDLCAccept() a bit

* Create DLCActionBuilder, move actions to there

* Move more actions into ActionBuilder

* Implement transactionally on database writes for actions
2021-11-30 12:38:55 -06:00
Chris Stewart
ba88fb1a03
2021 11 12 fix multiple tag issue (#3822)
* WIP

* Add test case to make sure we can tag an address multiple times

* Actually assert the tagnames/addresses
2021-11-12 15:57:39 -06:00
Chris Stewart
b3d61bc793
Add deleting address tags to to clearAllUtxosAndAddresses() (#3817)
* Add deleting address tags to to clearAllUtxosAndAddresses()

* assert address tag return is empty
2021-11-12 08:53:55 -06:00
Chris Stewart
31e8324522
Make sure exception is caught by Future inside of UtxoHandling.unmarkUTXOsAsReserved() (#3816) 2021-11-11 15:22:45 -06:00
rorp
e02c9bba12
Overridable Tor config (#3780) 2021-10-27 12:57:10 -05:00
benthecarman
92ab9faa45
Decide which coin selection solution to use based on waste metric (#3646)
* Decide which coin selection solution to use based on waste metric

* Fix test

* Fix NeutrinoNodeWithWalletTest
2021-10-19 09:47:28 -05:00
rorp
5a5f1e00c7
Store oracle name in the database (#3748)
* Store oracle name in the database

* fix pg tests

* respond to the PR comments
2021-10-14 09:35:11 -05:00
Chris Stewart
9e43a242fa
Fix logical error on getunconfirmed balance for an account (#3721)
* Fix logical error on getunconfirmed balance for an account

* Actually call getunconfirmedbalance

* remove println

* Run scalafmt
2021-10-05 06:23:30 -05:00
rorp
698fe9e800
Add backupwallet and backuporacle endpoints (#3694)
* Add backupwallet and backuporacle endpoints

* respond to the PR comments
2021-09-28 07:46:33 -05:00
Chris Stewart
48213d9b81
Implement MasterXPubDAO (#3451)
* Implement MasterXPubDAO

* Fix nits

* Add postgres migrations

* Make sure tables are getting cleaned on postgres

* Get dlcOracleTest test cases passing with validation of the master xpub in the database on startup

* Fix test case

* Fix more tests

* Move things around so keymanager does not need to depend on dbCommons

* Get all walletTest passing

* Add unit test for wallet startup

* Fix rebase

* Get all sqlite tests passing

* Get postgres test cases passing locally

* Remove early call to walletAppConfig.start() in node project

* Make KeyManagerAppConfig.externalEntropy private

* Fix MasterXpubDAO.existsOneXpub()

* Fix bug in migrating database for oralce project
2021-09-21 08:05:23 -05:00
Chris Stewart
132479d271
Implement ability to provide external entropy to bitcoin-s (#3672)
* Encapsulate initialization of DLCOracle.start() method

* Use internal WalletAppConfig.kmConf rather than passing in custom key manager parameters

* Add KeyManagerAppConfig.defaultAccountType

* Get all tests passing besides TrezorAddressTest

* Get TrezorAddressTest passing with provided entropy

* Add unit test to make sure we can always derive the seed

* Get docs compiling

* Fix dlcWalletTest test cases

* Add more test cases to keymanager

* Add the new configuration to the example configuration

* Add more test cases

* Remove coverage on 2.12 as it isn't accurate

* Rework DLCOracleAppConfig.start() to call kmConf.start() so the oracle can use entropy provided via bitcoin-s.conf
2021-09-18 09:49:11 -05:00
Chris Stewart
bedc152f33
Write unit test for SpendingInfoDAO.upsert() (#3620)
* Write unit test for SpendingInfoDAO.upsert(), match on outpoint rather than id

* Add more explicit error message for failing to clear utxos and addresses
2021-08-29 08:00:05 -05:00
Chris Stewart
f85f969500
Revert 60 second delay on broadcast to go back to 4 hour default to not break IBD (#3557) 2021-08-16 17:45:46 -05:00
rorp
3d4cf1fc91
Tor support for all sub-projects (#3506) 2021-08-10 08:07:12 -05:00
Chris Stewart
9666e9d296
2021 08 09 init delay broadcast freq (#3509)
* Adjust the init broadcast delay to be 60 seconds

* Fix import
2021-08-10 06:11:56 -05:00
Chris Stewart
4f7b6422ea
2021 08 06 app config refacotr (#3498)
* Move AppConfig into app-commons, reduce classpath deps on keyManager,tor

* Move test ServerArgParserTest

* Fix compile error
2021-08-07 14:36:11 -05:00
Chris Stewart
bc79a24f53
Get both bundle and app server logging working (#3362)
* Get both bundle and app server logging working, oracleServer does not compile

* Get everythig compiling

* Fix bug where bundle wouldn't use command line param rpc port for ConsoleCli

* Add unit tests to make sure we are translating to config properly

* Fix bug where app config wasn't re-created after parsing bundle args

* Fix datadir resolution in BundleGUI

* Implement force-recalc-chainwork

* fix bug

* Try not using tilde on windows

* Try to print datadir on CI

* Try using stackoverflow answer

* Try escaping windows path

* Take ben's suggestion of using AppConfig.safePathToString

* Just check the paths are equal

* Fix nullpointerexceptions on startup

* Fix bug where bitcoin-s-bundle.conf was not being read

* Remove usage of --conf flag in bundle when starting backend server

* Fix compile

* Rename usedDir -> networkDir

* Fix passing in datadir
2021-07-07 10:59:11 -05:00
benthecarman
629c2a2c31
Set sync height on new wallet (#3368) 2021-07-06 12:33:38 -05:00
benthecarman
0d2bc7a927
Remove unneeded asInstanceOf calls in DLCWallet (#3345) 2021-06-28 22:34:42 -05:00
benthecarman
fdba5ad6be
Silence warning log if no error (#3314) 2021-06-22 12:03:36 -05:00
benthecarman
bd11c84462
Add ability to sweep wallet (#3274)
* Add ability to sweep wallet

* Respond to review

* empty
2021-06-16 14:37:40 -05:00
Chris Stewart
5caf7ee38b
2021 06 10 cache spendinginfodbs for block (#3245)
* Add mark as spent optimization to batch writes rather than write inside the method

* Scaladoc refactor

* Refactor received spendingInfoDbs to be cached outside of processTransactionImpl()

* Implement processBlockCachedUtxos() to improve performance

* Fix typo
2021-06-11 15:14:02 -05:00
Chris Stewart
af8aaa7bad
2021 06 07 markasspent optimization (#3244)
* Add mark as spent optimization to batch writes rather than write inside the method

* Scaladoc refactor
2021-06-11 13:26:24 -05:00
Chris Stewart
8574edede7
Move addressDb database read out of inner loop (#3239)
* Move addressDb database read out of inner loop and into processReceivedUtxo() to hopefully improve performance

* Cleanup
2021-06-09 12:18:57 -05:00
benthecarman
be8e965367
DLC Wallet pulldown (#3138)
* DLC Wallet pulldown

* Respond to review
2021-05-25 17:31:01 -05:00
Chris Stewart
cea8802c05
parallelize matching compact filters as this is a bottleneck during IBD (#3137)
* parallelize matching compact filters as this is a bottleneck during IBD

* Use FutureUtil.makeAsync() to actually make it async

* Small optimization, move asmVec outside of lambda so it doesn't need to be re-computed everytime
2021-05-25 12:39:39 -05:00
Chris Stewart
68c7bc1040
2021 05 21 fetch height parallel (#3124)
* Fetch block height in parallel for Wallet.processCompactFilters()

* Make implementation safe against Vector.empty being passed as a parameter
2021-05-23 18:30:19 -05:00
Chris Stewart
9c9e27a8f5
Add optimzations for IBD when the wallet is empty, basically skip all logic for matching filters since there is nothing to match against (#3121) 2021-05-21 15:08:25 -05:00
Chris Stewart
7468cbec23
Optimize updateUtxoConfirmedStates() to fetch confirmations for blocks in parallel (#3094) 2021-05-16 17:10:58 -05:00
Chris Stewart
72636b7180
2021 05 09 received utxos (#3063)
* Make TransactionProcessing.processTransactionImpl() a bit more parallel, fetch all relevant received and spent txos in parallel

* More refactors, make sure return types are Vector rather than Seq to make sure we don't have any pathlogical surprises with scala collections

* Fix type annoation to be Vector

* Fix remaining usage of Seq in TransactionProcessing

* Fix pattern match on Vector on 2.12.x

* Change input to Vector rather than Seq in processSpentUtxos()
2021-05-12 10:45:23 -05:00
Nadav Kohen
78f4dfb8c6
Pubkey Refactor (#2936)
* Removed ExecutionContext from ECKey

* Refactored ECPublicKey to remove compression state and introduced ECPublicKeyBytes to handle cases where serialization of input is important

* Fixed the rest of bitcoin-s so that it passes all tests

* Made all ECKeys into case classes

* Successfully added isFullyValid invariant to ECPublicKey!

* Fixed docs

* Added scaladocs and fixed a RpcPsbtResult bug

* Reject private keys of length < 32 and fix WIF parsing bug
2021-05-06 13:19:52 -05:00
benthecarman
682a759977
Fix wallet v11 postgres migration (#3009) 2021-05-02 10:27:07 -05:00
Chris Stewart
a2911f31ed
Fix race condition with BitcoindChainHandlerViaZmqTest (#2990)
Turn off logging again

Try to move things around even more to make sure zmq is started

Turn logging level down to ERROR to hopefully get some meaningful output

Adjust both valdiation & ChainHandler to be ERROR level logging

switch zmqpubrawblock to be the port we know is free

Reduce logging to WARN

Bump bitcoind max retries to 120

Fix rebase

Revert logging

Remove some noisy logs

Restore logs to trace level

Add explicit tag
2021-05-01 13:11:13 -05:00
Chris Stewart
73939a15fc Call WalletAppConfig.stop() when destroying wallet in test fixtures (#2975)
* Fix bug introduced in PR 2597, we need to explicitly call WalletAppConfig.stop() now rather than transitively calling it via Wallet.stop()

* Fix calls to wallet.stop() -> wallet.walletAppConfig.stop() in BitcoindBlockPolling,BitcoindBackend. Also add some explicit type annotations
2021-04-26 15:48:33 -05:00