Commit Graph

351 Commits

Author SHA1 Message Date
rorp
cf22816003
Fix rescan batch boundary bug (#4549)
* Fix rescan batch boundary bug

* Make arguments for rescan required to avoid bugs

* WIP: Debug rescan batches

* Add forceGenerateSpks to fix one bug

* Remove addressCount != 0 check now that we have forceGenerateSpks

* Remove comment

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-08-06 14:07:35 -05:00
rorp
9f89ba9b7a
loadwallet endpoint (#4417)
* `loadwallet` endpoint

* Scala 2.12 compatibility

* remove `getwalletname`

* fix unit tests

* fix DbManagementTest

* fix wallet tests

* fix DbManagementTest

* fix RejectedExecutionException

* fix postgres tests

* fix password override

* update `loadwallet`

* Move WalletHolder into wallet module

* Kill use of null in WalletHolder

* Kill use of null in WalletHolder

* Refactor to use WalletAppConfig.DEFAULT_WALLET_NAME

* update cli and unit tests

* cleanup

* Refactors to get compatible with master

* Fix compile

* Distinguish between loadWalletNeutrino & loadWalletBitcoind

* Fix initialized bug with WalletHolder in loadWallet methods

* Refactor loadWalletBitcoindBackend/loadNeutrinoWallet to have a common helper method

* Introduce DLCWalletLoaderApi, refactor codebase to use it

* Fix dumb initialization bug

* Fix test:compile

* Cleanup

* Get loadwallet mostly working with neutrino wallet

* Fix compile

* scalafmt

* Move loadwallet into app-commons

* Remove {DLCWallet,Wallet}.stop()

* Implement LoadWallet json parsing unit test

* Implement stopping of NodeStreamManager callbacks

* Fix rebase, integrate setRescanState into WalletRoutes

* Fix import

* Fix unit tests

* Implement listwallets in ConsoleCli

* Fix imports

* Cleanup

* scalafmt

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-08-05 17:34:14 -05:00
benthecarman
2fa7c39f64
Use DBIOActions to speed up processing transactions (#4572) 2022-08-04 14:40:31 -05:00
Chris Stewart
c03b158f94
Implement RescanTerminatedEarly exception to terminate the stream, implement ArgumentSource (#4574)
* Implement RescanTerminatedEarly exception to terminate the stream, implement ArgumentSource to understand where arguments passed into bitcoin-s are coming from

* Bump timeout
2022-08-04 14:11:07 -05:00
Chris Stewart
a02e25b0ce
Refactor WalletRoutes to take DLCWalletLoaderApi as a paramete (#4565)
* Refactor WalletRoutes to take DLCWalletLoaderApi as a parameter, use the walletHolder inside of DLCWalletLoaderApi to fulfill requests

Fix rebase

Reduce parallelism in fetchFiltersFlow to 1 to reduce chance of hitting akka's max open requests

* Try fix loader test by catching RejectedExecutionException

* Add println to see if catching excpetion gets hit

* WIP: Try to fix recursive rescan

* Don't map on recursiveRescan so we don't block on it
2022-08-04 09:16:46 -05:00
benthecarman
524c0af536
Make wallet.getBalance more effienct (#4566) 2022-08-04 09:01:29 -05:00
Chris Stewart
8cb45e8208
Add unit tests for loader rescans (#4570)
* Add unit tests for loader rescans

* Move common methods into super class to be DRY

* Cleanup
2022-08-03 13:16:52 -05:00
Chris Stewart
c89491c3e7
2022 07 29 fix multiple batch test (#4556)
* Fix multiple batch unit test so it fails

* Fix compile

* Move generateScriptPubKeys into buildFilterMatchFlow()

* Rename param to filterBatchSize

* Rename to buildRescanFlow

* Remove println

* Cleanup
2022-08-03 12:33:37 -05:00
Chris Stewart
b1a7f92c67
Refactor to not use filterThenSum() when calculating balances (#4562)
* Refactor to not use filterThenSum() when calculating balances

* Add await for balance as blocks are processed async

* Add more assertions, fix bad assertions I introduced
2022-08-02 12:33:46 -05:00
Chris Stewart
191df09196
Move pollBitcoind out of startBitcoindBlockPolling (#4559)
* Move pollBitcoind out of startBitcoindBlockPolling

* Rework startBitcoindBlockPolling to not return Future[Cancellable]

* Stop NodeCallbacksStreamManager inside of NodeAppConfig.stop()

* Make BitcoindSyncState which encapsulates syncing with bitcoind and polling of mempool

* Fix bug where processingBitcoindBlocks cannot be top level val as that method may be used by different parts of the codebase

* Rename isPolling -> processingBitcoindBlocks

* Cleanup more resources in appServerTest/test

* Complete polling stream regardless if we synced blocks are not to complete the Future returned by pollBitcoind

* Empty commit to run CI
2022-08-02 06:40:17 -05:00
benthecarman
603b7e0aea
Make DLCWallet.listDLCs use DBIOActions (#4555)
* Make DLCWallet.listDLCs use DBIOActions

* Add comment

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-07-31 14:09:23 -05:00
Chris Stewart
936a65edd4
2022 07 21 Modify helper methods in BitcoindRpcBackendUtil to not materialize streams eagerly. (#4526)
* Refactor syncWalletToBitcoind impl to be completely stream based

* Add getParallelism, refactor usages in BitcoindRpcBackendUtil to use it

* Fix compile

* Replace usages of 'WalletApi with NeutrinoWalletApi' with NeutrinoHDWalletApi

* Revert wallet sync height logic

* Refactor to make things cleaner
2022-07-31 12:16:26 -05:00
Chris Stewart
af9e5eaeef
Expose DBIOActions for various models in wallet, remove address queue (#4524)
* Expose DBIOActions for various models in wallet, use them in generateScriptPubKeys

* Remove addresses queue

* Remove test case

* Fix rebase
2022-07-31 10:36:45 -05:00
Chris Stewart
4b83286922
2022 07 25 wallet api refactor (#4545)
* Move MockChainQueryApi/NodeApi out of BaseWalletTest

* refactor BitcoinSServerMain.start() to return WalletHolder

* Add walletConfig to WalletWithBitcoind

* Move findOutputsBeingSpent into WalletApi

* Add WalletApi.clearAllAddresses(), HDWalletApi.{findAccount, getnewaddress(account)}

* Add HDWalletApi.fundRawTransaction() with an account as parameter

* Add WalletApi.findByScriptPubKey()

* Fix lots of tests in WalletIntegrationTest

* Create WalletApi.processOurTransaction()

* Get things compiling

* Fix tag integration test

* Refactor AnyDLCHDWalletApi -> DLCNeutrinoHDWalletApi

* Fix docs

* Get postgres tests passing locally

* Move initBalance map before calling callback in CallBackUtilTest

* Get compile working, rename to destroyOnlyWalletWithBitcoindCached

* Fix docs

* Fix missing destroyWalletAppConfig

* Fix scalafmt

* Fix bug in dlcWalletTest where wallet db thread pools weren't being shutdown after unit test completes

* Empty commit
2022-07-31 08:04:39 -05:00
Chris Stewart
17ef1dcdd9
Fix rescan flag getting set, add unit test (#4554) 2022-07-29 15:17:48 -05:00
Chris Stewart
f608c5d5e4
Create a helper case class called FundRawTxHelper (#4544)
* Create a helper case class called FundRawTxHelper to encapsulate the values returned by fundRawTransactionInternal

* Return FundRawTxHelper[ShufflingNonInteractiveFinalizer]

* make FundRawTxHelper.signedTx be a lazy val
2022-07-27 17:12:42 -05:00
Chris Stewart
6c76639ba3
Move processBlock into WalletApi (#4543) 2022-07-27 09:32:22 -05:00
Chris Stewart
44e09270d9
Move getNewAddress(account) to HDWalletApi (#4540) 2022-07-25 10:48:02 -05:00
Chris Stewart
d508d65142
Implement ability to cancel bitcoind polling jobs (#4538)
* Implement ability to cancel bitcoind polling jobs

* Fix bug so we don't block application on fully syncing wallet against bitcoind
2022-07-24 19:56:53 -05:00
Chris Stewart
b905afa65e
2022 07 20 wallet rescan stream (#4530)
* Implement rescan with akka streams

Get basic stream working with rescan

Fix bug where we weren't using rescan specific threadpool

Implement killswitch for rescan

* WIP: Expose promise to allow external completion of rescan stream

* Rework RescanStarted.RescanStarted to contain a promise to complete the stream early, and a future that represents the completed streams materialized value

* Comment cleanup

* Fix compile errors, remove killswitch

* Fix 2.12.x compile

* Introduce ActorSystem into wallet, refactor rescans to use that ActorSystem

* Fix import

* Fix bug where we were prepending instead appending to batched Vector

* Propogate RescanState upwards into WalletRoutes

* Refactor fetching of filters to be a Flow
2022-07-24 12:26:21 -05:00
Chris Stewart
d4210fad65
Pull over changes from loadwallet branch (#4527) 2022-07-21 12:12:32 -05:00
benthecarman
c210052640
Refactor coin selection to be not be bitcoin-s specific (#4496)
* Refactor coin selection to be not be bitcoin-s specific

* Add to CoinSelectorUtxo
2022-07-20 08:40:11 -05:00
rorp
5d309fbae0
Fee rate WS notification (#4518)
* Fee rate WS notification

* make the test more CI friendly

* fix the default config

* always return errors

* fix build

* scalafmt

* send notifications every time
2022-07-19 16:41:06 -05:00
Chris Stewart
5c9092889b
Add abillity to clear callbacks (#4512)
Fix missing clearCallbacks()

Add some tests

Remove uncessary code from test
2022-07-17 09:10:57 -05:00
Chris Stewart
948d2b424e
Make KeyManagerAppConfig.walletName return String rather than Option[String] (#4507) 2022-07-16 09:33:10 -05:00
Chris Stewart
1b169d8fa0
Implement torstarted websocket callback (#4476)
* Implement tor started websocket callback

* Implement tor callbacks when tor is provided
2022-07-11 10:26:30 -05:00
Chris Stewart
1ae4d40dac
Implement rescancomplete callback (#4475)
* Implement rescancomplete callback

* Add walletName to payload

* Add to documentation
2022-07-09 16:23:15 -05:00
Shreyansh
d8fc8e588f
Remove Spv code (#4356)
* change node tests to neutrino

* get node test working

* merge DataMessageHandlerTest and DataMessageHandlerNeutrinoNodeTest

* delete unused files

* remove commented out spv parts

* formatting

* delete spv node

* remove merkle callback for neutrino node

* remove spv node wallet callbacks

* formatting

* remove SpvWalletApi

* replace SpvTestConfig with NeutrinoTestConfig

* more replace SpvTestConfig with NeutrinoTestConfig

* minor fix

* fix tests
2022-05-30 07:57:31 -05:00
Chris Stewart
bf88d0d93f
Remove exception when we have zero relevant outputs (#4352) 2022-05-26 12:39:09 -05:00
rorp
f680ab8691
Persist whether wallet is rescanning in the database (#4326)
* Persist whether wallet is rescanning in the database

* fix cli

* fix build

* fix unit tests

* fix postgres tests

* remove wallet_state table

* fix rescan bug

* cleanup

* revert Cancellable's

* Cleanup

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-05-23 19:03:02 -05:00
rorp
059f2f5fac
Fix ERROR: relation "txo_spending_info" does not exist (#4323)
* Fix `ERROR: relation "txo_spending_info" does not exist`

* Escape schema name

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-05-17 12:02:29 -05:00
Chris Stewart
341c712563
Validate bitcoin-s.wallet.walletName config option (#4336) 2022-05-17 10:29:17 -05:00
Chris Stewart
fac0713405
Reduce rescan threadpool size to just be number of available processors (#4306) 2022-05-01 07:05:43 -05:00
rorp
0c6c9180ed
Handle duplicate UTXOs (#4296)
* Handle duplicate UTXOs

* Add logging

* Add log for exception in handleDuplicateSpendingInfoDb

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-04-29 08:21:41 -05:00
Chris Stewart
486fa36d2c
Make _findOutputsBeingSpentQuery take at most 1,000 outpoints (#4300) 2022-04-29 07:20:23 -05:00
Chris Stewart
79b4f096ec
Improve logging around signDLC (#4299)
* Improve logging around signDLC

* Improve logging so we can see what specific transaction fails when processing a block
2022-04-28 17:09:27 -05:00
Chris Stewart
6db1f26625
2022 03 22 getrelevantoutputs upfront (#4219)
* WIP

Try and debug

WIP

Fix bug for incomingAmount

Clean up logs

more cleanups on logs

Remove another uncessary log

Don't process tx if we don't have a relevant received output for that specific tx

* Group hashes in batches of 1,000 before querying the db
2022-04-26 11:12:44 -05:00
rorp
3831b35817
Prevent the wallet from creating duplicate UTXOs (#4290)
* Prevent the wallet from creating duplicate UTXOs

* respond to the PR comments

* cleanup
2022-04-25 11:56:37 -05:00
Chris Stewart
27cb4a3c20
Add height to log message when processing block (#4251) 2022-04-10 11:04:17 -05:00
Chris Stewart
7c1c572081
Rescan failure logging (#4250)
* Rescan failure

* Set rescanning to false inFailure case
2022-04-10 10:39:11 -05:00
Chris Stewart
24125eaca3
Fix case where we broadcast tx with a tag, and then also add the tag when we receive the tx in a block (#4239) 2022-04-03 13:40:19 -05:00
Chris Stewart
8081772b57
Allow acceptor to upsert same outpoint multiple times for different offers (#4230)
* Allow acceptor to upsert same outpoint multiple times for different offers

* Remove extra log

* Revert DLCOffer invariant

* Fix OP_RETURN tests
2022-03-31 07:47:10 -05:00
Chris Stewart
33171fd069
Use the set of addresses in the wallet when rescanning rather than generating more (#4211) 2022-03-26 06:22:09 -05:00
Chris Stewart
413dbcacbb
Rename clearUtxosAndAddreses(account) -> clearUtxos(account) (#4206) 2022-03-22 13:08:03 -05:00
Chris Stewart
5475a994cf
Bump default address gap limit to 100, add log indicating we didn't find funds in the last addressBatchSize addresses (#4201) 2022-03-19 19:02:32 -05:00
Chris Stewart
d5807daeab
Remove SpendingInfoDb.txid parameter (#4199) 2022-03-19 15:21:36 -05:00
Chris Stewart
a3dba47970
Refactor pending receives/spent states into one spot (#4026) 2022-03-19 13:25:26 -05:00
Chris Stewart
46229d712c
Rename clearAllUtxosAndAddresses() -> clearAllUtxos() (#4193)
* Rename clearAllUtxosAndAddresses() -> clearAllUtxos()

* Fix unit test

* Fix docs
2022-03-16 08:53:08 -05:00
Chris Stewart
ef41cce32d
Log entire address path when the address is generated (#4191) 2022-03-15 08:04:40 -05:00
Chris Stewart
8214c0b931
Add the account to the address generation log (#4190) 2022-03-14 19:33:40 -05:00