Commit graph

152 commits

Author SHA1 Message Date
rorp
9f89ba9b7a
loadwallet endpoint (#4417)
* `loadwallet` endpoint

* Scala 2.12 compatibility

* remove `getwalletname`

* fix unit tests

* fix DbManagementTest

* fix wallet tests

* fix DbManagementTest

* fix RejectedExecutionException

* fix postgres tests

* fix password override

* update `loadwallet`

* Move WalletHolder into wallet module

* Kill use of null in WalletHolder

* Kill use of null in WalletHolder

* Refactor to use WalletAppConfig.DEFAULT_WALLET_NAME

* update cli and unit tests

* cleanup

* Refactors to get compatible with master

* Fix compile

* Distinguish between loadWalletNeutrino & loadWalletBitcoind

* Fix initialized bug with WalletHolder in loadWallet methods

* Refactor loadWalletBitcoindBackend/loadNeutrinoWallet to have a common helper method

* Introduce DLCWalletLoaderApi, refactor codebase to use it

* Fix dumb initialization bug

* Fix test:compile

* Cleanup

* Get loadwallet mostly working with neutrino wallet

* Fix compile

* scalafmt

* Move loadwallet into app-commons

* Remove {DLCWallet,Wallet}.stop()

* Implement LoadWallet json parsing unit test

* Implement stopping of NodeStreamManager callbacks

* Fix rebase, integrate setRescanState into WalletRoutes

* Fix import

* Fix unit tests

* Implement listwallets in ConsoleCli

* Fix imports

* Cleanup

* scalafmt

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-08-05 17:34:14 -05:00
Chris Stewart
5acbba9377
Replace BoundedSourceQueueWithComplete with SourceQueueWithComplete (#4576)
* Replace BoundedSourceQueueWithComplete with SourceQueueWithComplete so we can use the returned Future

* Recover the expected exceptions in CallBackUtilTest
2022-08-05 11:57:13 -05:00
Chris Stewart
c03b158f94
Implement RescanTerminatedEarly exception to terminate the stream, implement ArgumentSource (#4574)
* Implement RescanTerminatedEarly exception to terminate the stream, implement ArgumentSource to understand where arguments passed into bitcoin-s are coming from

* Bump timeout
2022-08-04 14:11:07 -05:00
Chris Stewart
a02e25b0ce
Refactor WalletRoutes to take DLCWalletLoaderApi as a paramete (#4565)
* Refactor WalletRoutes to take DLCWalletLoaderApi as a parameter, use the walletHolder inside of DLCWalletLoaderApi to fulfill requests

Fix rebase

Reduce parallelism in fetchFiltersFlow to 1 to reduce chance of hitting akka's max open requests

* Try fix loader test by catching RejectedExecutionException

* Add println to see if catching excpetion gets hit

* WIP: Try to fix recursive rescan

* Don't map on recursiveRescan so we don't block on it
2022-08-04 09:16:46 -05:00
Chris Stewart
8cb45e8208
Add unit tests for loader rescans (#4570)
* Add unit tests for loader rescans

* Move common methods into super class to be DRY

* Cleanup
2022-08-03 13:16:52 -05:00
GreyMcCarthy
b7c7cb8fc2
Moving duplicate dataStructures in cli and app-server into app-commons (#4541)
* duplicate data structures in (ConsoleCli.scala) and (ServerJsonModels.scala) moved to (Command.scala)

* removed unused imports

* rebase for merge conflitcts

* change output type for Accept DLC

* more error fixes

* error fixes

* fixed destintaion error

* scala reformatting

* send from outpoints test runs and passes

* removed unnecessary braces, removed incorrect extensions on objects

* scala reformatting

* removed commented out code
2022-08-02 11:55:43 -05:00
Chris Stewart
191df09196
Move pollBitcoind out of startBitcoindBlockPolling (#4559)
* Move pollBitcoind out of startBitcoindBlockPolling

* Rework startBitcoindBlockPolling to not return Future[Cancellable]

* Stop NodeCallbacksStreamManager inside of NodeAppConfig.stop()

* Make BitcoindSyncState which encapsulates syncing with bitcoind and polling of mempool

* Fix bug where processingBitcoindBlocks cannot be top level val as that method may be used by different parts of the codebase

* Rename isPolling -> processingBitcoindBlocks

* Cleanup more resources in appServerTest/test

* Complete polling stream regardless if we synced blocks are not to complete the Future returned by pollBitcoind

* Empty commit to run CI
2022-08-02 06:40:17 -05:00
Chris Stewart
4b83286922
2022 07 25 wallet api refactor (#4545)
* Move MockChainQueryApi/NodeApi out of BaseWalletTest

* refactor BitcoinSServerMain.start() to return WalletHolder

* Add walletConfig to WalletWithBitcoind

* Move findOutputsBeingSpent into WalletApi

* Add WalletApi.clearAllAddresses(), HDWalletApi.{findAccount, getnewaddress(account)}

* Add HDWalletApi.fundRawTransaction() with an account as parameter

* Add WalletApi.findByScriptPubKey()

* Fix lots of tests in WalletIntegrationTest

* Create WalletApi.processOurTransaction()

* Get things compiling

* Fix tag integration test

* Refactor AnyDLCHDWalletApi -> DLCNeutrinoHDWalletApi

* Fix docs

* Get postgres tests passing locally

* Move initBalance map before calling callback in CallBackUtilTest

* Get compile working, rename to destroyOnlyWalletWithBitcoindCached

* Fix docs

* Fix missing destroyWalletAppConfig

* Fix scalafmt

* Fix bug in dlcWalletTest where wallet db thread pools weren't being shutdown after unit test completes

* Empty commit
2022-07-31 08:04:39 -05:00
Chris Stewart
2cd8c80f67
Refactor so we don't create multiple WalletAppConfig in testkit (#4548)
* Refactor so we don't create multiple WalletAppConfig in testkit

Fix buildBip39PasswordConfig

* Remove bip39Password argument from dlcWalletTest fixtures

* Fix allowExternalDLCAddresses config in tests

* Remove unnecessary overrides of BitcoinSWalletTest.getFreshConfig, fix docs

* Add DLCExecutionBackendTest.afterAll()

* Fix nodeTest bip39Password usage

* Fix bug where we weren't using correct config inside of getFreshConfig

* Fix DLCExecutionBitcoindBackendtest.afterAll()

* Fix RescanDLCTest
2022-07-30 17:20:18 -05:00
Chris Stewart
a2e7c428cf
Bump sleeps on CallbackUtilTest (#4534)
* Bump sleeps on CallbackUtilTest

* Bump timeout even more

* Reduce timeout again

* Re-bump to 5 minutes

* Make sure we shutdown actor system in test fixture BitcoinSServerMainBitcoindFixture
2022-07-25 06:36:48 -05:00
Chris Stewart
b905afa65e
2022 07 20 wallet rescan stream (#4530)
* Implement rescan with akka streams

Get basic stream working with rescan

Fix bug where we weren't using rescan specific threadpool

Implement killswitch for rescan

* WIP: Expose promise to allow external completion of rescan stream

* Rework RescanStarted.RescanStarted to contain a promise to complete the stream early, and a future that represents the completed streams materialized value

* Comment cleanup

* Fix compile errors, remove killswitch

* Fix 2.12.x compile

* Introduce ActorSystem into wallet, refactor rescans to use that ActorSystem

* Fix import

* Fix bug where we were prepending instead appending to batched Vector

* Propogate RescanState upwards into WalletRoutes

* Refactor fetching of filters to be a Flow
2022-07-24 12:26:21 -05:00
Chris Stewart
d4210fad65
Pull over changes from loadwallet branch (#4527) 2022-07-21 12:12:32 -05:00
Chris Stewart
0a127368f0
2022 07 18 node callback stream manager (#4520)
* Implement akka stream proxy for nodecallbacks

Add killswitch to createBitcoindNodeCallbacksForWallet

Add unit test for killswitch in createBitcoindNodeCallbacksForWallet

Add delays to make sure callbacks are executed

Fix rebase

Move killswitch out of methods into class level val

* Bump timeout

* Attempt to implement NodeCallbacks proxy with akka streams

* Refactor CallbackUtil to use NodeCallbackStreamManager

* Rebase & remove killswitch

* Implement NodeCallbackStreamManager.+ method
2022-07-20 06:28:55 -05:00
rorp
5d309fbae0
Fee rate WS notification (#4518)
* Fee rate WS notification

* make the test more CI friendly

* fix the default config

* always return errors

* fix build

* scalafmt

* send notifications every time
2022-07-19 16:41:06 -05:00
Chris Stewart
0cb93ddbf4
2022 07 17 node callback akka streams (#4516)
* Implement akka stream proxy for nodecallbacks

Add killswitch to createBitcoindNodeCallbacksForWallet

Add unit test for killswitch in createBitcoindNodeCallbacksForWallet

Add delays to make sure callbacks are executed

Fix rebase

Move killswitch out of methods into class level val

* Bump timeout
2022-07-19 13:45:41 -05:00
rorp
e90e372e54
Add sync JSON field to the getinfo endpoint output (#4452)
* Add `sync` JSON field to the `getinfo` endpoint output

* improve test coverage

* update docs

* rename sync flag

* WebSocket notifications

* fix unit tests

* fix unit tests

* increase test timout
2022-07-12 14:41:43 -05:00
Chris Stewart
1ae4d40dac
Implement rescancomplete callback (#4475)
* Implement rescancomplete callback

* Add walletName to payload

* Add to documentation
2022-07-09 16:23:15 -05:00
rorp
fdf281b469
DLC <-> contact mapping (#4346)
* DLC <-> contact mapping

* updated docs

* populate dlc/contact mapping automatically

* typo

* respond to the PR comments

* rename `contact` to `peer`

* fix unit tests

* create a contact when an incoming offers gets created

* drop dlc_contact_mapping table

* fix build

* update the docs

* Revert "update the docs"

This reverts commit 2386adadcd.

* revert dlc-contact-* endpoints
t Please enter the commit message for your changes. Lines starting
2022-06-14 08:14:28 -05:00
Shreyansh
d8fc8e588f
Remove Spv code (#4356)
* change node tests to neutrino

* get node test working

* merge DataMessageHandlerTest and DataMessageHandlerNeutrinoNodeTest

* delete unused files

* remove commented out spv parts

* formatting

* delete spv node

* remove merkle callback for neutrino node

* remove spv node wallet callbacks

* formatting

* remove SpvWalletApi

* replace SpvTestConfig with NeutrinoTestConfig

* more replace SpvTestConfig with NeutrinoTestConfig

* minor fix

* fix tests
2022-05-30 07:57:31 -05:00
rorp
f680ab8691
Persist whether wallet is rescanning in the database (#4326)
* Persist whether wallet is rescanning in the database

* fix cli

* fix build

* fix unit tests

* fix postgres tests

* remove wallet_state table

* fix rescan bug

* cleanup

* revert Cancellable's

* Cleanup

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-05-23 19:03:02 -05:00
Chris Stewart
f4d864fab8
2022 04 26 Startup time of appServer (#4294)
* Make Server's route be async

* WIP

* Add StartedBitcoinSAppConfig to indicate when tor starts up

* Add torStarted flag to getinfo response
2022-04-28 13:50:28 -05:00
rorp
f5940c93d4
Contact list (#4234)
* Contact list

* fix unit tests

* Add contact rpc tests

* Add documentation

* Refactor ConsoleCli into app-commons

* Add commands to bitcoin-s-cli

* Fix timeout for rescan

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-04-21 13:02:35 -05:00
Chris Stewart
37da24b94b
Add SignDbState to fix rescan exception (#4246)
* Add SignDbState to fix rescan exception

* Cleanup

* Add test, doesn't pass

* Add unit tests, add invariants

* Add unit test to appServerTest

* Fix handling so an exception is not thrown internally when a contractId does not exist
2022-04-11 08:06:53 -05:00
Chris Stewart
f2fa56221d
2022 03 31 collateral invariant (#4232)
* DLCMessage.totalCollateral -> DLCMessage.collateral

* Fix bad names of {DLCOfferTLV,DLCAcceptTLV}.totalCollateralSatoshis, its not the total collateral, rather the collateral being contributed by the offerer and acceptor

* Fix rebase

* Fix appServerTest

* Fix docs
2022-03-31 09:48:08 -05:00
rorp
a747f84bbb
Mark extarnal payout addresses (#4173)
* Mark extarnal payout addresses

* fix unit tests
2022-03-09 15:37:16 -06:00
Chris Stewart
668ab21ca1
2022 03 09 label refactor (#4175)
* Rename existing getaddresslabels -> getaddresslabel

* Fix missing rename of GetAddressLabel

* Modify pk constraint on wallet_address_tags to be tag_name rather than tag_type

* Add dropaddresslabel for a specific address

* Fix migrations

* Add unit tests and fix existing tests

* Add docs
2022-03-09 15:36:44 -06:00
rorp
aeb3169884
getdlcoffer RPC (#4166)
* getdlcoffer RPC

* fix build

* unit tests
2022-03-07 14:28:00 -06:00
rorp
44b2ca3c3d
Improve validation error messages (#4141)
* Improve validation error messages

* fix unit tests
2022-03-01 05:27:32 -06:00
rorp
c1dccd7831
Offer inbox RPC (#4129)
* Offer inbox RPC

* fix syntax errors

* change parameter type

* change receivedAt type

* rename RPC methods

* WS support

* fix syntax errors
2022-02-28 14:25:46 -06:00
Chris Stewart
3ba95700b2
Add RescanState, make sure we don't start concurrent rescans (#4131) 2022-02-25 05:23:51 -06:00
rorp
5777ec1c31
Add an ability to set custom payout and change addresses (#4101)
* Add an ability to set custom payout and change addresses

* config changes

* formatting

* respond to the comments
2022-02-18 09:29:00 -06:00
Chris Stewart
e802254a20
2022 01 24 rm appconfig varargs (#4011)
* Get things compiling

* Get tests passing

* Fix docs
2022-01-25 07:25:05 -06:00
Chris Stewart
52dcf51e82
Automatically download binaries if they are used in test suite (#4005)
* Remove download binary scripts from CI, try to make binary downloads contingent on tests being run in a project

* Cleanup docs to remove unecessary downloadBitcoind
2022-01-23 17:43:30 -06:00
Chris Stewart
21de609ed8
2022 01 22 cetsignatures refactor (#4004)
* Get refactor working where we decouple CETSignatures and the partial refund signature

* Add DLCAcceptWithCetSigs() for the case where we have a refund sig, but no cet signatures

* Fix bugs

* Fix sighash parsing bugs
2022-01-23 16:13:36 -06:00
Chris Stewart
7ee1f0f406
Implement batching of database calls for our chain callback (#4003) 2022-01-23 07:39:10 -06:00
rorp
f438ce2897
Refactor zipDatadir (#3999)
* Refactor zipDatadir

* cleanup
2022-01-21 10:03:53 -06:00
rorp
ee3ee53191
RPC password authentication (#3961)
* RPC password authentication

* fix compiler error

* add more unit tests

* change default behavior

* Cleanup and add unit test for sad path for rpc authentication

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-01-13 05:55:33 -06:00
Chris Stewart
9dd126bb9f
Modify estimatefee endpoint to return a number rather than a string (#3973)
* Modify estimatefee endpoint to return a number rather than a string

* Fix compile

* Fix infinite loop on converting to sats/vb, add tests
2022-01-12 12:17:04 -06:00
Chris Stewart
66e23b61a8
2021 01 11 issue 3938 (#3971)
* WIP

* Remove blocking on Future that wont complete until the websocket is disconnected

* Add Sink.ignore to drain webscoket queue if no one is connected
2022-01-12 09:04:40 -06:00
Chris Stewart
a7af8cac4c
Patch bug where zipdatadir doesn't work if the directory was not created (#3959)
* Patch bug where zipdatadir doesn't work if the directory was not created

* Remove uneeded Files.createFile()
2022-01-04 15:11:41 -06:00
Chris Stewart
d06b064b6b
2021 12 28 blockprocessed callback (#3946)
* WIP

* Get appServerTest passing with chain notifications

* Reorder where SourceQueue gets initialized so that we can use it to construct the wallet's callbacks

* DRY

* Fix missing callback for neutrino chaincallbacks

* Bump timeout

* Bump timeout

* When wallet is empty call doSync() with the last blockheader bitcoind has seen

* Revert skipping compact filters on IBD for bitcoind

* Revert logging
2021-12-30 13:41:17 -06:00
Chris Stewart
50bec2abc6
2021 12 14 websockets (#3906)
* WIP

* Get basic websocket working (sort of)

* Push websocket fixes

* WIP: Implementing SpendingInfoDb json serializer

* Implement onreserved callbacks, add json serializers for SpendingInfoDb

* Get first unit test working for websockets when a address is generated

* Rework WalletNotification to have case classes, get unit test working for receiving addresses over the websocket

* WIP

* Add websocket callback for when a wallet processes a block

* Cleanup

* basic updates to unit test

* Fix compile

* Fix submodule

* Fix compile

* Get both unit tests passing when run by themselves

* Fix so both test cases can be run

* Implement unit tests for blockpressed and reservedutxos websockets

* Fix RoutesSpec get a block header test

* Implement configuration for wsbind and wsport via bitcoin-s.conf

* Add some nonblocking sleeps on all WebsocketTests

* Add documentation

* Properly close the server with .terminate() rather than .unbind()

* Add BitcoinSServerMainBitcoindFixture.afterAll()

* Add println

* Add more println

* Try to downgrade bitcoind version

* Fix datadir bug

* Cleanup callbacks so they don't have nested futures

* Fix SpendingInfoDb.apply() pattern match

* Add spendingInfoDb json serializer test, does not pass

* Fix SpendingInfoDb json serializer

* Make small refactors

* Fix compile

* Add maxBufferSize, change overflow strategy to OverflowStrategy.dropHead

* Address Nadav's code review

* Address ben's code review
2021-12-25 11:11:04 -06:00
rorp
4b07629d56
Add getmediantimepast RPC call (#3921)
* Add `getmediantimepast` RPC call

* update docs

* respond to the PR comments
2021-12-22 16:15:30 -06:00
Chris Stewart
4646ef6e19
Remove BitcoinSServerMain.startedFP (#3928) 2021-12-20 14:29:03 -06:00
Chris Stewart
3cd57d37f5
Remove backupwallet / backup oracle (#3920)
* Remove backupwallet / backup oracle

* Add zipdatadir documentation

* Fix submodule
2021-12-20 08:19:46 -06:00
Chris Stewart
d393848cc2
2021 11 30 issue 3847 (#3862)
* Start pulling over accept serializers from dlc message spec PR

* Get sign tlv json serialization working

* Implement decodeaccept,decodesign in CoreRoutes

* Move messages to DLCTestUtil, add documentation
2021-12-01 13:30:18 -06:00
Chris Stewart
8765c2f845
Disjoint union dlc (#3839)
* Implemented Disjoint Union DLC related data structures with dlcTest passing

Broke DLCClientTest up into multiple test files

Optimized DLC execution unit tests by checking all outcomes on a single setup DLC

Refactored DLCTest to allow for Disjoint Union DLC construction and validation, added tests

Responded to review

Fixed after cherry-pick

Fixed docs

* Fixed things after rebase

* Rebase

* Fix json serializer

* Finish fixing compile

* Start trying to make APIs better for multi oracle contract infos

* Clean things up in the GUI, try to make failures on disjoint union contracts as explicit as possible

* Use less numDigits as optimization for non secp CI test cases

* Fix compile

* Refactor BroadcastDLCDialog

* Fix test case optimization

* Clean up comment

Co-authored-by: nkohen <nadavk25@gmail.com>
2021-11-29 19:26:00 -06:00
Chris Stewart
a40ef1ab21
Break existing api to get new address to remove the requirement for null label (#3818) 2021-11-12 09:58:50 -06:00
rorp
21fab4ee70
Improve app server validation error handling (#3796) 2021-11-05 10:03:16 -05:00
Chris Stewart
aa748c012f
2021 11 03 protocol version (#3793)
* ProtocolVersion WIP

* Start incorporating protocol version into DLC Offer

* Fix doc
2021-11-03 15:13:19 -05:00