Commit Graph

575 Commits

Author SHA1 Message Date
benthecarman
64183568fe
Allow creation of TaprootTxSigComponent (#4445)
* Allow creation of TaprootTxSigComponent

* PreviousOutputMap using MapWrapper
2022-07-04 08:46:10 -05:00
Shreyansh
111df25df0
Fix sync issues post #4408 (#4441)
* fix node test shared fixtures bug

The cached bitcoind fixtures were used and stopped in UnsyncedNeutrinoNodeTest which causes an error if NeutrinoNodeTest is run at the same time on high performant systems, which is why it escaped CI. Merges NeutrinoNodeTest and UnsyncedNeutrinoNodeTest

* fix possible issues with PeerMessageReceiverTest

This reverts commit 55e7cafee6c6fd8515650cf65d96f9d92a2a8812.

* fix filter sync issue when wallet creation time indicates already synced

* move switch peer test to NeutrinoNodeWithUnachedBitcoindTest
2022-07-04 08:42:45 -05:00
Shreyansh
42564bc810
Find and switch peers (#4408)
* add support to find and switch peers

* fix compile on 2.12

* allow empty config peers in regtest

* fix test

* minor fixes

changes timeouts, fix issue with ipv6 dns seeds, initDisconnect when disconnected bug, dbPeers order fix

* fix: not removing peers on initialization timeout

* fix: query again when previous failed

* fix: wrong condition for deferred peers

* restore log levels

* clean up

* add PeerStack to allow trying peers based on priority values

* fix migrations

* changes from comments

* use StartStopAsync

* changes from comments

* fix switch if peer down test

* changes from comments
2022-06-28 10:19:13 -05:00
benthecarman
99b75d166f
Fixes support for bitcoind taproot descriptor wallets (#4415) 2022-06-23 07:10:51 -05:00
Shreyansh
4dc1bc7050
add bitcoind v23 rpc (#4368) 2022-06-21 13:20:47 -05:00
benthecarman
531845adfa
Accept keysend and amp in lnd fixtures (#4407) 2022-06-21 07:53:37 -05:00
benthecarman
a0ab0638f8
Allow for custom channel sizes in LndRpcTestUtil.createNodePair (#4398) 2022-06-16 19:36:03 -05:00
rorp
1ad540703c
Improve bitcoind connection retry logic (#4386) 2022-06-14 08:40:04 -05:00
rorp
fdf281b469
DLC <-> contact mapping (#4346)
* DLC <-> contact mapping

* updated docs

* populate dlc/contact mapping automatically

* typo

* respond to the PR comments

* rename `contact` to `peer`

* fix unit tests

* create a contact when an incoming offers gets created

* drop dlc_contact_mapping table

* fix build

* update the docs

* Revert "update the docs"

This reverts commit 2386adadcd.

* revert dlc-contact-* endpoints
t Please enter the commit message for your changes. Lines starting
2022-06-14 08:14:28 -05:00
benthecarman
c2d8735dd7
LND: Add configs for gossip in test env (#4378) 2022-06-13 11:58:50 -05:00
Shreyansh
d8fc8e588f
Remove Spv code (#4356)
* change node tests to neutrino

* get node test working

* merge DataMessageHandlerTest and DataMessageHandlerNeutrinoNodeTest

* delete unused files

* remove commented out spv parts

* formatting

* delete spv node

* remove merkle callback for neutrino node

* remove spv node wallet callbacks

* formatting

* remove SpvWalletApi

* replace SpvTestConfig with NeutrinoTestConfig

* more replace SpvTestConfig with NeutrinoTestConfig

* minor fix

* fix tests
2022-05-30 07:57:31 -05:00
benthecarman
6845caf778
Make all uint64 types from lnd a UInt64 (#4332)
* Make all uint64 types from lnd a UInt64

* Make all uint32 types from lnd a UInt32
2022-05-11 05:05:16 -05:00
Chris Stewart
2f4bbf7014
Upgrade flyway to 8.5.9 (#4311)
* Upgrade flyway to 8.5.9

* Remove comment
2022-05-04 11:24:51 -05:00
rorp
3dc709386a
Upgrade Eclair to v0.7.0 (#4308)
* Upgrade Eclair to v0.7.0

* Scala 2.12 compatibility

* reformat
2022-05-02 09:04:56 -05:00
Chris Stewart
f4d864fab8
2022 04 26 Startup time of appServer (#4294)
* Make Server's route be async

* WIP

* Add StartedBitcoinSAppConfig to indicate when tor starts up

* Add torStarted flag to getinfo response
2022-04-28 13:50:28 -05:00
Chris Stewart
5ad5993f1a
Refactor WalletAppConfig test fixture destructions to stop the config… (#4266)
* Refactor WalletAppConfig test fixture destructions to stop the config when tearing down

* Fix test fixture where walletappconfig might not be started
2022-04-17 09:23:21 -05:00
Chris Stewart
24125eaca3
Fix case where we broadcast tx with a tag, and then also add the tag when we receive the tx in a block (#4239) 2022-04-03 13:40:19 -05:00
Chris Stewart
f2fa56221d
2022 03 31 collateral invariant (#4232)
* DLCMessage.totalCollateral -> DLCMessage.collateral

* Fix bad names of {DLCOfferTLV,DLCAcceptTLV}.totalCollateralSatoshis, its not the total collateral, rather the collateral being contributed by the offerer and acceptor

* Fix rebase

* Fix appServerTest

* Fix docs
2022-03-31 09:48:08 -05:00
Chris Stewart
8081772b57
Allow acceptor to upsert same outpoint multiple times for different offers (#4230)
* Allow acceptor to upsert same outpoint multiple times for different offers

* Remove extra log

* Revert DLCOffer invariant

* Fix OP_RETURN tests
2022-03-31 07:47:10 -05:00
Chris Stewart
d5807daeab
Remove SpendingInfoDb.txid parameter (#4199) 2022-03-19 15:21:36 -05:00
Chris Stewart
fe86233489
Add better tor message to UI (#4181)
* Add better tor message to UI

* Fix compile
2022-03-11 11:17:52 -06:00
Chris Stewart
374c1d7b9f
Upgrade RescanHandlingTest to use the newest release of bitcoind (#4151) 2022-03-02 09:59:25 -06:00
Chris Stewart
7a5c2971dd
Refactor receive flow to use the same utxo state transitions (#4134)
* Refactor receive flow to use the same utxo state transitions as the rest of the wallet

Get tests passing again with refactored utxos

Remove unecessary wallet methods

Fix test cases

* Remove fixture on UTXOHandlingTest

* Fix bug where funding happens in parallel in fixture and can lead to blockchain reorgs
2022-02-26 19:08:50 -06:00
Chris Stewart
b46574c0c4
2022 02 18 sync since creationtime pt2 (#4109)
* Get things compiling and tests passing

* Fix bug where we weren't using filter height correctly

* Fix docs

* Move creationTime into KeyManagerAppConfig

* Fix nodeTest/test test fixtures
2022-02-22 08:00:14 -06:00
rorp
ffd7d921a8
Database support for incoming offers (#4105)
* Database support for incoming offers

* fix tests
2022-02-18 14:03:05 -06:00
rorp
5777ec1c31
Add an ability to set custom payout and change addresses (#4101)
* Add an ability to set custom payout and change addresses

* config changes

* formatting

* respond to the comments
2022-02-18 09:29:00 -06:00
Chris Stewart
dc47c070a2
Fix SpendingInfoDAO.findOutputsReceived() bug (#4090) 2022-02-14 10:11:23 -06:00
rorp
bce58ba33d
Validate announcement signatures on create/accept offer (#4071)
* Validate announcement signatures on create/accept offer

* unit tests
2022-02-11 06:50:45 -06:00
Chris Stewart
5aeecdb893
Reworking/refactoring acceptDLCOffer (#4048)
* Part 1 of reworking/refactoring acceptDLCOffer

* scalafmt

* WIP

* Move offer creation into initDLCForAccept

* Refactor method name to getDlcDbOfferDbAccountDb

* Push to github to force re-run of CI
2022-02-07 11:41:17 -06:00
benthecarman
142612f034
Make newConfigOfType use Vector[Config] (#4039) 2022-02-04 06:14:16 -06:00
Chris Stewart
b918cf78b7
Fix bugs where we were building internally inconsistent SpendingInfoDb (#4016)
* Fix bugs where we were building internally inconsistent SpendingInfoDb

* Fix error message
2022-01-27 07:22:09 -06:00
Chris Stewart
bd5bcfef3a
2022 01 18 issue 3969 Add serialization_version to global_dlc_data (#3992)
* Get refactor working where we decouple CETSignatures and the partial refund signature

* WIP

* Implement migration to specify serialization_version on global_dlc_data table

* Fix migrations test

* WIP

* implement contractid computation

* Get migration working for wallet on testnet

* Add check to make sure we only try to migrate old wallets

* Refactor DLCDataManagement.getOfferAndAcceptWithoutSigs to not use DLCTxBuilder

* WIP

* Bubble up Option[] into DLCWallet, trivially assert on it and then use .get. We will have to do the refactor of DLCWallet sometime in the future

* Change global_dlc_data serializationVersion at database level to not have an Option

* Add DLCDbState, propogate it through the codebase

* Remove log

* Remove invariant for making sure all DLCs are defined on migraiton, add log for the case of issue 4001

* Add filter on ALPHA serialization DLCs. We dont need to migrate DLCs that are using the beta serialization

* Rebase onto 4004

* Refactor to use the refund signature in the accept/sign message

* Add output index to the return of DLCTxBuilder.buildFundingTransaction()

* Fix compile

* Remove log
2022-01-26 14:57:45 -06:00
Chris Stewart
98c5d816ac
2022 01 25 issue 4014 (#4015)
* Refactor testkit tests to use same nodeCallbacks that BitcoinSServerMain uses for neutrino

* Add test case

* Add logs for callbacks

* Cleanup

* Add test case for when funds are spent when we are offline

* Turn off logging again

* Cleanup more logs and comments
2022-01-26 13:16:15 -06:00
Chris Stewart
e802254a20
2022 01 24 rm appconfig varargs (#4011)
* Get things compiling

* Get tests passing

* Fix docs
2022-01-25 07:25:05 -06:00
Chris Stewart
21de609ed8
2022 01 22 cetsignatures refactor (#4004)
* Get refactor working where we decouple CETSignatures and the partial refund signature

* Add DLCAcceptWithCetSigs() for the case where we have a refund sig, but no cet signatures

* Fix bugs

* Fix sighash parsing bugs
2022-01-23 16:13:36 -06:00
Chris Stewart
7ee1f0f406
Implement batching of database calls for our chain callback (#4003) 2022-01-23 07:39:10 -06:00
rorp
f438ce2897
Refactor zipDatadir (#3999)
* Refactor zipDatadir

* cleanup
2022-01-21 10:03:53 -06:00
rorp
ee3ee53191
RPC password authentication (#3961)
* RPC password authentication

* fix compiler error

* add more unit tests

* change default behavior

* Cleanup and add unit test for sad path for rpc authentication

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-01-13 05:55:33 -06:00
Chris Stewart
bf8b165fe9
Implement recovery for when we cannot receive a fee rate from a FeeRateApi (#3975)
* Implement recovery for when we cannot receive a fee rate from a FeeRateApi. Now we return -1 sats/vbyte if we cannot get the fee rate from a fee provider rather than throwing an internal server error

* Fix compile
2022-01-12 14:09:15 -06:00
Chris Stewart
9dd126bb9f
Modify estimatefee endpoint to return a number rather than a string (#3973)
* Modify estimatefee endpoint to return a number rather than a string

* Fix compile

* Fix infinite loop on converting to sats/vb, add tests
2022-01-12 12:17:04 -06:00
Shreyansh
a58ef1cd02
Storing peers in database (#3773)
* rebase

* fix

* fix

* remove unwanted diffs

* fix

* formatting

* changes from comments

* Revert "changes from comments"

This reverts commit eb8a795718dc590802a19c7ce4cce4501bd2de1f.

* changes from comments

* add PeerManager

* add sha3-256

* add NetworkUtilTest

* formatting
2022-01-09 07:21:59 -06:00
Chris Stewart
d06b064b6b
2021 12 28 blockprocessed callback (#3946)
* WIP

* Get appServerTest passing with chain notifications

* Reorder where SourceQueue gets initialized so that we can use it to construct the wallet's callbacks

* DRY

* Fix missing callback for neutrino chaincallbacks

* Bump timeout

* Bump timeout

* When wallet is empty call doSync() with the last blockheader bitcoind has seen

* Revert skipping compact filters on IBD for bitcoind

* Revert logging
2021-12-30 13:41:17 -06:00
Chris Stewart
50bec2abc6
2021 12 14 websockets (#3906)
* WIP

* Get basic websocket working (sort of)

* Push websocket fixes

* WIP: Implementing SpendingInfoDb json serializer

* Implement onreserved callbacks, add json serializers for SpendingInfoDb

* Get first unit test working for websockets when a address is generated

* Rework WalletNotification to have case classes, get unit test working for receiving addresses over the websocket

* WIP

* Add websocket callback for when a wallet processes a block

* Cleanup

* basic updates to unit test

* Fix compile

* Fix submodule

* Fix compile

* Get both unit tests passing when run by themselves

* Fix so both test cases can be run

* Implement unit tests for blockpressed and reservedutxos websockets

* Fix RoutesSpec get a block header test

* Implement configuration for wsbind and wsport via bitcoin-s.conf

* Add some nonblocking sleeps on all WebsocketTests

* Add documentation

* Properly close the server with .terminate() rather than .unbind()

* Add BitcoinSServerMainBitcoindFixture.afterAll()

* Add println

* Add more println

* Try to downgrade bitcoind version

* Fix datadir bug

* Cleanup callbacks so they don't have nested futures

* Fix SpendingInfoDb.apply() pattern match

* Add spendingInfoDb json serializer test, does not pass

* Fix SpendingInfoDb json serializer

* Make small refactors

* Fix compile

* Add maxBufferSize, change overflow strategy to OverflowStrategy.dropHead

* Address Nadav's code review

* Address ben's code review
2021-12-25 11:11:04 -06:00
rorp
4b07629d56
Add getmediantimepast RPC call (#3921)
* Add `getmediantimepast` RPC call

* update docs

* respond to the PR comments
2021-12-22 16:15:30 -06:00
Chris Stewart
4ca586ca46
2021 12 19 dlc callbacks (#3923)
* Implement DLC wallet callbacks

Add callbacks and tests for Claimed/RemoteClaimed

Add unit test for confirmed state

* Add some docs

* Address ben's code review

* Fix scalafmt
2021-12-20 14:28:52 -06:00
Chris Stewart
2d9d12816b
Add WalletCallbacks.onBlockProcessed() (#3912) 2021-12-16 12:26:48 -06:00
Chris Stewart
f71d3567ed
2021 11 23 crud action (#3851)
* WIP

* Finish implementing {CRUD,CRUDAction}.createAllAction()

* Try to make database inserts more atomic

* Consolidate more database calls in DLCWallet.initDLCForAccept()

* WIP2

* More refactor and using actions

* Add DLCIdDaoUtilNoPK for querying when dlcId is not a primary key

* Add DLCDataManagement.deleteDLC()

* make DLCWalletApi.createDLCOffer() use actions

* Rework DLCWallet.createNewDLCAccept() to use DBIOAction

* Use actions in DLCWallet.registerDLCAccept() a bit

* Create DLCActionBuilder, move actions to there

* Move more actions into ActionBuilder

* Implement transactionally on database writes for actions
2021-11-30 12:38:55 -06:00
Chris Stewart
8765c2f845
Disjoint union dlc (#3839)
* Implemented Disjoint Union DLC related data structures with dlcTest passing

Broke DLCClientTest up into multiple test files

Optimized DLC execution unit tests by checking all outcomes on a single setup DLC

Refactored DLCTest to allow for Disjoint Union DLC construction and validation, added tests

Responded to review

Fixed after cherry-pick

Fixed docs

* Fixed things after rebase

* Rebase

* Fix json serializer

* Finish fixing compile

* Start trying to make APIs better for multi oracle contract infos

* Clean things up in the GUI, try to make failures on disjoint union contracts as explicit as possible

* Use less numDigits as optimization for non secp CI test cases

* Fix compile

* Refactor BroadcastDLCDialog

* Fix test case optimization

* Clean up comment

Co-authored-by: nkohen <nadavk25@gmail.com>
2021-11-29 19:26:00 -06:00
benthecarman
155301fc1d
Allow lnd remote to work with certificate string (#3840) 2021-11-20 18:59:46 -06:00
rorp
a6898defe2
Support for Bitcoin Core v22 (#3834)
* Support for Bitcoin Core v22

* respond to the PR comments
2021-11-19 13:58:27 -06:00
benthecarman
079fa62073
Lnd v0.14.0 (#3835) 2021-11-19 06:55:56 -06:00
Chris Stewart
aa748c012f
2021 11 03 protocol version (#3793)
* ProtocolVersion WIP

* Start incorporating protocol version into DLC Offer

* Fix doc
2021-11-03 15:13:19 -05:00
rorp
403c78fd8f
Server Docker config (#3775) 2021-10-25 06:35:02 -05:00
rorp
d71208cf0f
Support Eclair v0.6.2 (#3765)
* Support Eclair v0.6.2

* cleanup
2021-10-19 11:46:54 -05:00
benthecarman
6d43d443ba
Initial c-lightning rpc support (#3755)
* Initial c-lightning support

* Respond to review
2021-10-19 10:58:29 -05:00
Chris Stewart
9668358807
2021 10 04 issue 3715 (#3724)
* WIP

* Setup DLC fixtures to work with bitcoind

* Implement test case for issue 3715

* turn logging back down, add another assertion for making sure funding tx is still unconfirmed

* Make sure we are broadcasting from acceptor

* Cleanup
2021-10-10 07:51:00 -05:00
Shreyansh
99107b61ea
Add PeerDAO (#3738)
* add PeerDAO

* update DbManagementTest expected migration count
2021-10-10 07:50:11 -05:00
benthecarman
f6169cc3af
Add LndInstanceRemote (#3710) 2021-10-02 16:15:40 -05:00
Chris Stewart
4a44f9cd58
Run appServerTest on both windows and mac (#3609)
* Run appServerTest on both windows and mac

* Add just mac test name

* Add downloadBitcoind task to windows matrix

* Make exception easier to be hit

* Remove deleting of directory as that shouldn't be needed

* Add support for not running bitcoind in test suites as a daemon on windows

* Fix bug where i forgot to re-add daemon config
2021-09-27 10:25:48 -05:00
Chris Stewart
48213d9b81
Implement MasterXPubDAO (#3451)
* Implement MasterXPubDAO

* Fix nits

* Add postgres migrations

* Make sure tables are getting cleaned on postgres

* Get dlcOracleTest test cases passing with validation of the master xpub in the database on startup

* Fix test case

* Fix more tests

* Move things around so keymanager does not need to depend on dbCommons

* Get all walletTest passing

* Add unit test for wallet startup

* Fix rebase

* Get all sqlite tests passing

* Get postgres test cases passing locally

* Remove early call to walletAppConfig.start() in node project

* Make KeyManagerAppConfig.externalEntropy private

* Fix MasterXpubDAO.existsOneXpub()

* Fix bug in migrating database for oralce project
2021-09-21 08:05:23 -05:00
Chris Stewart
132479d271
Implement ability to provide external entropy to bitcoin-s (#3672)
* Encapsulate initialization of DLCOracle.start() method

* Use internal WalletAppConfig.kmConf rather than passing in custom key manager parameters

* Add KeyManagerAppConfig.defaultAccountType

* Get all tests passing besides TrezorAddressTest

* Get TrezorAddressTest passing with provided entropy

* Add unit test to make sure we can always derive the seed

* Get docs compiling

* Fix dlcWalletTest test cases

* Add more test cases to keymanager

* Add the new configuration to the example configuration

* Add more test cases

* Remove coverage on 2.12 as it isn't accurate

* Rework DLCOracleAppConfig.start() to call kmConf.start() so the oracle can use entropy provided via bitcoin-s.conf
2021-09-18 09:49:11 -05:00
Chris Stewart
0fb2cd0149
Remove implicit conversions from BitcoinSAppConfig -> {WalletAppConfig,ChainAppConfig,NodeAppConfig} etc (#3652) 2021-09-08 13:54:18 -05:00
Shruthii RG
d53f164478
BitcoindRpcClient Improvements (#3600)
* Changes in bitcoind-rpc module

* Get things compiling

* Implement BitcoindInstanceRemote & BitcoindInstanceLocal

* Fix doc compile

* bitcoind remote instance test

* rename BitcoindInstanceRemote

* Scalafmt

* Applied requested changes(except one)

* implemented fromConfigFile and fromDataDir for BitcoindInstanceRemote

* implemented fromNetworkVersion

* BitcoindInstanceRemote changes

* Get things compiling

* Add implict type parameter to InstanceFactory

* Fix recursive reference of version <-> getNetworkInfo

* bug fix

* bug fix 2

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2021-09-07 08:49:01 -05:00
benthecarman
656e9b1b5d
Sort DLC table by last updated (#3607)
* Sort DLC table by last updated

* Add LastUpdatedDb

* Make visible

* Fix compile

* Add time, fix sorting

* Switch to local time

* fix compile

* Fix test

* Fix postgres migration

* Specify zoneId
2021-08-31 14:27:36 -05:00
Chris Stewart
0746b14331
Move BitcoindRpcAppConfig into the bitcoind-rpc project (#3610) 2021-08-27 06:43:31 -05:00
rorp
704b02ae6c
Use random ports for pre-packaged Tor daemon (#3604)
* Use random ports for pre-packaged Tor daemon

* fix unit tests

* configurable random ports

* shutdown hook
2021-08-26 13:02:49 -05:00
Chris Stewart
a5f4f9663c
Add BitcoinSServerMain test to connect to bitcoind (#3591)
* Add BitcoinSServerMain test to connect to bitcoind

* Add tor tests

* Fix compile

* Remove println

* Allow to reset BitcoinSServer promise

* Remove println

Co-authored-by: benthecarman <benthecarman@live.com>
2021-08-26 09:08:38 -05:00
Chris Stewart
b980f4894d
Implement BitcoinSAppConfig with ActorSystem (#3596) 2021-08-24 07:41:05 -05:00
benthecarman
495ab62104
Tor in Testkit (#3484)
* Tor packaged for testkit

* Add CI row

* Small fixes

* More fixes

* Keep tor running

* Skip tor test for old version of bitcoind

* Respond to review

* Fix with master

* Bump tor start time to same as app config

* Use cached tor for DLCNodeTest

* Disable parallel execution for dlc node test with tor

* Fix DLCNodeTest for tor and clearnet

* Fix name of CI rows
2021-08-23 13:06:14 -05:00
rorp
54ce2ebeb2
P2P reconnect logic (#3572)
* P2P reconnect logic

* some fixes

* more changes

* fix DuplicateFilters exception in case of  disconnect in the middle of syncing filters

* more fixes

* Repurpose maxconnections test to be just a generic reconnection test

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2021-08-20 05:51:51 -05:00
benthecarman
addb1254ee
Fix for if tor is already running (#3562) 2021-08-17 09:47:31 -05:00
benthecarman
621e8e9033
Rescan DLC Tests (#3515) 2021-08-11 07:29:33 -05:00
rorp
3d4cf1fc91
Tor support for all sub-projects (#3506) 2021-08-10 08:07:12 -05:00
Chris Stewart
4f7b6422ea
2021 08 06 app config refacotr (#3498)
* Move AppConfig into app-commons, reduce classpath deps on keyManager,tor

* Move test ServerArgParserTest

* Fix compile error
2021-08-07 14:36:11 -05:00
benthecarman
1051e6365a
DLC P2P Client (#3402)
* DLC P2P WIP

* P2PClient refactor (#10)

* Add to CI

* Remove unused

* Attempt to create DLCNode and Tests

* Full Tor support

* Get DLCNegotiationTest passing

* Config options, connect & send func

* Test for DLCNode

* Add createDLCNode to config

* Fix formatting

* Update DLC state after all other data is set

* Remove unneeded line

* Respond to some review

* 2021 07 26 dlc node code review (#13)

* WIP

* WIP2

* Rewrite tests not use Await.result()

* Skip Tor test on CI

* Cleanup threadpool leaks in tests

* Handle actor pattern matching better

* Respond to review

* Implement DLCNode.stop

* sock5 -> socks5

* Use Tcp.Unbind

* Respond to review

* Implement postStop

* Switch to unbind

Co-authored-by: rorp <rorp@users.noreply.github.com>
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2021-08-02 18:15:56 -05:00
Shreyansh
1426c31483
Multi peer: Sync with random peer (#3454)
* ensure random peer used for sync supports compact filters

* test file

* fix

* refactor

* rerun ci

* rerun ci

* changes from comments

* fix

* fix tests

* changes from comments

* refactor

* remove unintended diffs
2021-07-31 07:54:31 -05:00
Shruthii RG
e2c2798e60
Implemented the ability to fetch unconfirmed transactions (#3429)
* relay flag in version message made configurable

* modified test classes as well

* Update configuration.md

* reverted few files and cleaned up format

* fixed mistake

* enabled neutrino node to receive transactions as a callback

* added txid to log message

* changed txId to txIdBE

* unit test added for checking if we're getting transaction as a callback
2021-07-23 13:01:07 -05:00
Chris Stewart
1032669f21
Add unit test for moving seeds and making sure we can get the same public key after moving (#3441) 2021-07-23 13:00:27 -05:00
benthecarman
2597904019
Move DLCWalletApi to core module (#3438) 2021-07-21 15:25:18 -05:00
Shreyansh
4be2b2109b
[WIP] Adding multi-peer support (#3401)
* establish conn with multiple peers

squash 1

squash 2

squash 3

* add tests

* change log msg level

* minor changes

* fix

* formatting

* fix docs

* commit to rerun ci

* changes from comments

* fix bug

* rerun ci
2021-07-21 15:18:45 -05:00
Chris Stewart
b2b2ca45db
Add ability to configure postgres database backends for test cases via reference.conf (#3421) 2021-07-19 09:55:36 -05:00
Chris Stewart
b7b2a7099f
2021 07 15 dlc oracle pg (#3413)
* WIP

* Get postgres test passing by not allowing 0 byte (NULL character in UTF 8 string) in the utf8 generator
2021-07-17 12:19:49 -05:00
rorp
8b663d91b6
Tor support for P2P (#3311)
* Tor support for P2P

* fix stuck connections

* unit tests

* docs

* update docs

* respond to the PR comments

* fix docs
2021-07-13 14:27:24 -05:00
benthecarman
cada6fdc63
Fix DLC not storing nSequence for funding inputs (#3342)
* Fix DLC wallet not storing nSequence for funding inputs

* Fix DLCTest
2021-06-29 09:36:30 -05:00
benthecarman
cfe0c2f0d8
Multi wallet support for DLC database (#3289)
* Multi wallet support for DLC database

* Fix for postgres
2021-06-22 12:56:26 -05:00
benthecarman
fafc564da8
Update LND to v0.13.0-beta (#3290) 2021-06-21 16:20:08 -05:00
benthecarman
e63a12e47f
Better error message for invalid DLC refunds (#3209) 2021-06-15 10:57:50 -05:00
Chris Stewart
aaa7b42ae7
Add unit test we can handle spending funds and receiving funds in same tx (#3185)
* Add unit test we can handle spending funds and receiving funds in the same transaction inside the wallet

* Use fundRawTranaction() to make sure the utxo is reserved
2021-06-14 08:57:30 -05:00
Chris Stewart
7ba7f8b9ba
Try to add block generate to address in fixture setup to get around compact filter sync edge case (#3231)
* Try to add block generate to address in fixture setup to get around the case where block headers & filter headers are synced but compact filters are not

* Push to github to force re-run of CI 2

* Add ChainApi.getBestFilter(), use it in sync to determine if filter headers & filters are in sync

* Push to github to force re-run of CI 3

* Simplify getBestFilter, add unit test

* Add more comments, clean up some code
2021-06-08 12:47:28 -05:00
Chris Stewart
bf02e89faa
2021 05 29 dlc pnl (#3198)
* WIP

* WIP2

* Get tableview rendering something

* Add basic unit tests for accounting

* Add rate of return

* Add myPayout and theirPayout to json objects

* Address code review

* Fix 2.12 compile, adjust table view

* drop isInit column on table view

* Fix scala 2.13
2021-05-31 11:12:24 -05:00
benthecarman
0e701bc9d0
Fix verify funding sigs (#3194)
* Fix verify funding sigs

* Sort funding inputs

* Sort utxos in dlc signer

* Other multi input fixes & tests

* Fix compile error
2021-05-28 19:59:45 -05:00
Nadav Kohen
17d1145504
Removed extraneous findAll call from CRUD.updateAll (#3154)
* Removed extraneous findAll call from CRUD.updateAll

* Made update tests deal with unchanged element
2021-05-26 14:22:18 -06:00
benthecarman
be8e965367
DLC Wallet pulldown (#3138)
* DLC Wallet pulldown

* Respond to review
2021-05-25 17:31:01 -05:00
Nadav Kohen
ac3bae403b
Pulled down all remaining non-wallet non-gui code on adaptor-dlc (#3101) 2021-05-18 06:29:46 -05:00
Nadav Kohen
58070f4120
Pulled down dlc and dlcTest projects into core and dlcTest (#3068)
* Pulled down dlc and dlcTest projects into core and dlcTest

Added dlcTest to CI runs and fixed compilation issues

* Fixed docs

* loosened fee rate bound assertions to only sanity test
2021-05-14 12:27:21 -05:00
Chris Stewart
a2911f31ed
Fix race condition with BitcoindChainHandlerViaZmqTest (#2990)
Turn off logging again

Try to move things around even more to make sure zmq is started

Turn logging level down to ERROR to hopefully get some meaningful output

Adjust both valdiation & ChainHandler to be ERROR level logging

switch zmqpubrawblock to be the port we know is free

Reduce logging to WARN

Bump bitcoind max retries to 120

Fix rebase

Revert logging

Remove some noisy logs

Restore logs to trace level

Add explicit tag
2021-05-01 13:11:13 -05:00
Chris Stewart
38baea5e24 refactor BitcoindRpcTestUtil test methods to take ZmqConfig rather than zmqPort (#3002) 2021-04-30 10:59:16 -05:00
Chris Stewart
f792fb3480 Fix database pool name for postgres database connection pools (#2997)
* Fix database pool name for postgres database connection pools

* Add timestamp to pool name
2021-04-30 06:38:37 -05:00
Chris Stewart
77cd94ac41 2021 04 27 wallet fixtures config (#2980)
* Refactor MultiWalletTest to actually shutdown the walletAppConfigs

* Remove 'implicit' modifier from getFreshWalletConfig. This is a problem as it can be used implicitly to bind resources (threads, files etc) that need to be cleaned up. Since it is implicitly passed, it is very hard to remember to clean up these resources

* Cleanup TrezorAddressTest

* Remove unecessary BitcoinSWalletTest.afterAll()

* Fix docs

* Removal of datadir race condition

* Cleanup nodeTest to shutdown the transitive chainAppConfig required by NodeAppConfig

* remove experimental tag on NeutrinoNodeTest

* Remove extra line

* Cleanup after BitcoindChainhandlerViaZmqTest

* Push to github to force re-run of CI 2
2021-04-29 06:30:58 -05:00
Chris Stewart
85fed08c58 Reduce pg connections from 300 -> 50 in test cases (#2974)
* Reduce pg connections from 300 -> 50 in test cases

* Reduce connection count to 25

* Push to github to force re-run of CI

* Switch explorer over to prod for now as test is down to get CI passing

* Reduce shared buffers memory size

* Push to github to force re-run of CI 2
2021-04-28 06:22:56 -05:00
benthecarman
e3017fd17d Peer Message Receiver Refactor (#2938) 2021-04-27 15:54:44 -05:00
Chris Stewart
73939a15fc Call WalletAppConfig.stop() when destroying wallet in test fixtures (#2975)
* Fix bug introduced in PR 2597, we need to explicitly call WalletAppConfig.stop() now rather than transitively calling it via Wallet.stop()

* Fix calls to wallet.stop() -> wallet.walletAppConfig.stop() in BitcoindBlockPolling,BitcoindBackend. Also add some explicit type annotations
2021-04-26 15:48:33 -05:00
Chris Stewart
3483a461f1 Don't wrap pg.close() in a Try and then do nothing with it, propogate the exception (#2972) 2021-04-26 09:45:19 -05:00
Chris Stewart
a27d4acd9f Get FilterSync test working with cached bitcoind in chainTest project (#2952)
* Get FilterSync test working with cached bitcoind in chainTest project

* Small refactor to be DRY

* Fix docs

* Refactor ChainSyncTest to use 1 cached bitcoind, as a by product add ChainWithBitcoindNewestCachedUnitTest

* Remove unecessary mixin trait

* Fix missing ChainWithBitcoindNewestCachedUnitTest.afterAll()

* Reduce thread pool size for akka's internal dispatcher in unit tests from 2 -> 1. Same with the blocking dispatcher

* Add comment
2021-04-26 07:41:30 -05:00
Chris Stewart
e064cd77ea Fix missing teardown code for MultiWalletRpcTest (#2946) 2021-04-23 09:31:12 -05:00
Chris Stewart
16538980e3 Fix missing super.stop() to shutdown DbAppConfig db connection pool (#2943)
* Fix missing super.stop() to shutdown DbAppConfig db connection pool inside of all DbAppConfig subclasses that override stop()

* Fix import

* Remove implicit modifier to NodeUnitTest.getFreshConfig(), make sure we are shutting down chainAppConfig inside NodeUnitTest

* Remove chainTest subclasses implementing ActorSystem, make sure we do correct cleanup in ChainUnitTest

* Decouple CachedChainAppConfig & CachedAppConfig. When using only one sub module in the CachedAppConfig, teardown may fail as the other modules might not be started such as using chainAppConfig in chainTest, BitcoinSAppConfig.stop() requires us to call nodAppConfig.stop() first, which isn't started.

* Push to github to force re-run of CI 4
2021-04-23 08:21:56 -05:00
Chris Stewart
c95c0f9706 Move wallet scheduler into WalletAppConfig (#2933)
* Move wallet scheuler into WalletAppConfig, use scheduler.shutdownNow() so we actually shutdown the scheduler rather than wait for all queued tasks to finish executing before shutting down

* Fix typo

* Move rescan thread pool in to walletAppConfig, shut it down on wallet shutdown

* Fix compile
2021-04-21 11:06:29 -05:00
Chris Stewart
de5f7fc7f9 Reduce number of threads in postgres connection pool for tests (#2931)
* Reduce number of threads in postgres connection pool from 10 -> 2

* Try reducing to just 1 thread
2021-04-20 12:47:28 -05:00
Chris Stewart
19319494cd 2021 04 19 Cleanup after ourselves in postgres tests (#2921)
* Fix missing database shutdown on WalletAppConfig.stop()

* Cleanup BitcoindBlockPollingTest

* Cleanup ProcessBlockTest and all tests that use CachedBitcoind

* Cleanup BitcoindBackendTest

* Cache rescan thread pool rather than creating a new one everytime
2021-04-19 13:58:00 -05:00
Chris Stewart
a3954dbcae 2021 04 17 spendinfodb invariant (#2912)
* Add invariant to spendingInfoDb to that requires if the spendinginfodb is in a TxoState.spentStates, the SpendingInfoDb.spendingTxIdOpt is defined

* Remove unused SpendingInfoDAO.updateTxoState()

* Remove comment

* Make TxoState and explicit parameter for test methods. Most test methods depend on a sample utxo state, so make the caller of the method specify what state they want the txo to be in

* Fix github actions config

* Fix RoutesSpec test
2021-04-18 09:02:49 -05:00
Chris Stewart
bb379ecfcf Add docs for using CachedBitcoind (#2880)
* Add docs for using CachedBitcoind

* revert paths in testkit.md to be references to the appropriate scaladoc
2021-04-09 09:43:22 -05:00
benthecarman
d0629486ab Wallet Rebroadcast Transactions thread (#2711)
* Wallet Rebroadcast Logic

* Use wallet scheduler for address queue rather than it's own thread

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>

* Bump default

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2021-04-07 08:13:43 -05:00
benthecarman
4055de7690 Inital LND rpc with some tests (#2836)
* Inital LND rpc with some tests

* Fix ZMQ test

* Correct datadirs

* Most tests

* Fix eclair tests

* Make tests less flaky

* Have CI download lnd

* Respond to some review

* Fix ZMQ tests

* Cached bitcoind for lnd tests

* Fix fixtures, bump timeout

* InstanceFactory & ConfigFactory

* Fix compile issue for 2.12.12

* Move factories to core.api.commons
2021-04-05 04:52:56 -05:00
rorp
911fca5825 Schnorr js (#2805)
* Schnorr sigs for Scala.js

* fix build

* put BIP340 test vectors in a shared space

* remove teskit dependency, fix point edge cases

* fix build

* add unit tests for point addition

* scaladoc

* cleanup

* respond to the comments

* Fix usage of BitcoinSLogger

Co-authored-by: christewart <stewart.chris1234@gmail.com>
2021-03-22 12:56:40 -05:00
benthecarman
c6c4e83e9e Remove logging from testkit core (#2813)
* Remove logging from testkit core

* Fixup deps
2021-03-22 11:48:27 -05:00
benthecarman
07514e2348 Remove logging from core (#2810)
* Remove logging from core

* Remove slf4j from testkit core

* Remove logback from coretest
2021-03-21 19:11:55 -05:00
Chris Stewart
2287c6ced9 Implement caching of bitcoind in the walletTest,nodeTest, and partially bitcoindRpcTest project (#2792)
* Create CachedBitcoind, implement it in FundTransactionHandlingTest

* Add BaseWalletTest, extend it with BitcoinSWalletTest & BitcoinSWalletTestCachedBitcoind, add CachedBitcoinV19 and use it RescanHandlingTest

* Make ProcessBlockTest work with cached bitcoind

* Make trait for CachedBitcoindNewest for the newest version of bitcoind

* Make UTXOLifeCycleTest use cached bitcoind

* Add WalletBloom, WalletSyncTest to use cached bitcoinds

* Add WalletIntegrationTest

* Rework beforeAll() and afterAll() into the super trait like BaseWalletTest

* Add standlone BitcoindFixtures, use it in BitcoindBackendTest

* Use new BitcoindFixtures in BitcoindBlockPollingTest

* Introduce BaseNodeTest, start implementing the usage of cached bitcoinds in the nodeTest project

* Use cached bitcoind's with SpvNodeTest & SpvNodeWithWalletTest

* Fix bug on postgres with reusing database, upsert the genesis header rather than create it

* Get NeutrinoNode tests workign with cached bitcoinds

* Fix NeutrinoNodeWithWallet by destroying wallet state for Postgres

* Add teardown helper method for bitcoind

* Teardown chain project when using node fixtures since node is dependent upon the chain project.

* Turn off parallelExecution again

* Switch the parallelExecution flag to only be set on CI, so we can get better performance when running locally

* Start implementing BitcoindFixtures, use BitcoindFixturesCachedTriple on TestUtilRpcTest

* Fix compiler errors, begin implementing NodePair

* Refactor TestRpcUtilTest to use 2 bitcoinds rather than 2

* Reduce the number of bitcoinds that MultiWalletRpcTest needs from 3 -> 1

* Reduce number of bitcoinds used in WalletRpcTest from 3 -> 2

* Add some documentation

* Try to re-add parallelExecution

* Reduce the number of bitcoinds used in PsbtRpcTest from 3 -> 2

* Disable parallelExecution in Test again

* Make BitcoindV21RpcClientTest & BitcoindV20RpcClientTest reduce bitcoind usage from 2 -> 1

* Make BitcoindV19RpcClienttest reduce bitcoind usage from 2 -> 1

* Rework MempoolRpcTest to use fixtures, add BitcoindVersion to CachedBitcoindCollection

* Make sure clientAccumm has to be specified as a paramter now rather than filling in by default

* Begin parameterizing NodePair/NodeTriple to retain type information for the specific version of bitcoind that was used

* Don't implement version in super trait

* Fix docs

* Fix async issue in V21 test suite

* Append to vectors in CachedBitcoinCollection rather than replace

* Fix rebase issues

* Add scaladocs

* Fix BitcoindV18RpcClient address info test

* Implement fixtures in BitcoindV17RpcClientTest fixtures

* Cleanup v17 PsbtRpcTest

* Reduce bitcoind usage from 3 -> 1 in BitcoindV18RpcClientTest

* Remove abandon transaction test, this allows us to reduce the number of bitcoind's used in MempoolRpcTest from 3 -> 2

* Remove the requirement to inject  BitcoinSAsyncFixtureTest, add it in the test traits explicitly to make things easier. Also add explicit afterAll() method to tear down both the CachedBitcoind & BitcoinSAsyncFixtureTest

* Fix missing Await.result() in BitcoindRpcTest.afterAll()

* Rework MultiWalletRpcTest to use a NodePair

* Rework BlockchainRpcTest to use fixtures

* Rework Client start()/stop() methods. Now use an AtomicBoolean to indicate when a user has requested a client to start/stop rather than sending pings to bitcoind that can fail because the conneciton pool has been shutdown in test cases

* Try my luck with turning on parallelExecution in CI again

* Revert parallelExecution, now testes do not run in parallel on CI

* Only turn off parallelExecution for bitcoindRpcTest

* Adjust build to only have bitcoindRpcTest NOT in run parallel on mac, reduce number of blocks used in BitcoindRpcTestUtil.createNodeSequence

* Run less tests in the rpc test suite as that takes the longest, move them over to node/wallet/dlc test suite on mac osx CI

* Don't run eclair tests in parallel either

* Remove CachedBitcoind from BitcoinSWalletTest

* Fix async bug in test case

* Push to github to force re-run of CI

* Push to github to force re-run of CI

* Push to github to force re-run of CI
2021-03-19 06:37:53 -05:00
Chris Stewart
78448b277c Revert "Schnorr sigs for Scala.js (#2784)" (#2802)
This reverts commit 8e7bde0ed9.
2021-03-16 13:19:16 -05:00
rorp
8e7bde0ed9 Schnorr sigs for Scala.js (#2784) 2021-03-16 10:21:47 -07:00
benthecarman
9494eec1b8 Move blockhash to tx table from spending info table (#2744)
* Move blockhash to tx table from spending info table

* Add test, fix spending tx id col name

* Scaladocs, add test

* Add more unit test

* Make id not comparable in process tx test

* Fix tests

* attempt to fix

* Add mempool comment to scaladoc

* Deparallelize process inputs & outputs
2021-03-16 10:05:29 -05:00
Chris Stewart
49544fc7f3 Turn off parallelExecution and remove extra AsyncUtil test class (#2790) 2021-03-11 09:36:54 -06:00
Chris Stewart
5b4aac5178 Refactor starting second bitcoind in MempoolRpcTest, remove Thread.sleep (#2776)
* Refactor starting second bitcoind in MempoolRpcTest, remove Thread.sleep

* Add AkkaUtil.nonBlockingSleep(), replace usages of Thread.sleep() with it where possible

* Remove more Thread.sleep()

* Try bumping threadpool for scheduler

* Reduce scheduler threadpool size to 2

* Push to github to force re-run of CI
2021-03-10 09:36:34 -06:00
Chris Stewart
1959495cec Add testkit-core module (#2726)
* Refactor AsyncUtil to just use an execution context rather than an actor system

* Create testkit-core, start moving data structures over to testkit-core

* WIP, getting rid of the hard dependency on BitcoinSAsyncTest and akka

* Rework test traits to not be dependnent on akka, move BaseAsyncTest into testkitCore, add a test trait that just uses the default scala executionContext

* Get everything compiling

* Move logback-test.xml to testkit-core so it applies to coreTest,cryptoTest as well

* Cleanup rebase issues

* Fix Deps.scala

* Address ben's code review

* Segregate jvm settings and compiler settings

* Filter out -Xfatal-warning for scalajs for now since there is a ton of errors

* Move maintainer settings into CommonSettings.settings
2021-03-02 07:05:21 -06:00
rorp
c90f318fd7 Refactor crypto module to be compatible with Scala.js part 1 (#2719)
* Refactor crypto module to be compatible with Scala.js

* more changes

* some more changes

* abstract out Schnorr stuff

* abstract out adapter stuff

* cleanup

* some more cleanup

* fix build

* Removed references to ECPoint outside of .jvm scope

* remove references to ECPoint from the shared code

* cleanup

* remove cirlular dependencies

* more cleanup

* cleanup

* move SipHash to CryptoContext

* scaladoc

* scalafmt

Co-authored-by: nkohen <nadavk25@gmail.com>
2021-02-27 05:58:20 -06:00
Chris Stewart
7a068ac036 2021 02 25 async utils (#2725)
* Refactor AsyncUtil to just use an execution context rather than an actor system

* Create async util project, use it in various projects that depend on it
2021-02-25 13:26:38 -06:00
benthecarman
86566c575d Simplify oracle server RPC api (#2656) 2021-02-23 13:17:20 -06:00
Chris Stewart
be18b1baf2 Cache httpClient in bitcoind, rename Test.akkaHttp -> Test.akkaHttpTestkit (#2702) 2021-02-20 13:42:39 -06:00
Chris Stewart
392eb316f6 Add guard for the case when listFiles returns null (#2696) 2021-02-19 13:36:45 -06:00
Aris
85087b0f70 Refactoring chain (#2662) 2021-02-13 14:10:40 -06:00
Chris Stewart
8f958bf5d9 Update scalafmt (#2619) 2021-02-03 16:37:12 -06:00
Nadav Kohen
93f33d26b6 Update dlc before release (#2543)
* Pulled down DLC data structure work

* Responded to Ben's review

* Gave Vector types names

* Pulled down Chris' work
2021-02-02 08:14:24 -06:00
benthecarman
03ca6f617e Optimize filter sync and fetching filter heights (#2568) 2021-01-30 14:56:47 -06:00
Chris Stewart
756bfc4427 2021 01 27 conectionpool (#2578)
* Give connection pool example config

* Get hikari logging working on postgres

* fix 2.12.x

* Bump metrics core to a supported version

* Add hikari-logging and hikari-logging-interval configuration options to turn off/on hikari and schedule how often logs appear

* Turn off hikari logging in the db-commons/reference.conf and add it to appserver/reference.conf

* default the logging to off in scala code

* Make sure connection pool is enabled in db-commons, add default database configuration to website

* Address code review, fix things that didn't need to be changed
2021-01-28 09:44:03 -06:00
rorp
05e21b1200 Bump Eclair version (#2405)
* Bump Eclair version

* cleanup

* Bump JVM version for CI

* Update docs

* Fix docs
2021-01-22 10:47:19 -06:00
benthecarman
523f1098fc Implement OracleAttestmentV0TLV, save outcome to db (#2516)
* Implement oracleAttestmentV0TLV, save outcome to db

* Update with tlv changes

* Update tlv to not have 2 list sizes

* small change

* Update OracleEvent.verifyAttestations

* Use correct pub key

* Verify correct outcomes
2021-01-22 09:45:45 -06:00
rorp
fc48964721 Fix 'client1.getDaemon.datadir.exists() was true' (#2544) 2021-01-21 12:38:17 -06:00
benthecarman
811e96bb4f Fix BitcoindV21RpcClient testkit errors (#2533) 2021-01-18 16:19:00 -06:00
Chris Stewart
abc0a2b459 Refactor test case to be more idiomatic in hopes this kills CI failures (#2524) 2021-01-16 14:26:51 -06:00
Chris Stewart
6193af7709 Remove WalletSync.sync() -> WalletSync.syncFullBlocks() (#2522) 2021-01-15 17:05:11 -06:00
benthecarman
d69c60b8b2 Bitcoind v0.21.0 support (#2414)
* Bitcoind v0.21.0rc3 support

* Use v0.21.0 instead of Experimental in node tests

* Bump to rc5

* Use official release

* Fix rebase issues

* Remove DoNotDiscover tag

* Fix count for NeutrinoNodeTest
2021-01-14 14:01:44 -06:00
Chris Stewart
51e9796b39 2021 01 11 issue 2493 (#2503)
* Rename NodeUnitTest.confg -> NodeUnitTest.getFreshConfig()

* Rename CachedBitcoinSAppConfig.config -> CachedBitcoinSAppConfig.cachedConfig

* Make CachedChainAppConfig extend CachedBitcoinSAppConfig

* use cached configs in both wallet/chain project as well taking ben's suggestion to fix those
2021-01-11 16:45:10 -06:00
Chris Stewart
68af9760b3 Clean up fixture shutdown code a bit to try and see if this resolves … (#2498)
* Clean up fixture shutdown code a bit to try and see if this resolves issue on CI

* Rework fixtures so we aren't starting app configs in multiple places

* Fix scaladoc
2021-01-09 17:07:26 -06:00
Chris Stewart
98ace6f14e 2021 01 02 issue 2457 (#2461)
* WIP

* Get neutrino node with wallet 'receive information about received payments' working again

* Fix compile

* Remove initial sync logic from test case

* Remove sync logic in NeutrinoNodeWithWallet test cases

* Improve logging and rename a few things

* WIP2

* WIP3

* Get NeutrinoNodeWithWallet tests working

* Implement WalletSync, which allows you to sync a wallet from a arbitrary data source

* Get all tests passing again

* Use spv.appConfig in DataMessageHandlerTest rather than caching the config

* Modify cleanup to hopefully get CI passing

* Fix postgres tests by cleaning the table during the destroy phase of the test fixture. This is needed because the same postgres database is shared between tests in the same test suite

* Revert logback-test.xml

* Get sqlite/postgres tests passing pt 2

* syncHeight -> syncDescriptorOpt()

* Add case for genesis block hash in WalletSync

* Fix SpvNodeWithWallet test case to actually test spv functionality

* Remove nodeCallbacks parameters, callbacks should be registered on nodeAppConfig
2021-01-09 09:33:37 -06:00
Chris Stewart
6eff248f3d Add officially supported version of bitcoind by eclair, also add the ability to specify which version of bitcoind you are using for EclairRpcTestUtil.getBitcoindRpc (#2490) 2021-01-08 11:43:48 -06:00
benthecarman
6b441631f3 Limit bech32 addresses to segwitv0 (#2471)
* Limit bech32 addresses to segwitv0

* extend ScriptFactory, make check exhuastive
2021-01-07 08:13:31 -06:00
benthecarman
46c73970ff Remove callbacks param from DataMessageHandler & PeerMessageReceiver (#2476) 2021-01-06 13:23:47 -06:00
Chris Stewart
4e1a6c69f0 2020 12 18 enable lint options (#2454)
* Enable -Xlint:adapted-args,nullary-unit on our for compile scope

* Add Xlint:infer-any compiler option

* Fix build so test compiler options are actually enabled

* Enable more scalac linting options

* Add '-Xlint:eta-sam'

* Add 'Xlint:inaccessible,Xlint:missing-interpolator'
2021-01-03 08:01:26 -06:00
Chris Stewart
5411c89f22 Remove extra logback files (#2456)
* Remove extra logback files

* Adjust logging levels in testkit/logback-test.xml
2021-01-03 07:25:45 -06:00
Chris Stewart
b2560c4606 Rework BlockHeaderDAO.chainTips into two methods: BlockHeaderDAO.{get… (#2443)
* Rework BlockHeaderDAO.chainTips into two methods: BlockHeaderDAO.{getBestChainTips,getForkedChainTips}. getForkedChainTips is needed for reorg situations in the case a block header is received that builds off a stale tip

* Deduplicate BlockHeaderDAO.getBlockchains() that are subchains for the best chains
2021-01-01 09:04:29 -06:00
Nadav Kohen
493673c68b Outstanding DLC branch diff (#2432) 2020-12-23 18:13:57 -06:00
Ben Carman
67e7426260 Windows Secp Update & fix for parsing Windows paths (#2398)
* Windows Secp Update

* Add docs about errors

* Fix paths in config

* Bump timeout
2020-12-21 16:34:53 -06:00
Ben Carman
8912fcbc82 Fail broadcasting transaction when disconnected (#2336)
* Fail broadcasting transaction when disconnected

* Wait until disconnected

* Move test

* Move to separate file

* Attempt fix

* Make test never have a peer

* Improve reliablity
2020-12-17 13:28:32 -06:00
Nadav Kohen
f7fdca3e22 DLC Data Structures on Master Cleanup (#2375)
* Moved dlc data structures from commons to core

* Renamed DLC payout curve classes

* Split OutcomeValuePoint up into an ADT

* Added utility for computing Schnorr multiple-signature points

* Replaced tuples in RoundingIntervals with types

* Replaced tuples in DLCPayoutCurve with Indexed

* Fixed a compile bug
2020-12-17 09:09:57 -06:00
Ben Carman
1225584526 Multi Wallet support (#2345)
* Multi Wallet support

* Fix Oracle tests

* Fix some CI errors

* Increase KeyManagerAppConfig code coverage

* Add test for multiple seeds in seed folder

* Move things in reference.conf files

* Lower key manager coverage requirement

* Fix postgres test issues

* Create DbAppConfig

* Add some docs, fix hardcoded postgres db name

* Clarify db vs schema

* Add character types

* Fix chaindb in conf

* Always copy default seed

* Remove need for driverName config
2020-12-16 17:27:56 -06:00
Ben Carman
61d3a14c07 Don't allow fee unit gen to be 0 (#2346)
* Don't allow fee unit gen to be 0

* Fix other 0

* Fix always 0 fee unit gen
2020-12-12 07:34:26 -06:00
Nadav Kohen
b7ffea2525 Pulled down work from adaptor-dlc onto master (#2339)
* Pulled down work from adaptor-dlc onto master

* Reverted some accidental deletions

* Removed unused import

* Added scaladocs

* Responded to Ben's review

* Added some scaladocs and invariants

* Responded to chris' review

* Responded to more review

* Added some comments
2020-12-11 17:32:35 -06:00
Chris Stewart
7df02c4198 Start refactoring testkit to allow for specifying a different binary … (#2325)
* Start refactoring testkit to allow for specifying a different binary directory than the sbt default

* Fix BitcoindRpcTestUtil

* Add BitcoindRpcTestClient
2020-12-07 13:14:40 -06:00
Chris Stewart
b8e0847ff3 Remove hard coded bitcoind version eclair was depending on and switch to latest version of bitcoind (#2324) 2020-12-06 08:06:39 -06:00
Ben Carman
794f6e073e Fix TLV parsing for non-standard strings (#2312)
* Fix TLV parsing for non-standard strings

* Create function

* Fix oracle migrations

* Forced all TLV strings to be normalized implicitly

* Removed redundant normalization

* Fix oracle

* Bump migration test

* Fix 2.12.12 compile

* Use NetworkElement & StringFactory

Co-authored-by: nkohen <nadavk25@gmail.com>
2020-12-03 12:30:17 -06:00
Chris Stewart
80d3eaf6d7 Make BitcoinSAyncTest more thread safe and make sure we have all the … (#2292)
* Make BitcoinSAyncTest more thread safe and make sure we have all the expected test cases aggregated before we starting reducing over the results

* remove extra code

* Fix scala 2.12.x compile
2020-11-26 07:53:33 -06:00
Chris Stewart
66ec96b1f5 2020 11 13 issue 2258 (#2260)
* Introduce ChainHandlerCached which behaves like the old ChainHandler. Now Chainhandler.getBestBlockHeader() will read headers from the database

* Remove ChainHandler.blockchains field, now it's only available in ChainHandlerCached

* De-futurify ChainHandler.fromDatabase()

* Adjust logging

* Patch test case

* Use BlockHeaderDAO.chainTips when getting best header rather thean BlockHeaderDAO.getBlockchains(). Implement a helper method ChainHandler.toChainHandlerCached()

* Fix chain.md,wallet.md

* Make ChainHandler.getBestBlockHeader() consider time of header if chainwork is the same. Make test cases less strict on what header is the best header when both chainwork and time are the same on the eader

* Only execute callbacks on headers that are going to be created in the database, not all headers passed into ChainHandler.processHeadersWithChains()

* Turn up log level again

* Small optimizations, check if we have seen a header before before processing it in ChainHandler.processHeadersWithChains(). Fix FilterSyncMarker.toString(). Use ChainHandlerCached in Node

* Remove ChainHandlerCached in appServer, re-add it in Node.scala
2020-11-17 06:19:07 -06:00
Ben Carman
ec6afbb271 Use same config option for key manager projects (#2252)
* Use same config option for key manager projects

* Use BIP39KeyManager.initialize
2020-11-11 12:18:02 -06:00
Ben Carman
7ac9cd1525 Use New Oracle TLVs in DLCOracle (#2162)
* Update Oracle to use new TLVs

* Rename things to use new names, scaladoc, small clean ups

* Add descomposition tests, docs, sign numbers outside of range
2020-11-10 06:08:43 -06:00
Ben Carman
fcf0c842ad Update Oracle TLVs (#2185)
* Update Oracle TLVs

* Change string size to be BigSizeUInt

* Move nonces to oracle event tlv

* Introduced EventDescriptor tests and NumericEventDescriptor methods to query possible outcome data

* Added contains function

* Added tests for containsToPrecision

* Remove EventDescriptor.outcomes and other related fields

* remove unused formatNum method

* Address code review from Ben

* Move oracle pub key to announcement

Co-authored-by: nkohen <nadavk25@gmail.com>
Co-authored-by: christewart <stewart.chris1234@gmail.com>
2020-11-07 16:26:21 -06:00
Ben Carman
685d5b0273 Let wallet sign PSBTs (#2236)
* Let wallet sign PSBTs

* Add example to docs

* Add logs and test case
2020-11-07 09:25:59 -06:00
Ben Carman
47a38f88db Make aesPassword option for wallet config (#2217)
* Make aesPassword option for wallet config

* Add to docs

* Make AesPassword optional

* Small touchups

* Fix for oracle server

* Fix docs

* Increase code coverage
2020-11-06 07:00:18 -06:00
Chris Stewart
581d46c2c8 2020 11 02 cleanup (#2233)
* Cleanup a bunch of imports in test

* Use @nowarn annotation for testing things that are deprecated

* Fix base58 test

* Cleanup crypto-test

* Use scala collection compat dependency to get access to the @nowarn annotation on scala 2.12.x

* Rework more scala 2.13.x compile failures in chain-test and key-manager-test

* Optimize imports for entire project

* Fix nits
2020-11-03 11:07:04 -06:00
Ben Carman
9787e89403 Add Wallet State Descriptors (#2157)
* Add Wallet State Descriptors

* Sync blocks while offline w/ bitcoind backend

* Add to WalletDbManagement

* Update db management test

* Respond to review

* Rename function
2020-10-14 14:17:51 -05:00
Ben Carman
f628c265ae Add DbManagement tests for Oracle (#2186) 2020-10-14 08:04:00 -05:00
Chris Stewart
112ec90286 Refactor db configuration to use the key 'user' rather than 'username' inline with slick documentation (#2184) 2020-10-13 10:02:02 -05:00
Ben Carman
77e8cabf5b Oracle Announcement TLVs (#2149)
* Oracle Announcement TLV

* Add pubkey, event uri & descriptor

* TLVParentFactory, EnumEventDescriptorTLV

* Add trailing V0 to types

* Make names match spec pr

* Add range descriptor

* Add num outcomes to enum descriptor
2020-10-08 22:47:40 -05:00
Chris Stewart
3625c5bf88 2020 10 08 issue 2147 (#2153)
* Add test case for resolving typesafe config variables and making sure we honor system properties

* Remove println

* Move AppConfigTest test caess into the db-commons project
2020-10-08 10:37:13 -05:00
Ben Carman
7b53b02cfa Make ZMQ Listeners typed (#2144) 2020-10-08 01:21:50 -05:00
Ben Carman
52ba32769a Increase DLC Oracle test coverage (#2128) 2020-10-05 22:06:31 -05:00
Chris Stewart
ade2503702 2020 10 05 redo config (#2121)
* Use absolute paths for resolving configurations

Make log level OFF again

WIP: Move resolution of config into AppConfig.start(), things are totally broken

WIP: Make AppConfig.config private

Get all tests passing again but using absolute path in configuration

* Cherry-pick onto master

* Fix BitcoinSAppConfig configuration keys to use the full path

* Apply code review

* Fix log, move invariant for requiredConfirmations back into the lazy val

* Fix DlcOracleAppConfig to use absolute path
2020-10-05 18:00:30 -05:00
Chris Stewart
9b09b04230 Upgrade to scalac 2.13.3 (#2115) 2020-10-04 07:42:44 -05:00
Ben Carman
d57c059921 Add basic DLC Oracle (#2094)
* Add basic DLC Oracle

* Respond to review

* Respond to more review

* Add maturation time

* Add to testkit, tag hashes, better val names

* More clear vals, version tagged hashes

* Signing key clean up

* Add pubkey to db
2020-10-03 08:04:57 -05:00
Nadav Kohen
e71b664e1a Brought down ecdsa adaptor signatures implemented in scala from the dlc-crypto branch (#2034) 2020-10-02 10:43:59 -05:00
Ben Carman
ad6c2563bd Create Util functions for wallets with a bitcoind backend (#2076)
* Create Util functions for wallets with a bitcoind backend

* Move function, delete reduant function

* Fix imports
2020-09-30 08:50:49 -05:00
Ben Carman
920e30e5a2 Bitcoind v0.20 updated rpcs & tests (#2061)
* Bitcoind v0.20 updated rpcs & tests

* Test sorted multi
2020-09-28 14:25:28 -05:00
Ben Carman
e53317b409 Create initial BitcoindV20RpcClient (#2060) 2020-09-25 13:29:22 -05:00
Ben Carman
11e6b7d9eb Initial SigNet support (#2057) 2020-09-25 12:29:13 -05:00
Ben Carman
1b57cac8ae trivial: Set timezone and print it in logs (#2046) 2020-09-22 14:03:42 -05:00
Ben Carman
1e3aee55c3 Merge pull request #2019
* Refactor logging to only use grizzled slf4j

* Fix docs

* test travis config change
2020-09-19 14:09:12 -05:00
Chris Stewart
76b9577181 Introduce 'FilterSyncMarker' to ChainApi, make it clearier what exact… (#2003)
* Introduce 'FilterSyncMarker' to ChainApi, make it clearier what exactly the (Int,DoubleSha256Digest) tuple is returned from ChainApi.nextBlockHeaderRange()

* Fix doc

* Add scaladoc to FilterSyncMarker

* Rebase onto master, fix conflicts to use FilterSyncMarker
2020-09-11 13:48:40 -05:00
Chris Stewart
2980e3fc01 Resolve issues with reorgs and syncing filters (#1969)
* Pull over test cases, WIP

* Rework ChainHandler.nextBlockHeaderRange() to use hashes rather than heights to very integrity of header chain, add test case

* Modify test case to make sure we are generating the 'heaviest chain work' header as first in the tuple that is used in reorg test cases

* Refactor to helper method called 'findNextHeader()' to make things simpler

* Add scaladoc to ChainHandler.findNextHeader()

* WIP

* Fix bugs in ChainApi.nextBlockHeaderRange(), now return (startHeight,stopHash) rather than (stopHeight,stopHash)

* Get chain handler tests passing, remove genesis filter header/fitler from default chain project fixture, make it a specialized one to have genesis filter header/filter in the database. Use that specialized fixture in ChainHandlerTest for now

* FilterSync work without having the genesis filter and filter header inserted into the database

* fix bug in DataMessageHandler where we were using sendNextGetCompactFilterHeadersCommand rather than using peerMsgSender.sendGetCompactFilterHeadersMessage

* Refactor ChainHandler.findNextHeader() to centralize where chains are fetched

* Address ben's code review
2020-09-11 12:41:25 -05:00
Chris Stewart
39ef35b9ee Rename parameter names from 'duration' -> 'interval' just like our ActorSystem.schedule() names the parameter (#2005) 2020-09-11 11:10:59 -05:00
Chris Stewart
00480cc391 Use randomized fee rates for wallet tests (#1977)
Co-authored-by: Ben Carman <benthecarman@live.com>
2020-09-09 16:46:33 -05:00
rorp
c62be8b5f6 Update callbacks for LN support (#1938)
Update callbacks for LN support
2020-09-09 11:02:31 -07:00
Chris Stewart
4149271b77 Segregate mainnet chainhandler tests and regtest chainhandler tests (#1988) 2020-09-09 09:50:24 -05:00
Ben Carman
edc80bf13e Don't spend immature coinbases, fix tests (#1981) 2020-09-07 15:08:41 -05:00
Chris Stewart
98236835a5 Rename ChainApi.nextHeaderBatchRange -> ChainApi.nextBlockHeaderBatchrnage (#1957) 2020-09-02 13:19:20 -05:00
Chris Stewart
3ab280a12b Remove AppConfig.initialize() in favor of AppConfig.start() (#1907)
* Remove AppConfig.initialize() in favor of AppConfig.start()

* Fix docs
2020-08-26 16:24:38 -05:00
rorp
23685f124e Add an ability to one Postgres database for all sub-projects (#1897)
* Add an ability to one Postgres database for all sub-projects
2020-08-26 12:20:18 -07:00
Ben Carman
34ef07ef5a Create ADT for NodeType instead of booleans (#1901) 2020-08-26 07:28:35 -05:00
Ben Carman
f468fff7ea Move ChainApi to core (#1888)
* Move Chain Db Reps to core

* Move ChainApi to Core
2020-08-25 07:08:42 -05:00
Scala Steward
bef9c2648e Update scalafmt-core to 2.6.4 (#1686)
* Update scalafmt-core to 2.6.4

* Format everything

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-08-24 14:24:16 -05:00
Ben Carman
cfde581d0c Move all remaining wallet db representations (#1885) 2020-08-24 13:13:52 -05:00
Chris Stewart
172744f47c Db conf on pr 1877 (#1879)
* Extend StartStopAsync with BitcoinSAppConfig, create 'CachedAppConfig' test trait, clean up P2PClientTest

* Start cleaning up after ourselves in the chainTest test suite

* Call .stop() for appConfig's spun up in ChainAppConfigTest

* Database configuration defaults

* increase number of Postgres connections

* add more logging

* close connections pools in tests

* update afterAll()

* Fix conflict

Co-authored-by: rorp <rorp@users.noreply.github.com>
2020-08-24 12:55:04 -05:00
Chris Stewart
3e8e9b0e85 2020 08 22 chaintest cleanup (#1877)
* Clean up code for BroadcastDAO test

* Extend StartStopAsync with BitcoinSAppConfig, create 'CachedAppConfig' test trait, clean up P2PClientTest

* Make MerkleBuffersTest use CachedAppConfig

* Start cleaning up after ourselves in the chainTest test suite

* Cancel background runnable for generating blocks after test case, add nodeAppConfig.stop() to tests in NodeAppConfigTest

* Stop mainnet nodeAppConfig in test as per ben's code review

* Refactor ChainUnitTest.withChainFixture() to use makeDependentFixture()

* Call .stop() for appConfig's spun up in ChainAppConfigTest

* Less parallelization on destruction of chainAppConfig

* Revert parallelExecution/fork jvm in CommonSettings
2020-08-24 11:20:02 -05:00