Commit Graph

131 Commits

Author SHA1 Message Date
benthecarman
1d1af1d52e
Add generate helper function for bitcoind (#4852) 2022-10-19 14:22:30 -05:00
Chris Stewart
1a220a3937
2022 10 14 v19 testkit refactor (#4843)
* Break versioning on V19 of bitcoind rpc client in chain handler, use BitcoindRpcClient with V19BlockFilterRpc

* Make NodeTestWithCachedBitcoindNoP2pBlockFilters for test case with unsupported peers

* Fix docs

* Empty commit

* Empty commit

* Turn logging off again
2022-10-15 13:50:44 -05:00
Chris Stewart
718053668d
2022 10 07 node test fixes (#4819)
* Add CachedBitcoindPairNewest

* Try to get ReConnectionTest to use test fixtures

* Empty commit to run CI
2022-10-07 09:49:25 -05:00
Shreyansh
2cae3f803d
Fix infinite invalid header loop (#4667)
* fix infinite invalid header loop

* Adjust log levels to WARN

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-08-26 09:43:07 -05:00
Shreyansh
2c2e03b279
Header sync validation (#4456)
* add header sync validation

* fix docs, minor fixes

* Refactor to us InvalidBlockHeader ChainException, also refactor recovery to private helper method

* changes from comments

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-08-22 17:05:41 -05:00
Chris Stewart
d241e6f9e4
Remove merklebuffers (#4615) 2022-08-16 09:46:31 -05:00
Shreyansh
dea99457b5
fix mac node-wallet test failure (#4585) 2022-08-12 14:25:11 -05:00
rorp
0404cffe26
Embedded Pg v1.0.1 (#4567)
* Embedded Pg v1.0.1

* fix GitHub action evironment

* fix Windows CI tests

* test db-commons in different environments

* remove dbCommontTest from Windows and Mac Github actions

* cleanup

* configurable postgres startup wait

* Empty commit to run CI

* Bump default timeout to 60

* Empty commit

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2022-08-06 07:30:26 -05:00
Shreyansh
c4d358061a
Add P2PClientSupervisor (#4509)
* add P2PClientSupervisor

* changes from comments: made P2PClient Future

* empty commit to see if mac failures are consistent on ci

* changes from comments

* changes from comments
2022-08-04 10:06:04 -05:00
Chris Stewart
2cd8c80f67
Refactor so we don't create multiple WalletAppConfig in testkit (#4548)
* Refactor so we don't create multiple WalletAppConfig in testkit

Fix buildBip39PasswordConfig

* Remove bip39Password argument from dlcWalletTest fixtures

* Fix allowExternalDLCAddresses config in tests

* Remove unnecessary overrides of BitcoinSWalletTest.getFreshConfig, fix docs

* Add DLCExecutionBackendTest.afterAll()

* Fix nodeTest bip39Password usage

* Fix bug where we weren't using correct config inside of getFreshConfig

* Fix DLCExecutionBitcoindBackendtest.afterAll()

* Fix RescanDLCTest
2022-07-30 17:20:18 -05:00
Shreyansh
0b53c7be12
Change tests to use disconnectnode rpc instead of restarting bitcoind (#4468)
* change tests to use disconnectnode rpc instead of restarting bitcoind

* fix issue with querying for node uri when not initialized
2022-07-08 18:53:37 -05:00
Shreyansh
7c649d39db
Fix filter sync if headers received while syncing (#4463)
* fix filter sync if headers received while syncing

* remove unintended diffs
2022-07-08 13:44:15 -05:00
Shreyansh
111df25df0
Fix sync issues post #4408 (#4441)
* fix node test shared fixtures bug

The cached bitcoind fixtures were used and stopped in UnsyncedNeutrinoNodeTest which causes an error if NeutrinoNodeTest is run at the same time on high performant systems, which is why it escaped CI. Merges NeutrinoNodeTest and UnsyncedNeutrinoNodeTest

* fix possible issues with PeerMessageReceiverTest

This reverts commit 55e7cafee6c6fd8515650cf65d96f9d92a2a8812.

* fix filter sync issue when wallet creation time indicates already synced

* move switch peer test to NeutrinoNodeWithUnachedBitcoindTest
2022-07-04 08:42:45 -05:00
Shreyansh
42564bc810
Find and switch peers (#4408)
* add support to find and switch peers

* fix compile on 2.12

* allow empty config peers in regtest

* fix test

* minor fixes

changes timeouts, fix issue with ipv6 dns seeds, initDisconnect when disconnected bug, dbPeers order fix

* fix: not removing peers on initialization timeout

* fix: query again when previous failed

* fix: wrong condition for deferred peers

* restore log levels

* clean up

* add PeerStack to allow trying peers based on priority values

* fix migrations

* changes from comments

* use StartStopAsync

* changes from comments

* fix switch if peer down test

* changes from comments
2022-06-28 10:19:13 -05:00
Shreyansh
d8fc8e588f
Remove Spv code (#4356)
* change node tests to neutrino

* get node test working

* merge DataMessageHandlerTest and DataMessageHandlerNeutrinoNodeTest

* delete unused files

* remove commented out spv parts

* formatting

* delete spv node

* remove merkle callback for neutrino node

* remove spv node wallet callbacks

* formatting

* remove SpvWalletApi

* replace SpvTestConfig with NeutrinoTestConfig

* more replace SpvTestConfig with NeutrinoTestConfig

* minor fix

* fix tests
2022-05-30 07:57:31 -05:00
Chris Stewart
46229d712c
Rename clearAllUtxosAndAddresses() -> clearAllUtxos() (#4193)
* Rename clearAllUtxosAndAddresses() -> clearAllUtxos()

* Fix unit test

* Fix docs
2022-03-16 08:53:08 -05:00
Chris Stewart
c069b01e53
2022 02 24 rm prune addresses after rescan (#4130)
* Fix bug where we prune unused addreses on rescan. This doesn't let us discover funds for addresses sent that were not paid to until after the rescan was completed

* Add test case

* Fix neutrino node test case

* Some cleanup
2022-02-26 07:43:23 -06:00
Chris Stewart
b46574c0c4
2022 02 18 sync since creationtime pt2 (#4109)
* Get things compiling and tests passing

* Fix bug where we weren't using filter height correctly

* Fix docs

* Move creationTime into KeyManagerAppConfig

* Fix nodeTest/test test fixtures
2022-02-22 08:00:14 -06:00
Chris Stewart
98c5d816ac
2022 01 25 issue 4014 (#4015)
* Refactor testkit tests to use same nodeCallbacks that BitcoinSServerMain uses for neutrino

* Add test case

* Add logs for callbacks

* Cleanup

* Add test case for when funds are spent when we are offline

* Turn off logging again

* Cleanup more logs and comments
2022-01-26 13:16:15 -06:00
Chris Stewart
e802254a20
2022 01 24 rm appconfig varargs (#4011)
* Get things compiling

* Get tests passing

* Fix docs
2022-01-25 07:25:05 -06:00
Chris Stewart
52dcf51e82
Automatically download binaries if they are used in test suite (#4005)
* Remove download binary scripts from CI, try to make binary downloads contingent on tests being run in a project

* Cleanup docs to remove unecessary downloadBitcoind
2022-01-23 17:43:30 -06:00
Shreyansh
a58ef1cd02
Storing peers in database (#3773)
* rebase

* fix

* fix

* remove unwanted diffs

* fix

* formatting

* changes from comments

* Revert "changes from comments"

This reverts commit eb8a795718dc590802a19c7ce4cce4501bd2de1f.

* changes from comments

* add PeerManager

* add sha3-256

* add NetworkUtilTest

* formatting
2022-01-09 07:21:59 -06:00
rorp
41b96c4c7e
Add rescan field to walletinfo response (#3933) 2021-12-23 14:30:26 -08:00
rorp
4b07629d56
Add getmediantimepast RPC call (#3921)
* Add `getmediantimepast` RPC call

* update docs

* respond to the PR comments
2021-12-22 16:15:30 -06:00
rorp
a6898defe2
Support for Bitcoin Core v22 (#3834)
* Support for Bitcoin Core v22

* respond to the PR comments
2021-11-19 13:58:27 -06:00
benthecarman
92ab9faa45
Decide which coin selection solution to use based on waste metric (#3646)
* Decide which coin selection solution to use based on waste metric

* Fix test

* Fix NeutrinoNodeWithWalletTest
2021-10-19 09:47:28 -05:00
Shreyansh
99107b61ea
Add PeerDAO (#3738)
* add PeerDAO

* update DbManagementTest expected migration count
2021-10-10 07:50:11 -05:00
Chris Stewart
db46e35172
Fix case where we forget to correctly assign currentPeerMsgHandlerRecv (#3662)
* Fix case where we forget to correctly assign currentPeerMsgHandlerRecv when we disconnect

* Make handleNodeCommand() so we handle P2PClient.CloseCommand

* Enable DEBUG logging to see what is happening on CI

* Add PeerMessageReceiverTest

* Add InitializeDisconnectDone

* Bump timeout on condition
2021-09-14 06:38:38 -05:00
Chris Stewart
b6ef27eb3e
Move block generation and synchronization of th wallet before clearing utxo/addresses (#3623) 2021-08-29 14:39:13 -05:00
rorp
4ef1e6bcb6
CI fixes (#3597)
* CI fixes

* fix workflow names

* cleanup unused code

* fix ServerRunTest

* change number attempts to start Node
2021-08-25 15:07:27 -05:00
benthecarman
495ab62104
Tor in Testkit (#3484)
* Tor packaged for testkit

* Add CI row

* Small fixes

* More fixes

* Keep tor running

* Skip tor test for old version of bitcoind

* Respond to review

* Fix with master

* Bump tor start time to same as app config

* Use cached tor for DLCNodeTest

* Disable parallel execution for dlc node test with tor

* Fix DLCNodeTest for tor and clearnet

* Fix name of CI rows
2021-08-23 13:06:14 -05:00
Chris Stewart
ac8bdb120c
Implement PeerMessageReceiverState.InitializedDisconnect. This allows us to distinguish between disconnections we initalized and connections the peer intialized. This is needed for determining whether we should reconnect or not (#3583) 2021-08-21 17:04:50 -05:00
Chris Stewart
20575bcd68
Fix missing afterAll in ReConnectionTest (#3584) 2021-08-20 15:52:42 -05:00
rorp
54ce2ebeb2
P2P reconnect logic (#3572)
* P2P reconnect logic

* some fixes

* more changes

* fix DuplicateFilters exception in case of  disconnect in the middle of syncing filters

* more fixes

* Repurpose maxconnections test to be just a generic reconnection test

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2021-08-20 05:51:51 -05:00
Shreyansh
1426c31483
Multi peer: Sync with random peer (#3454)
* ensure random peer used for sync supports compact filters

* test file

* fix

* refactor

* rerun ci

* rerun ci

* changes from comments

* fix

* fix tests

* changes from comments

* refactor

* remove unintended diffs
2021-07-31 07:54:31 -05:00
Shreyansh
85975b8b07
complete handshake with peers (#3446)
fix
2021-07-25 11:11:01 -05:00
Shruthii RG
e2c2798e60
Implemented the ability to fetch unconfirmed transactions (#3429)
* relay flag in version message made configurable

* modified test classes as well

* Update configuration.md

* reverted few files and cleaned up format

* fixed mistake

* enabled neutrino node to receive transactions as a callback

* added txid to log message

* changed txId to txIdBE

* unit test added for checking if we're getting transaction as a callback
2021-07-23 13:01:07 -05:00
Shreyansh
4be2b2109b
[WIP] Adding multi-peer support (#3401)
* establish conn with multiple peers

squash 1

squash 2

squash 3

* add tests

* change log msg level

* minor changes

* fix

* formatting

* fix docs

* commit to rerun ci

* changes from comments

* fix bug

* rerun ci
2021-07-21 15:18:45 -05:00
rorp
8b663d91b6
Tor support for P2P (#3311)
* Tor support for P2P

* fix stuck connections

* unit tests

* docs

* update docs

* respond to the PR comments

* fix docs
2021-07-13 14:27:24 -05:00
Chris Stewart
77cd94ac41 2021 04 27 wallet fixtures config (#2980)
* Refactor MultiWalletTest to actually shutdown the walletAppConfigs

* Remove 'implicit' modifier from getFreshWalletConfig. This is a problem as it can be used implicitly to bind resources (threads, files etc) that need to be cleaned up. Since it is implicitly passed, it is very hard to remember to clean up these resources

* Cleanup TrezorAddressTest

* Remove unecessary BitcoinSWalletTest.afterAll()

* Fix docs

* Removal of datadir race condition

* Cleanup nodeTest to shutdown the transitive chainAppConfig required by NodeAppConfig

* remove experimental tag on NeutrinoNodeTest

* Remove extra line

* Cleanup after BitcoindChainhandlerViaZmqTest

* Push to github to force re-run of CI 2
2021-04-29 06:30:58 -05:00
benthecarman
e3017fd17d Peer Message Receiver Refactor (#2938) 2021-04-27 15:54:44 -05:00
Chris Stewart
16538980e3 Fix missing super.stop() to shutdown DbAppConfig db connection pool (#2943)
* Fix missing super.stop() to shutdown DbAppConfig db connection pool inside of all DbAppConfig subclasses that override stop()

* Fix import

* Remove implicit modifier to NodeUnitTest.getFreshConfig(), make sure we are shutting down chainAppConfig inside NodeUnitTest

* Remove chainTest subclasses implementing ActorSystem, make sure we do correct cleanup in ChainUnitTest

* Decouple CachedChainAppConfig & CachedAppConfig. When using only one sub module in the CachedAppConfig, teardown may fail as the other modules might not be started such as using chainAppConfig in chainTest, BitcoinSAppConfig.stop() requires us to call nodAppConfig.stop() first, which isn't started.

* Push to github to force re-run of CI 4
2021-04-23 08:21:56 -05:00
Chris Stewart
2287c6ced9 Implement caching of bitcoind in the walletTest,nodeTest, and partially bitcoindRpcTest project (#2792)
* Create CachedBitcoind, implement it in FundTransactionHandlingTest

* Add BaseWalletTest, extend it with BitcoinSWalletTest & BitcoinSWalletTestCachedBitcoind, add CachedBitcoinV19 and use it RescanHandlingTest

* Make ProcessBlockTest work with cached bitcoind

* Make trait for CachedBitcoindNewest for the newest version of bitcoind

* Make UTXOLifeCycleTest use cached bitcoind

* Add WalletBloom, WalletSyncTest to use cached bitcoinds

* Add WalletIntegrationTest

* Rework beforeAll() and afterAll() into the super trait like BaseWalletTest

* Add standlone BitcoindFixtures, use it in BitcoindBackendTest

* Use new BitcoindFixtures in BitcoindBlockPollingTest

* Introduce BaseNodeTest, start implementing the usage of cached bitcoinds in the nodeTest project

* Use cached bitcoind's with SpvNodeTest & SpvNodeWithWalletTest

* Fix bug on postgres with reusing database, upsert the genesis header rather than create it

* Get NeutrinoNode tests workign with cached bitcoinds

* Fix NeutrinoNodeWithWallet by destroying wallet state for Postgres

* Add teardown helper method for bitcoind

* Teardown chain project when using node fixtures since node is dependent upon the chain project.

* Turn off parallelExecution again

* Switch the parallelExecution flag to only be set on CI, so we can get better performance when running locally

* Start implementing BitcoindFixtures, use BitcoindFixturesCachedTriple on TestUtilRpcTest

* Fix compiler errors, begin implementing NodePair

* Refactor TestRpcUtilTest to use 2 bitcoinds rather than 2

* Reduce the number of bitcoinds that MultiWalletRpcTest needs from 3 -> 1

* Reduce number of bitcoinds used in WalletRpcTest from 3 -> 2

* Add some documentation

* Try to re-add parallelExecution

* Reduce the number of bitcoinds used in PsbtRpcTest from 3 -> 2

* Disable parallelExecution in Test again

* Make BitcoindV21RpcClientTest & BitcoindV20RpcClientTest reduce bitcoind usage from 2 -> 1

* Make BitcoindV19RpcClienttest reduce bitcoind usage from 2 -> 1

* Rework MempoolRpcTest to use fixtures, add BitcoindVersion to CachedBitcoindCollection

* Make sure clientAccumm has to be specified as a paramter now rather than filling in by default

* Begin parameterizing NodePair/NodeTriple to retain type information for the specific version of bitcoind that was used

* Don't implement version in super trait

* Fix docs

* Fix async issue in V21 test suite

* Append to vectors in CachedBitcoinCollection rather than replace

* Fix rebase issues

* Add scaladocs

* Fix BitcoindV18RpcClient address info test

* Implement fixtures in BitcoindV17RpcClientTest fixtures

* Cleanup v17 PsbtRpcTest

* Reduce bitcoind usage from 3 -> 1 in BitcoindV18RpcClientTest

* Remove abandon transaction test, this allows us to reduce the number of bitcoind's used in MempoolRpcTest from 3 -> 2

* Remove the requirement to inject  BitcoinSAsyncFixtureTest, add it in the test traits explicitly to make things easier. Also add explicit afterAll() method to tear down both the CachedBitcoind & BitcoinSAsyncFixtureTest

* Fix missing Await.result() in BitcoindRpcTest.afterAll()

* Rework MultiWalletRpcTest to use a NodePair

* Rework BlockchainRpcTest to use fixtures

* Rework Client start()/stop() methods. Now use an AtomicBoolean to indicate when a user has requested a client to start/stop rather than sending pings to bitcoind that can fail because the conneciton pool has been shutdown in test cases

* Try my luck with turning on parallelExecution in CI again

* Revert parallelExecution, now testes do not run in parallel on CI

* Only turn off parallelExecution for bitcoindRpcTest

* Adjust build to only have bitcoindRpcTest NOT in run parallel on mac, reduce number of blocks used in BitcoindRpcTestUtil.createNodeSequence

* Run less tests in the rpc test suite as that takes the longest, move them over to node/wallet/dlc test suite on mac osx CI

* Don't run eclair tests in parallel either

* Remove CachedBitcoind from BitcoinSWalletTest

* Fix async bug in test case

* Push to github to force re-run of CI

* Push to github to force re-run of CI

* Push to github to force re-run of CI
2021-03-19 06:37:53 -05:00
Chris Stewart
1959495cec Add testkit-core module (#2726)
* Refactor AsyncUtil to just use an execution context rather than an actor system

* Create testkit-core, start moving data structures over to testkit-core

* WIP, getting rid of the hard dependency on BitcoinSAsyncTest and akka

* Rework test traits to not be dependnent on akka, move BaseAsyncTest into testkitCore, add a test trait that just uses the default scala executionContext

* Get everything compiling

* Move logback-test.xml to testkit-core so it applies to coreTest,cryptoTest as well

* Cleanup rebase issues

* Fix Deps.scala

* Address ben's code review

* Segregate jvm settings and compiler settings

* Filter out -Xfatal-warning for scalajs for now since there is a ton of errors

* Move maintainer settings into CommonSettings.settings
2021-03-02 07:05:21 -06:00
Chris Stewart
7a068ac036 2021 02 25 async utils (#2725)
* Refactor AsyncUtil to just use an execution context rather than an actor system

* Create async util project, use it in various projects that depend on it
2021-02-25 13:26:38 -06:00
Aris
85087b0f70 Refactoring chain (#2662) 2021-02-13 14:10:40 -06:00
Chris Stewart
8f958bf5d9 Update scalafmt (#2619) 2021-02-03 16:37:12 -06:00
rorp
05e21b1200 Bump Eclair version (#2405)
* Bump Eclair version

* cleanup

* Bump JVM version for CI

* Update docs

* Fix docs
2021-01-22 10:47:19 -06:00
benthecarman
fcc258c5b2 Add unit test to make sure DataMessageHandler exception doesn't stop node (#2536) 2021-01-18 16:18:23 -06:00
Chris Stewart
abc0a2b459 Refactor test case to be more idiomatic in hopes this kills CI failures (#2524) 2021-01-16 14:26:51 -06:00