Commit graph

470 commits

Author SHA1 Message Date
Chris Stewart
16538980e3 Fix missing super.stop() to shutdown DbAppConfig db connection pool (#2943)
* Fix missing super.stop() to shutdown DbAppConfig db connection pool inside of all DbAppConfig subclasses that override stop()

* Fix import

* Remove implicit modifier to NodeUnitTest.getFreshConfig(), make sure we are shutting down chainAppConfig inside NodeUnitTest

* Remove chainTest subclasses implementing ActorSystem, make sure we do correct cleanup in ChainUnitTest

* Decouple CachedChainAppConfig & CachedAppConfig. When using only one sub module in the CachedAppConfig, teardown may fail as the other modules might not be started such as using chainAppConfig in chainTest, BitcoinSAppConfig.stop() requires us to call nodAppConfig.stop() first, which isn't started.

* Push to github to force re-run of CI 4
2021-04-23 08:21:56 -05:00
Chris Stewart
c95c0f9706 Move wallet scheduler into WalletAppConfig (#2933)
* Move wallet scheuler into WalletAppConfig, use scheduler.shutdownNow() so we actually shutdown the scheduler rather than wait for all queued tasks to finish executing before shutting down

* Fix typo

* Move rescan thread pool in to walletAppConfig, shut it down on wallet shutdown

* Fix compile
2021-04-21 11:06:29 -05:00
Chris Stewart
de5f7fc7f9 Reduce number of threads in postgres connection pool for tests (#2931)
* Reduce number of threads in postgres connection pool from 10 -> 2

* Try reducing to just 1 thread
2021-04-20 12:47:28 -05:00
Chris Stewart
19319494cd 2021 04 19 Cleanup after ourselves in postgres tests (#2921)
* Fix missing database shutdown on WalletAppConfig.stop()

* Cleanup BitcoindBlockPollingTest

* Cleanup ProcessBlockTest and all tests that use CachedBitcoind

* Cleanup BitcoindBackendTest

* Cache rescan thread pool rather than creating a new one everytime
2021-04-19 13:58:00 -05:00
Chris Stewart
a3954dbcae 2021 04 17 spendinfodb invariant (#2912)
* Add invariant to spendingInfoDb to that requires if the spendinginfodb is in a TxoState.spentStates, the SpendingInfoDb.spendingTxIdOpt is defined

* Remove unused SpendingInfoDAO.updateTxoState()

* Remove comment

* Make TxoState and explicit parameter for test methods. Most test methods depend on a sample utxo state, so make the caller of the method specify what state they want the txo to be in

* Fix github actions config

* Fix RoutesSpec test
2021-04-18 09:02:49 -05:00
Chris Stewart
bb379ecfcf Add docs for using CachedBitcoind (#2880)
* Add docs for using CachedBitcoind

* revert paths in testkit.md to be references to the appropriate scaladoc
2021-04-09 09:43:22 -05:00
benthecarman
d0629486ab Wallet Rebroadcast Transactions thread (#2711)
* Wallet Rebroadcast Logic

* Use wallet scheduler for address queue rather than it's own thread

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>

* Bump default

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2021-04-07 08:13:43 -05:00
benthecarman
4055de7690 Inital LND rpc with some tests (#2836)
* Inital LND rpc with some tests

* Fix ZMQ test

* Correct datadirs

* Most tests

* Fix eclair tests

* Make tests less flaky

* Have CI download lnd

* Respond to some review

* Fix ZMQ tests

* Cached bitcoind for lnd tests

* Fix fixtures, bump timeout

* InstanceFactory & ConfigFactory

* Fix compile issue for 2.12.12

* Move factories to core.api.commons
2021-04-05 04:52:56 -05:00
rorp
911fca5825 Schnorr js (#2805)
* Schnorr sigs for Scala.js

* fix build

* put BIP340 test vectors in a shared space

* remove teskit dependency, fix point edge cases

* fix build

* add unit tests for point addition

* scaladoc

* cleanup

* respond to the comments

* Fix usage of BitcoinSLogger

Co-authored-by: christewart <stewart.chris1234@gmail.com>
2021-03-22 12:56:40 -05:00
benthecarman
c6c4e83e9e Remove logging from testkit core (#2813)
* Remove logging from testkit core

* Fixup deps
2021-03-22 11:48:27 -05:00
benthecarman
07514e2348 Remove logging from core (#2810)
* Remove logging from core

* Remove slf4j from testkit core

* Remove logback from coretest
2021-03-21 19:11:55 -05:00
Chris Stewart
2287c6ced9 Implement caching of bitcoind in the walletTest,nodeTest, and partially bitcoindRpcTest project (#2792)
* Create CachedBitcoind, implement it in FundTransactionHandlingTest

* Add BaseWalletTest, extend it with BitcoinSWalletTest & BitcoinSWalletTestCachedBitcoind, add CachedBitcoinV19 and use it RescanHandlingTest

* Make ProcessBlockTest work with cached bitcoind

* Make trait for CachedBitcoindNewest for the newest version of bitcoind

* Make UTXOLifeCycleTest use cached bitcoind

* Add WalletBloom, WalletSyncTest to use cached bitcoinds

* Add WalletIntegrationTest

* Rework beforeAll() and afterAll() into the super trait like BaseWalletTest

* Add standlone BitcoindFixtures, use it in BitcoindBackendTest

* Use new BitcoindFixtures in BitcoindBlockPollingTest

* Introduce BaseNodeTest, start implementing the usage of cached bitcoinds in the nodeTest project

* Use cached bitcoind's with SpvNodeTest & SpvNodeWithWalletTest

* Fix bug on postgres with reusing database, upsert the genesis header rather than create it

* Get NeutrinoNode tests workign with cached bitcoinds

* Fix NeutrinoNodeWithWallet by destroying wallet state for Postgres

* Add teardown helper method for bitcoind

* Teardown chain project when using node fixtures since node is dependent upon the chain project.

* Turn off parallelExecution again

* Switch the parallelExecution flag to only be set on CI, so we can get better performance when running locally

* Start implementing BitcoindFixtures, use BitcoindFixturesCachedTriple on TestUtilRpcTest

* Fix compiler errors, begin implementing NodePair

* Refactor TestRpcUtilTest to use 2 bitcoinds rather than 2

* Reduce the number of bitcoinds that MultiWalletRpcTest needs from 3 -> 1

* Reduce number of bitcoinds used in WalletRpcTest from 3 -> 2

* Add some documentation

* Try to re-add parallelExecution

* Reduce the number of bitcoinds used in PsbtRpcTest from 3 -> 2

* Disable parallelExecution in Test again

* Make BitcoindV21RpcClientTest & BitcoindV20RpcClientTest reduce bitcoind usage from 2 -> 1

* Make BitcoindV19RpcClienttest reduce bitcoind usage from 2 -> 1

* Rework MempoolRpcTest to use fixtures, add BitcoindVersion to CachedBitcoindCollection

* Make sure clientAccumm has to be specified as a paramter now rather than filling in by default

* Begin parameterizing NodePair/NodeTriple to retain type information for the specific version of bitcoind that was used

* Don't implement version in super trait

* Fix docs

* Fix async issue in V21 test suite

* Append to vectors in CachedBitcoinCollection rather than replace

* Fix rebase issues

* Add scaladocs

* Fix BitcoindV18RpcClient address info test

* Implement fixtures in BitcoindV17RpcClientTest fixtures

* Cleanup v17 PsbtRpcTest

* Reduce bitcoind usage from 3 -> 1 in BitcoindV18RpcClientTest

* Remove abandon transaction test, this allows us to reduce the number of bitcoind's used in MempoolRpcTest from 3 -> 2

* Remove the requirement to inject  BitcoinSAsyncFixtureTest, add it in the test traits explicitly to make things easier. Also add explicit afterAll() method to tear down both the CachedBitcoind & BitcoinSAsyncFixtureTest

* Fix missing Await.result() in BitcoindRpcTest.afterAll()

* Rework MultiWalletRpcTest to use a NodePair

* Rework BlockchainRpcTest to use fixtures

* Rework Client start()/stop() methods. Now use an AtomicBoolean to indicate when a user has requested a client to start/stop rather than sending pings to bitcoind that can fail because the conneciton pool has been shutdown in test cases

* Try my luck with turning on parallelExecution in CI again

* Revert parallelExecution, now testes do not run in parallel on CI

* Only turn off parallelExecution for bitcoindRpcTest

* Adjust build to only have bitcoindRpcTest NOT in run parallel on mac, reduce number of blocks used in BitcoindRpcTestUtil.createNodeSequence

* Run less tests in the rpc test suite as that takes the longest, move them over to node/wallet/dlc test suite on mac osx CI

* Don't run eclair tests in parallel either

* Remove CachedBitcoind from BitcoinSWalletTest

* Fix async bug in test case

* Push to github to force re-run of CI

* Push to github to force re-run of CI

* Push to github to force re-run of CI
2021-03-19 06:37:53 -05:00
Chris Stewart
78448b277c Revert "Schnorr sigs for Scala.js (#2784)" (#2802)
This reverts commit 8e7bde0ed9.
2021-03-16 13:19:16 -05:00
rorp
8e7bde0ed9 Schnorr sigs for Scala.js (#2784) 2021-03-16 10:21:47 -07:00
benthecarman
9494eec1b8 Move blockhash to tx table from spending info table (#2744)
* Move blockhash to tx table from spending info table

* Add test, fix spending tx id col name

* Scaladocs, add test

* Add more unit test

* Make id not comparable in process tx test

* Fix tests

* attempt to fix

* Add mempool comment to scaladoc

* Deparallelize process inputs & outputs
2021-03-16 10:05:29 -05:00
Chris Stewart
49544fc7f3 Turn off parallelExecution and remove extra AsyncUtil test class (#2790) 2021-03-11 09:36:54 -06:00
Chris Stewart
5b4aac5178 Refactor starting second bitcoind in MempoolRpcTest, remove Thread.sleep (#2776)
* Refactor starting second bitcoind in MempoolRpcTest, remove Thread.sleep

* Add AkkaUtil.nonBlockingSleep(), replace usages of Thread.sleep() with it where possible

* Remove more Thread.sleep()

* Try bumping threadpool for scheduler

* Reduce scheduler threadpool size to 2

* Push to github to force re-run of CI
2021-03-10 09:36:34 -06:00
Chris Stewart
1959495cec Add testkit-core module (#2726)
* Refactor AsyncUtil to just use an execution context rather than an actor system

* Create testkit-core, start moving data structures over to testkit-core

* WIP, getting rid of the hard dependency on BitcoinSAsyncTest and akka

* Rework test traits to not be dependnent on akka, move BaseAsyncTest into testkitCore, add a test trait that just uses the default scala executionContext

* Get everything compiling

* Move logback-test.xml to testkit-core so it applies to coreTest,cryptoTest as well

* Cleanup rebase issues

* Fix Deps.scala

* Address ben's code review

* Segregate jvm settings and compiler settings

* Filter out -Xfatal-warning for scalajs for now since there is a ton of errors

* Move maintainer settings into CommonSettings.settings
2021-03-02 07:05:21 -06:00
rorp
c90f318fd7 Refactor crypto module to be compatible with Scala.js part 1 (#2719)
* Refactor crypto module to be compatible with Scala.js

* more changes

* some more changes

* abstract out Schnorr stuff

* abstract out adapter stuff

* cleanup

* some more cleanup

* fix build

* Removed references to ECPoint outside of .jvm scope

* remove references to ECPoint from the shared code

* cleanup

* remove cirlular dependencies

* more cleanup

* cleanup

* move SipHash to CryptoContext

* scaladoc

* scalafmt

Co-authored-by: nkohen <nadavk25@gmail.com>
2021-02-27 05:58:20 -06:00
Chris Stewart
7a068ac036 2021 02 25 async utils (#2725)
* Refactor AsyncUtil to just use an execution context rather than an actor system

* Create async util project, use it in various projects that depend on it
2021-02-25 13:26:38 -06:00
benthecarman
86566c575d Simplify oracle server RPC api (#2656) 2021-02-23 13:17:20 -06:00
Chris Stewart
be18b1baf2 Cache httpClient in bitcoind, rename Test.akkaHttp -> Test.akkaHttpTestkit (#2702) 2021-02-20 13:42:39 -06:00
Chris Stewart
392eb316f6 Add guard for the case when listFiles returns null (#2696) 2021-02-19 13:36:45 -06:00
Aris
85087b0f70 Refactoring chain (#2662) 2021-02-13 14:10:40 -06:00
Chris Stewart
8f958bf5d9 Update scalafmt (#2619) 2021-02-03 16:37:12 -06:00
Nadav Kohen
93f33d26b6 Update dlc before release (#2543)
* Pulled down DLC data structure work

* Responded to Ben's review

* Gave Vector types names

* Pulled down Chris' work
2021-02-02 08:14:24 -06:00
benthecarman
03ca6f617e Optimize filter sync and fetching filter heights (#2568) 2021-01-30 14:56:47 -06:00
Chris Stewart
756bfc4427 2021 01 27 conectionpool (#2578)
* Give connection pool example config

* Get hikari logging working on postgres

* fix 2.12.x

* Bump metrics core to a supported version

* Add hikari-logging and hikari-logging-interval configuration options to turn off/on hikari and schedule how often logs appear

* Turn off hikari logging in the db-commons/reference.conf and add it to appserver/reference.conf

* default the logging to off in scala code

* Make sure connection pool is enabled in db-commons, add default database configuration to website

* Address code review, fix things that didn't need to be changed
2021-01-28 09:44:03 -06:00
rorp
05e21b1200 Bump Eclair version (#2405)
* Bump Eclair version

* cleanup

* Bump JVM version for CI

* Update docs

* Fix docs
2021-01-22 10:47:19 -06:00
benthecarman
523f1098fc Implement OracleAttestmentV0TLV, save outcome to db (#2516)
* Implement oracleAttestmentV0TLV, save outcome to db

* Update with tlv changes

* Update tlv to not have 2 list sizes

* small change

* Update OracleEvent.verifyAttestations

* Use correct pub key

* Verify correct outcomes
2021-01-22 09:45:45 -06:00
rorp
fc48964721 Fix 'client1.getDaemon.datadir.exists() was true' (#2544) 2021-01-21 12:38:17 -06:00
benthecarman
811e96bb4f Fix BitcoindV21RpcClient testkit errors (#2533) 2021-01-18 16:19:00 -06:00
Chris Stewart
abc0a2b459 Refactor test case to be more idiomatic in hopes this kills CI failures (#2524) 2021-01-16 14:26:51 -06:00
Chris Stewart
6193af7709 Remove WalletSync.sync() -> WalletSync.syncFullBlocks() (#2522) 2021-01-15 17:05:11 -06:00
benthecarman
d69c60b8b2 Bitcoind v0.21.0 support (#2414)
* Bitcoind v0.21.0rc3 support

* Use v0.21.0 instead of Experimental in node tests

* Bump to rc5

* Use official release

* Fix rebase issues

* Remove DoNotDiscover tag

* Fix count for NeutrinoNodeTest
2021-01-14 14:01:44 -06:00
Chris Stewart
51e9796b39 2021 01 11 issue 2493 (#2503)
* Rename NodeUnitTest.confg -> NodeUnitTest.getFreshConfig()

* Rename CachedBitcoinSAppConfig.config -> CachedBitcoinSAppConfig.cachedConfig

* Make CachedChainAppConfig extend CachedBitcoinSAppConfig

* use cached configs in both wallet/chain project as well taking ben's suggestion to fix those
2021-01-11 16:45:10 -06:00
Chris Stewart
68af9760b3 Clean up fixture shutdown code a bit to try and see if this resolves … (#2498)
* Clean up fixture shutdown code a bit to try and see if this resolves issue on CI

* Rework fixtures so we aren't starting app configs in multiple places

* Fix scaladoc
2021-01-09 17:07:26 -06:00
Chris Stewart
98ace6f14e 2021 01 02 issue 2457 (#2461)
* WIP

* Get neutrino node with wallet 'receive information about received payments' working again

* Fix compile

* Remove initial sync logic from test case

* Remove sync logic in NeutrinoNodeWithWallet test cases

* Improve logging and rename a few things

* WIP2

* WIP3

* Get NeutrinoNodeWithWallet tests working

* Implement WalletSync, which allows you to sync a wallet from a arbitrary data source

* Get all tests passing again

* Use spv.appConfig in DataMessageHandlerTest rather than caching the config

* Modify cleanup to hopefully get CI passing

* Fix postgres tests by cleaning the table during the destroy phase of the test fixture. This is needed because the same postgres database is shared between tests in the same test suite

* Revert logback-test.xml

* Get sqlite/postgres tests passing pt 2

* syncHeight -> syncDescriptorOpt()

* Add case for genesis block hash in WalletSync

* Fix SpvNodeWithWallet test case to actually test spv functionality

* Remove nodeCallbacks parameters, callbacks should be registered on nodeAppConfig
2021-01-09 09:33:37 -06:00
Chris Stewart
6eff248f3d Add officially supported version of bitcoind by eclair, also add the ability to specify which version of bitcoind you are using for EclairRpcTestUtil.getBitcoindRpc (#2490) 2021-01-08 11:43:48 -06:00
benthecarman
6b441631f3 Limit bech32 addresses to segwitv0 (#2471)
* Limit bech32 addresses to segwitv0

* extend ScriptFactory, make check exhuastive
2021-01-07 08:13:31 -06:00
benthecarman
46c73970ff Remove callbacks param from DataMessageHandler & PeerMessageReceiver (#2476) 2021-01-06 13:23:47 -06:00
Chris Stewart
4e1a6c69f0 2020 12 18 enable lint options (#2454)
* Enable -Xlint:adapted-args,nullary-unit on our for compile scope

* Add Xlint:infer-any compiler option

* Fix build so test compiler options are actually enabled

* Enable more scalac linting options

* Add '-Xlint:eta-sam'

* Add 'Xlint:inaccessible,Xlint:missing-interpolator'
2021-01-03 08:01:26 -06:00
Chris Stewart
5411c89f22 Remove extra logback files (#2456)
* Remove extra logback files

* Adjust logging levels in testkit/logback-test.xml
2021-01-03 07:25:45 -06:00
Chris Stewart
b2560c4606 Rework BlockHeaderDAO.chainTips into two methods: BlockHeaderDAO.{get… (#2443)
* Rework BlockHeaderDAO.chainTips into two methods: BlockHeaderDAO.{getBestChainTips,getForkedChainTips}. getForkedChainTips is needed for reorg situations in the case a block header is received that builds off a stale tip

* Deduplicate BlockHeaderDAO.getBlockchains() that are subchains for the best chains
2021-01-01 09:04:29 -06:00
Nadav Kohen
493673c68b Outstanding DLC branch diff (#2432) 2020-12-23 18:13:57 -06:00
Ben Carman
67e7426260 Windows Secp Update & fix for parsing Windows paths (#2398)
* Windows Secp Update

* Add docs about errors

* Fix paths in config

* Bump timeout
2020-12-21 16:34:53 -06:00
Ben Carman
8912fcbc82 Fail broadcasting transaction when disconnected (#2336)
* Fail broadcasting transaction when disconnected

* Wait until disconnected

* Move test

* Move to separate file

* Attempt fix

* Make test never have a peer

* Improve reliablity
2020-12-17 13:28:32 -06:00
Nadav Kohen
f7fdca3e22 DLC Data Structures on Master Cleanup (#2375)
* Moved dlc data structures from commons to core

* Renamed DLC payout curve classes

* Split OutcomeValuePoint up into an ADT

* Added utility for computing Schnorr multiple-signature points

* Replaced tuples in RoundingIntervals with types

* Replaced tuples in DLCPayoutCurve with Indexed

* Fixed a compile bug
2020-12-17 09:09:57 -06:00
Ben Carman
1225584526 Multi Wallet support (#2345)
* Multi Wallet support

* Fix Oracle tests

* Fix some CI errors

* Increase KeyManagerAppConfig code coverage

* Add test for multiple seeds in seed folder

* Move things in reference.conf files

* Lower key manager coverage requirement

* Fix postgres test issues

* Create DbAppConfig

* Add some docs, fix hardcoded postgres db name

* Clarify db vs schema

* Add character types

* Fix chaindb in conf

* Always copy default seed

* Remove need for driverName config
2020-12-16 17:27:56 -06:00
Ben Carman
61d3a14c07 Don't allow fee unit gen to be 0 (#2346)
* Don't allow fee unit gen to be 0

* Fix other 0

* Fix always 0 fee unit gen
2020-12-12 07:34:26 -06:00