* Encapsulate initialization of DLCOracle.start() method
* Use internal WalletAppConfig.kmConf rather than passing in custom key manager parameters
* Add KeyManagerAppConfig.defaultAccountType
* Get all tests passing besides TrezorAddressTest
* Get TrezorAddressTest passing with provided entropy
* Add unit test to make sure we can always derive the seed
* Get docs compiling
* Fix dlcWalletTest test cases
* Add more test cases to keymanager
* Add the new configuration to the example configuration
* Add more test cases
* Remove coverage on 2.12 as it isn't accurate
* Rework DLCOracleAppConfig.start() to call kmConf.start() so the oracle can use entropy provided via bitcoin-s.conf
* Sort DLC table by last updated
* Add LastUpdatedDb
* Make visible
* Fix compile
* Add time, fix sorting
* Switch to local time
* fix compile
* Fix test
* Fix postgres migration
* Specify zoneId
* Tor packaged for testkit
* Add CI row
* Small fixes
* More fixes
* Keep tor running
* Skip tor test for old version of bitcoind
* Respond to review
* Fix with master
* Bump tor start time to same as app config
* Use cached tor for DLCNodeTest
* Disable parallel execution for dlc node test with tor
* Fix DLCNodeTest for tor and clearnet
* Fix name of CI rows
* P2P reconnect logic
* some fixes
* more changes
* fix DuplicateFilters exception in case of disconnect in the middle of syncing filters
* more fixes
* Repurpose maxconnections test to be just a generic reconnection test
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* DLC P2P WIP
* P2PClient refactor (#10)
* Add to CI
* Remove unused
* Attempt to create DLCNode and Tests
* Full Tor support
* Get DLCNegotiationTest passing
* Config options, connect & send func
* Test for DLCNode
* Add createDLCNode to config
* Fix formatting
* Update DLC state after all other data is set
* Remove unneeded line
* Respond to some review
* 2021 07 26 dlc node code review (#13)
* WIP
* WIP2
* Rewrite tests not use Await.result()
* Skip Tor test on CI
* Cleanup threadpool leaks in tests
* Handle actor pattern matching better
* Respond to review
* Implement DLCNode.stop
* sock5 -> socks5
* Use Tcp.Unbind
* Respond to review
* Implement postStop
* Switch to unbind
Co-authored-by: rorp <rorp@users.noreply.github.com>
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* relay flag in version message made configurable
* modified test classes as well
* Update configuration.md
* reverted few files and cleaned up format
* fixed mistake
* enabled neutrino node to receive transactions as a callback
* added txid to log message
* changed txId to txIdBE
* unit test added for checking if we're getting transaction as a callback
* Add unit test we can handle spending funds and receiving funds in the same transaction inside the wallet
* Use fundRawTranaction() to make sure the utxo is reserved
* Try to add block generate to address in fixture setup to get around the case where block headers & filter headers are synced but compact filters are not
* Push to github to force re-run of CI 2
* Add ChainApi.getBestFilter(), use it in sync to determine if filter headers & filters are in sync
* Push to github to force re-run of CI 3
* Simplify getBestFilter, add unit test
* Add more comments, clean up some code
* Pulled down dlc and dlcTest projects into core and dlcTest
Added dlcTest to CI runs and fixed compilation issues
* Fixed docs
* loosened fee rate bound assertions to only sanity test
Turn off logging again
Try to move things around even more to make sure zmq is started
Turn logging level down to ERROR to hopefully get some meaningful output
Adjust both valdiation & ChainHandler to be ERROR level logging
switch zmqpubrawblock to be the port we know is free
Reduce logging to WARN
Bump bitcoind max retries to 120
Fix rebase
Revert logging
Remove some noisy logs
Restore logs to trace level
Add explicit tag
* Refactor MultiWalletTest to actually shutdown the walletAppConfigs
* Remove 'implicit' modifier from getFreshWalletConfig. This is a problem as it can be used implicitly to bind resources (threads, files etc) that need to be cleaned up. Since it is implicitly passed, it is very hard to remember to clean up these resources
* Cleanup TrezorAddressTest
* Remove unecessary BitcoinSWalletTest.afterAll()
* Fix docs
* Removal of datadir race condition
* Cleanup nodeTest to shutdown the transitive chainAppConfig required by NodeAppConfig
* remove experimental tag on NeutrinoNodeTest
* Remove extra line
* Cleanup after BitcoindChainhandlerViaZmqTest
* Push to github to force re-run of CI 2
* Reduce pg connections from 300 -> 50 in test cases
* Reduce connection count to 25
* Push to github to force re-run of CI
* Switch explorer over to prod for now as test is down to get CI passing
* Reduce shared buffers memory size
* Push to github to force re-run of CI 2
* Fix bug introduced in PR 2597, we need to explicitly call WalletAppConfig.stop() now rather than transitively calling it via Wallet.stop()
* Fix calls to wallet.stop() -> wallet.walletAppConfig.stop() in BitcoindBlockPolling,BitcoindBackend. Also add some explicit type annotations
* Get FilterSync test working with cached bitcoind in chainTest project
* Small refactor to be DRY
* Fix docs
* Refactor ChainSyncTest to use 1 cached bitcoind, as a by product add ChainWithBitcoindNewestCachedUnitTest
* Remove unecessary mixin trait
* Fix missing ChainWithBitcoindNewestCachedUnitTest.afterAll()
* Reduce thread pool size for akka's internal dispatcher in unit tests from 2 -> 1. Same with the blocking dispatcher
* Add comment
* Fix missing super.stop() to shutdown DbAppConfig db connection pool inside of all DbAppConfig subclasses that override stop()
* Fix import
* Remove implicit modifier to NodeUnitTest.getFreshConfig(), make sure we are shutting down chainAppConfig inside NodeUnitTest
* Remove chainTest subclasses implementing ActorSystem, make sure we do correct cleanup in ChainUnitTest
* Decouple CachedChainAppConfig & CachedAppConfig. When using only one sub module in the CachedAppConfig, teardown may fail as the other modules might not be started such as using chainAppConfig in chainTest, BitcoinSAppConfig.stop() requires us to call nodAppConfig.stop() first, which isn't started.
* Push to github to force re-run of CI 4
* Move wallet scheuler into WalletAppConfig, use scheduler.shutdownNow() so we actually shutdown the scheduler rather than wait for all queued tasks to finish executing before shutting down
* Fix typo
* Move rescan thread pool in to walletAppConfig, shut it down on wallet shutdown
* Fix compile
* Fix missing database shutdown on WalletAppConfig.stop()
* Cleanup BitcoindBlockPollingTest
* Cleanup ProcessBlockTest and all tests that use CachedBitcoind
* Cleanup BitcoindBackendTest
* Cache rescan thread pool rather than creating a new one everytime
* Add invariant to spendingInfoDb to that requires if the spendinginfodb is in a TxoState.spentStates, the SpendingInfoDb.spendingTxIdOpt is defined
* Remove unused SpendingInfoDAO.updateTxoState()
* Remove comment
* Make TxoState and explicit parameter for test methods. Most test methods depend on a sample utxo state, so make the caller of the method specify what state they want the txo to be in
* Fix github actions config
* Fix RoutesSpec test
* Wallet Rebroadcast Logic
* Use wallet scheduler for address queue rather than it's own thread
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* Bump default
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* Schnorr sigs for Scala.js
* fix build
* put BIP340 test vectors in a shared space
* remove teskit dependency, fix point edge cases
* fix build
* add unit tests for point addition
* scaladoc
* cleanup
* respond to the comments
* Fix usage of BitcoinSLogger
Co-authored-by: christewart <stewart.chris1234@gmail.com>
* Create CachedBitcoind, implement it in FundTransactionHandlingTest
* Add BaseWalletTest, extend it with BitcoinSWalletTest & BitcoinSWalletTestCachedBitcoind, add CachedBitcoinV19 and use it RescanHandlingTest
* Make ProcessBlockTest work with cached bitcoind
* Make trait for CachedBitcoindNewest for the newest version of bitcoind
* Make UTXOLifeCycleTest use cached bitcoind
* Add WalletBloom, WalletSyncTest to use cached bitcoinds
* Add WalletIntegrationTest
* Rework beforeAll() and afterAll() into the super trait like BaseWalletTest
* Add standlone BitcoindFixtures, use it in BitcoindBackendTest
* Use new BitcoindFixtures in BitcoindBlockPollingTest
* Introduce BaseNodeTest, start implementing the usage of cached bitcoinds in the nodeTest project
* Use cached bitcoind's with SpvNodeTest & SpvNodeWithWalletTest
* Fix bug on postgres with reusing database, upsert the genesis header rather than create it
* Get NeutrinoNode tests workign with cached bitcoinds
* Fix NeutrinoNodeWithWallet by destroying wallet state for Postgres
* Add teardown helper method for bitcoind
* Teardown chain project when using node fixtures since node is dependent upon the chain project.
* Turn off parallelExecution again
* Switch the parallelExecution flag to only be set on CI, so we can get better performance when running locally
* Start implementing BitcoindFixtures, use BitcoindFixturesCachedTriple on TestUtilRpcTest
* Fix compiler errors, begin implementing NodePair
* Refactor TestRpcUtilTest to use 2 bitcoinds rather than 2
* Reduce the number of bitcoinds that MultiWalletRpcTest needs from 3 -> 1
* Reduce number of bitcoinds used in WalletRpcTest from 3 -> 2
* Add some documentation
* Try to re-add parallelExecution
* Reduce the number of bitcoinds used in PsbtRpcTest from 3 -> 2
* Disable parallelExecution in Test again
* Make BitcoindV21RpcClientTest & BitcoindV20RpcClientTest reduce bitcoind usage from 2 -> 1
* Make BitcoindV19RpcClienttest reduce bitcoind usage from 2 -> 1
* Rework MempoolRpcTest to use fixtures, add BitcoindVersion to CachedBitcoindCollection
* Make sure clientAccumm has to be specified as a paramter now rather than filling in by default
* Begin parameterizing NodePair/NodeTriple to retain type information for the specific version of bitcoind that was used
* Don't implement version in super trait
* Fix docs
* Fix async issue in V21 test suite
* Append to vectors in CachedBitcoinCollection rather than replace
* Fix rebase issues
* Add scaladocs
* Fix BitcoindV18RpcClient address info test
* Implement fixtures in BitcoindV17RpcClientTest fixtures
* Cleanup v17 PsbtRpcTest
* Reduce bitcoind usage from 3 -> 1 in BitcoindV18RpcClientTest
* Remove abandon transaction test, this allows us to reduce the number of bitcoind's used in MempoolRpcTest from 3 -> 2
* Remove the requirement to inject BitcoinSAsyncFixtureTest, add it in the test traits explicitly to make things easier. Also add explicit afterAll() method to tear down both the CachedBitcoind & BitcoinSAsyncFixtureTest
* Fix missing Await.result() in BitcoindRpcTest.afterAll()
* Rework MultiWalletRpcTest to use a NodePair
* Rework BlockchainRpcTest to use fixtures
* Rework Client start()/stop() methods. Now use an AtomicBoolean to indicate when a user has requested a client to start/stop rather than sending pings to bitcoind that can fail because the conneciton pool has been shutdown in test cases
* Try my luck with turning on parallelExecution in CI again
* Revert parallelExecution, now testes do not run in parallel on CI
* Only turn off parallelExecution for bitcoindRpcTest
* Adjust build to only have bitcoindRpcTest NOT in run parallel on mac, reduce number of blocks used in BitcoindRpcTestUtil.createNodeSequence
* Run less tests in the rpc test suite as that takes the longest, move them over to node/wallet/dlc test suite on mac osx CI
* Don't run eclair tests in parallel either
* Remove CachedBitcoind from BitcoinSWalletTest
* Fix async bug in test case
* Push to github to force re-run of CI
* Push to github to force re-run of CI
* Push to github to force re-run of CI
* Move blockhash to tx table from spending info table
* Add test, fix spending tx id col name
* Scaladocs, add test
* Add more unit test
* Make id not comparable in process tx test
* Fix tests
* attempt to fix
* Add mempool comment to scaladoc
* Deparallelize process inputs & outputs
* Refactor starting second bitcoind in MempoolRpcTest, remove Thread.sleep
* Add AkkaUtil.nonBlockingSleep(), replace usages of Thread.sleep() with it where possible
* Remove more Thread.sleep()
* Try bumping threadpool for scheduler
* Reduce scheduler threadpool size to 2
* Push to github to force re-run of CI
* Refactor AsyncUtil to just use an execution context rather than an actor system
* Create testkit-core, start moving data structures over to testkit-core
* WIP, getting rid of the hard dependency on BitcoinSAsyncTest and akka
* Rework test traits to not be dependnent on akka, move BaseAsyncTest into testkitCore, add a test trait that just uses the default scala executionContext
* Get everything compiling
* Move logback-test.xml to testkit-core so it applies to coreTest,cryptoTest as well
* Cleanup rebase issues
* Fix Deps.scala
* Address ben's code review
* Segregate jvm settings and compiler settings
* Filter out -Xfatal-warning for scalajs for now since there is a ton of errors
* Move maintainer settings into CommonSettings.settings
* Refactor crypto module to be compatible with Scala.js
* more changes
* some more changes
* abstract out Schnorr stuff
* abstract out adapter stuff
* cleanup
* some more cleanup
* fix build
* Removed references to ECPoint outside of .jvm scope
* remove references to ECPoint from the shared code
* cleanup
* remove cirlular dependencies
* more cleanup
* cleanup
* move SipHash to CryptoContext
* scaladoc
* scalafmt
Co-authored-by: nkohen <nadavk25@gmail.com>
* Refactor AsyncUtil to just use an execution context rather than an actor system
* Create async util project, use it in various projects that depend on it
* Give connection pool example config
* Get hikari logging working on postgres
* fix 2.12.x
* Bump metrics core to a supported version
* Add hikari-logging and hikari-logging-interval configuration options to turn off/on hikari and schedule how often logs appear
* Turn off hikari logging in the db-commons/reference.conf and add it to appserver/reference.conf
* default the logging to off in scala code
* Make sure connection pool is enabled in db-commons, add default database configuration to website
* Address code review, fix things that didn't need to be changed
* Implement oracleAttestmentV0TLV, save outcome to db
* Update with tlv changes
* Update tlv to not have 2 list sizes
* small change
* Update OracleEvent.verifyAttestations
* Use correct pub key
* Verify correct outcomes
* Bitcoind v0.21.0rc3 support
* Use v0.21.0 instead of Experimental in node tests
* Bump to rc5
* Use official release
* Fix rebase issues
* Remove DoNotDiscover tag
* Fix count for NeutrinoNodeTest
* Rename NodeUnitTest.confg -> NodeUnitTest.getFreshConfig()
* Rename CachedBitcoinSAppConfig.config -> CachedBitcoinSAppConfig.cachedConfig
* Make CachedChainAppConfig extend CachedBitcoinSAppConfig
* use cached configs in both wallet/chain project as well taking ben's suggestion to fix those
* Clean up fixture shutdown code a bit to try and see if this resolves issue on CI
* Rework fixtures so we aren't starting app configs in multiple places
* Fix scaladoc
* WIP
* Get neutrino node with wallet 'receive information about received payments' working again
* Fix compile
* Remove initial sync logic from test case
* Remove sync logic in NeutrinoNodeWithWallet test cases
* Improve logging and rename a few things
* WIP2
* WIP3
* Get NeutrinoNodeWithWallet tests working
* Implement WalletSync, which allows you to sync a wallet from a arbitrary data source
* Get all tests passing again
* Use spv.appConfig in DataMessageHandlerTest rather than caching the config
* Modify cleanup to hopefully get CI passing
* Fix postgres tests by cleaning the table during the destroy phase of the test fixture. This is needed because the same postgres database is shared between tests in the same test suite
* Revert logback-test.xml
* Get sqlite/postgres tests passing pt 2
* syncHeight -> syncDescriptorOpt()
* Add case for genesis block hash in WalletSync
* Fix SpvNodeWithWallet test case to actually test spv functionality
* Remove nodeCallbacks parameters, callbacks should be registered on nodeAppConfig
* Rework BlockHeaderDAO.chainTips into two methods: BlockHeaderDAO.{getBestChainTips,getForkedChainTips}. getForkedChainTips is needed for reorg situations in the case a block header is received that builds off a stale tip
* Deduplicate BlockHeaderDAO.getBlockchains() that are subchains for the best chains
* Fail broadcasting transaction when disconnected
* Wait until disconnected
* Move test
* Move to separate file
* Attempt fix
* Make test never have a peer
* Improve reliablity
* Moved dlc data structures from commons to core
* Renamed DLC payout curve classes
* Split OutcomeValuePoint up into an ADT
* Added utility for computing Schnorr multiple-signature points
* Replaced tuples in RoundingIntervals with types
* Replaced tuples in DLCPayoutCurve with Indexed
* Fixed a compile bug
* Multi Wallet support
* Fix Oracle tests
* Fix some CI errors
* Increase KeyManagerAppConfig code coverage
* Add test for multiple seeds in seed folder
* Move things in reference.conf files
* Lower key manager coverage requirement
* Fix postgres test issues
* Create DbAppConfig
* Add some docs, fix hardcoded postgres db name
* Clarify db vs schema
* Add character types
* Fix chaindb in conf
* Always copy default seed
* Remove need for driverName config
* Pulled down work from adaptor-dlc onto master
* Reverted some accidental deletions
* Removed unused import
* Added scaladocs
* Responded to Ben's review
* Added some scaladocs and invariants
* Responded to chris' review
* Responded to more review
* Added some comments
* Start refactoring testkit to allow for specifying a different binary directory than the sbt default
* Fix BitcoindRpcTestUtil
* Add BitcoindRpcTestClient
* Make BitcoinSAyncTest more thread safe and make sure we have all the expected test cases aggregated before we starting reducing over the results
* remove extra code
* Fix scala 2.12.x compile
* Introduce ChainHandlerCached which behaves like the old ChainHandler. Now Chainhandler.getBestBlockHeader() will read headers from the database
* Remove ChainHandler.blockchains field, now it's only available in ChainHandlerCached
* De-futurify ChainHandler.fromDatabase()
* Adjust logging
* Patch test case
* Use BlockHeaderDAO.chainTips when getting best header rather thean BlockHeaderDAO.getBlockchains(). Implement a helper method ChainHandler.toChainHandlerCached()
* Fix chain.md,wallet.md
* Make ChainHandler.getBestBlockHeader() consider time of header if chainwork is the same. Make test cases less strict on what header is the best header when both chainwork and time are the same on the eader
* Only execute callbacks on headers that are going to be created in the database, not all headers passed into ChainHandler.processHeadersWithChains()
* Turn up log level again
* Small optimizations, check if we have seen a header before before processing it in ChainHandler.processHeadersWithChains(). Fix FilterSyncMarker.toString(). Use ChainHandlerCached in Node
* Remove ChainHandlerCached in appServer, re-add it in Node.scala
* Update Oracle to use new TLVs
* Rename things to use new names, scaladoc, small clean ups
* Add descomposition tests, docs, sign numbers outside of range
* Update Oracle TLVs
* Change string size to be BigSizeUInt
* Move nonces to oracle event tlv
* Introduced EventDescriptor tests and NumericEventDescriptor methods to query possible outcome data
* Added contains function
* Added tests for containsToPrecision
* Remove EventDescriptor.outcomes and other related fields
* remove unused formatNum method
* Address code review from Ben
* Move oracle pub key to announcement
Co-authored-by: nkohen <nadavk25@gmail.com>
Co-authored-by: christewart <stewart.chris1234@gmail.com>
* Make aesPassword option for wallet config
* Add to docs
* Make AesPassword optional
* Small touchups
* Fix for oracle server
* Fix docs
* Increase code coverage
* Cleanup a bunch of imports in test
* Use @nowarn annotation for testing things that are deprecated
* Fix base58 test
* Cleanup crypto-test
* Use scala collection compat dependency to get access to the @nowarn annotation on scala 2.12.x
* Rework more scala 2.13.x compile failures in chain-test and key-manager-test
* Optimize imports for entire project
* Fix nits
* Add Wallet State Descriptors
* Sync blocks while offline w/ bitcoind backend
* Add to WalletDbManagement
* Update db management test
* Respond to review
* Rename function
* Add test case for resolving typesafe config variables and making sure we honor system properties
* Remove println
* Move AppConfigTest test caess into the db-commons project
* Use absolute paths for resolving configurations
Make log level OFF again
WIP: Move resolution of config into AppConfig.start(), things are totally broken
WIP: Make AppConfig.config private
Get all tests passing again but using absolute path in configuration
* Cherry-pick onto master
* Fix BitcoinSAppConfig configuration keys to use the full path
* Apply code review
* Fix log, move invariant for requiredConfirmations back into the lazy val
* Fix DlcOracleAppConfig to use absolute path
* Add basic DLC Oracle
* Respond to review
* Respond to more review
* Add maturation time
* Add to testkit, tag hashes, better val names
* More clear vals, version tagged hashes
* Signing key clean up
* Add pubkey to db
* Introduce 'FilterSyncMarker' to ChainApi, make it clearier what exactly the (Int,DoubleSha256Digest) tuple is returned from ChainApi.nextBlockHeaderRange()
* Fix doc
* Add scaladoc to FilterSyncMarker
* Rebase onto master, fix conflicts to use FilterSyncMarker
* Pull over test cases, WIP
* Rework ChainHandler.nextBlockHeaderRange() to use hashes rather than heights to very integrity of header chain, add test case
* Modify test case to make sure we are generating the 'heaviest chain work' header as first in the tuple that is used in reorg test cases
* Refactor to helper method called 'findNextHeader()' to make things simpler
* Add scaladoc to ChainHandler.findNextHeader()
* WIP
* Fix bugs in ChainApi.nextBlockHeaderRange(), now return (startHeight,stopHash) rather than (stopHeight,stopHash)
* Get chain handler tests passing, remove genesis filter header/fitler from default chain project fixture, make it a specialized one to have genesis filter header/filter in the database. Use that specialized fixture in ChainHandlerTest for now
* FilterSync work without having the genesis filter and filter header inserted into the database
* fix bug in DataMessageHandler where we were using sendNextGetCompactFilterHeadersCommand rather than using peerMsgSender.sendGetCompactFilterHeadersMessage
* Refactor ChainHandler.findNextHeader() to centralize where chains are fetched
* Address ben's code review
* Extend StartStopAsync with BitcoinSAppConfig, create 'CachedAppConfig' test trait, clean up P2PClientTest
* Start cleaning up after ourselves in the chainTest test suite
* Call .stop() for appConfig's spun up in ChainAppConfigTest
* Database configuration defaults
* increase number of Postgres connections
* add more logging
* close connections pools in tests
* update afterAll()
* Fix conflict
Co-authored-by: rorp <rorp@users.noreply.github.com>
* Clean up code for BroadcastDAO test
* Extend StartStopAsync with BitcoinSAppConfig, create 'CachedAppConfig' test trait, clean up P2PClientTest
* Make MerkleBuffersTest use CachedAppConfig
* Start cleaning up after ourselves in the chainTest test suite
* Cancel background runnable for generating blocks after test case, add nodeAppConfig.stop() to tests in NodeAppConfigTest
* Stop mainnet nodeAppConfig in test as per ben's code review
* Refactor ChainUnitTest.withChainFixture() to use makeDependentFixture()
* Call .stop() for appConfig's spun up in ChainAppConfigTest
* Less parallelization on destruction of chainAppConfig
* Revert parallelExecution/fork jvm in CommonSettings
* Clean up code for BroadcastDAO test
* Extend StartStopAsync with BitcoinSAppConfig, create 'CachedAppConfig' test trait, clean up P2PClientTest
* Make MerkleBuffersTest use CachedAppConfig
* Start cleaning up after ourselves in the chainTest test suite
* Cancel background runnable for generating blocks after test case, add nodeAppConfig.stop() to tests in NodeAppConfigTest
* Stop mainnet nodeAppConfig in test as per ben's code review
* Output logger configuration on startup now
* Remove core-test/logback-test.xml and core/common-logback.xml, consolidate and move them into testkit/src/main/resources/logback.xml
* Add comment for logging configuration
* Cap the amount of threads an actor system can spin up to 2 in tests
* Add limits for internal dispatcher and default blocking io dispatcher
* Fix config
* Increase thread count to 2, remove unneeded lines
* Update scalafmt-core to 2.6.0
* Upgrade scalafmt, fix all formatting errors
* Scalafmt on 2.12
* fix test
* More fixes
Co-authored-by: Ben Carman <benthecarman@live.com>
* Implement unit test cases for initializing a wallet twice, and failing with an exception when we initialize a wallet with a different key-manager
* Use bip39PasswordOpt cached in the WalletUnitTest suite
* Turn off logging
* Run scalafmt
* Turn log level back to WARN
* Run scalafmt
* Pulled down crypto things from dlc branch including: Schnorr data structures, Schnorr Bouncy Castle implementation, FieldElement, and tests for these things
* Call fromValidHex in CryptoUtil
* Implement best block hash correctly
* Handle no common history
* Fix getBlockProof to be BigInt instead of UInt32
* Fix warnings
* Make get best filter use chainwork
* Remove unused function, test isMissingChainWork
* Fix comparisons for chain work in sql
* Fix migrations from rebase
* Fix CI error
* Fix postgresql
* Made the standard non-interactive tx finalizer simple using composition of its (reusable) pieces
* Responded to code review and made minor improvements
* Fixed docs
* Refactored Transaction
Created RawTxBuilder
Created RawTxFinalizer as layer on top of RawTxBuilder
Introduced RawTxSigner and replaced TxBuilder!
Deleted TxBuilder!
* fixed things after rebase
* Made RawTxBuilder compatible with older versions of scala
* Began responding to review
* Made Finalizer take a Builder rather than the other way around
* Added composition for finalizers
* Ran scalafmt
* Updated txbuilder example documentation
* Moved tests from old TxBuilderTest files to relevant new test files
* Added scaladocs
* Reduce number of property based tests on BIP32PathTest and add a Future to NetworkPayloadTest's property based tests so it hopefully runs faster on CI
* Run scalafmt
* Make a few more property based tests async
* Make forAllParallel, refactor some test cases to use it
* Make BlockMessageTest async
* Bump timeout
Added OutputReference and introduced public key computation on SPKs
Made InputInfo into an ADT
Made UTXOSpendingInfo use InputInfo
Replaced UTXOSpendingInfo with NewSpendingInfo and got non-test things compiling
Made aliases for (NewSpendingInfo/NewSpendingInfoFull/NewSpendingInfoSingle)[InputInfo]
Got rid of source code mention of UTXOSpendingInfo
sbt compile passes
tests compile
Fixed all coreTest tests
Renamed to UTXOInfo
Some cleanup
Moved redeem script and script witness accessors to companion object
Responded to code review
Added OutputReference and scaladocs
Moved p2pkhPreImageOpt downstream and generalized to hashPreImages
Fixed adding-spks.md
Fixed psbts.md and txbuilder.md
Renamed UTXOInfo
Apply renaming to docs
* Make Bech32Address.toString actually print the string representation not the object one
* Remove Bech32Address.toString all together and use the super class definition
* Add test case
* Run scalafmt
* Added crypto project and decoupled BitcoinSLogger from NetworkElement
Decoupled BitcoinSLogger from Factory
Moved NetworkElement into crypto project
Moved Factory and BitcoinSUtil (renamed to BytesUtil) to crypto project
Moved MaskedToString to crypto project
Added BytesUtil to imports and cleaned up CryptoUtil.recoverPoint
Moved the rest of crypto stuff to the crypto project
Moved crypto tests to crypto-test project
* Added documentation for crypto project
* Add wallet creation time for rescans
* Fix docs
* Clean up and add test
* Remove account bday
* Fix compile issue and docs
* Add more chain handler tests
* Use Instant over Long, add docs
* Fix docs
* Re-enable NeutrinoNodeWithWalletTest with it disabled for mac
* Change to only Linux
* Only disable for CI runs
* Move IsCI to BitcoinSUtil
* Fix compile issue for earlier versions
* Fix syncing compact filters to not skip last filter
* Fix test to be compatible with new start state
* Use correct isCI
* Fix compile issue
* set CI env
* Implement Slick's multi database support via our DAOs
WIP2 work on chain project
WIP make JdbcProfieComponent so we have a common JdbcProfile
WIP2
Clean up a few things
WIP got CRUDAutoInc compiling
Get closer to everything compiling in the chain project
Begin moving database code into JdbcProfileComponent
WIP source compiling
Get dbCommonsTest/ test cases passing
Get rest of tests compiling and passing
* Rebase onto master, fix conflicts
* Run scalafmt
* Run scalafmt again
* Cleanup a few compiler warnings by adding type annotations to thigns
* Explicitly cast tables where they are needed in the wallet project where we need TableQuery for foreignkey / joins in slick
* Redo definition of 'table' to use the profile.api.TableQuery in scope
* Rename all *Table.scala files to *Db.scala
* First crack at address ben's code review
* Fix docs compile issue
* Fixed casting (#30)
* Fixed AppConfig casting
* Fixed TableQuery casting in DbManagement subtypes
* Fixed casts in DAOs relating to internal types
* Address nadav's code review, move JdbcProfileComponent into it's own file
* Remove unused listTables() method
Co-authored-by: Nadav Kohen <nadavk25@gmail.com>
* WIP: Build queue for getnewaddress
* Implement background thread to process requests to getNewAddress so that we fix issue 1009 with it's async issues
* Run scalafmt
* Replace the mutable.ArrayBuffer in AddressHandling with ConcurrentLinkedQueue
* Put FundTransactionhandling.fundRawTransactionInternal's call to getNewChangeAddress() into the for expression so we don't generate an address when the funding of the transaction fails when selecting utxos
* Move thread initialization out of method
* Switch to using ArrayBlockingQueue, which will block thread with .take() so we don't need to loop constantly, add 'addressQueueSize' and 'addressQueueTimeout' configurations
* Update wallet-test/src/test/scala/org/bitcoins/wallet/AddressHandlingTest.scala
Co-Authored-By: rorp <rorp@users.noreply.github.com>
* Add error handling to return the failed future if the queue is full, add a unit test for when the queue is full
* Run scalafmt
* Rebase
* Add scaladoc for throwing an exception
* Run scalafmt again
Co-authored-by: rorp <rorp@users.noreply.github.com>
* Incoming and Outgoing Transaction Tables
* Remove script sigs for witness txs
* Create parent tx_table for incoming and outgoing txs
* Response to review
* Use isCloseEnough
* Fix test
* Fix rebase error
* Test that tx is tracking all sats correctly
* Add wallet rescan tests
* Create FilterSync, which gives us an API inside of the chain project to sync filters with
Add another unit test to filter sync
Add more unit tests for ChainSync and FilterSync
Clean up some docs, remove some extra lines of code
Run scalafmt
Add filter-sync.md
Cleanup some nits
Add more information of how FilterSync.syncFilters() works
Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project
Run scalafmt, hide imports in filter-sync.md so code appears cleaner
Move implicits out of invisible block as it seems to cause errors
Make it so FilterSync processes filters in batches rather than fetching them all at once
Fix compile error
* WIP bitcoind implement ChainQueryApi
* rework fixtures to be able to support injecting ChainQueryApi implemented by bitcoind into our fixture infrastructure for creating wallets
* Fix rebase problem
* Implement getFiltersBetweenHeight() with ben's solution
* WIP Start implementing NodeApi against bitcoind
* Actually inject the bitcoind backed nodeApi into our fixture
* Get first rescan test working for rescanning the entire blockchain
* Implement test case for rescanning from a specific height
* Fix NeutrinoNodeWalletTest test case that uses a experimental version of bitcoind, for now i believe our experimental binary is on v18
* Add wallet-rescan.md, add helper method to WalletApi that allows you to clear out the wallet's utxos/addresses
* Add another log to try to debug CI
* Address code review
* Fix wrong ordering of deletion of tables in clearUtxosAndAddresses()
* reset logging level
* Create FilterSync, which gives us an API inside of the chain project to sync filters with
Add another unit test to filter sync
Add more unit tests for ChainSync and FilterSync
Clean up some docs, remove some extra lines of code
Run scalafmt
Add filter-sync.md
Cleanup some nits
Add more information of how FilterSync.syncFilters() works
Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project
Run scalafmt, hide imports in filter-sync.md so code appears cleaner
Move implicits out of invisible block as it seems to cause errors
Make it so FilterSync processes filters in batches rather than fetching them all at once
Fix compile error
* Add comment about trust model
* Run scalafmt
* Create migration to drop confirmations column from txo_spending_info table in the wallet database, also fix a long standing issue in BitcoinSFixture where failure cases in the testing suite were not being handled properly
* Fix scala 2.11.12 compile issue
* PSBT Serialization/Deserialization (#917)
* PSBT Serialization and Deserialization
* Remove debug code
* Create PSBTParseResult, Add more psbt test cases, and various small nits
* PSBT version checking
* Remove PSBTHelper, give PSBTRecord and PSBTMap a fromBytes
* Fix compile issue, preserve ADT
* PSBT Combiner
* PSBT Updater
* Fix PSBT version number to be read as little endian
* Make function names more clear, add scaladoc, require not finalized
* Combiner scaladocs
* Test case from BIP
* Make compatible with older versions
* PSBT: Fix for EmptyWitness changes (#968)
* PSBTInputMap to UTXOSigningInfo
* Remove vars
* Add missing param
* Fix byteVectorOrdering for when x == y
* getUTXOSpendingInfo Tests
* Fix formatting errors
* Give conditional path default value
* formatting fix
* Fixed InputPSBTRecord.FinalizedScriptSig using asmBytes (#1004)
* Fixed PSBTTest after rebasing onto master
* PSBT Constructors and Extraction (#997)
* Added PSBT constructors
* Revived option to use signers in toUTXOSpendingInfo
* Added transaction extraction
* Responded to code review
* Added extraction test from BIP
* Added validation (option) to PSBT extraction
* Added property based test for PSBT.extractTransactionAndValidate
* Marked PSBT as new code for PropertyCheckConfiguration
* PSBT Finalizer (#1002)
* Added finalizing functionality to PSBTs
* Cleaned things up
* Added finalizer test, fails
* Check that PSBT is not already finalized when finalizing PSBT
* Added P2PKWithTimeout cases
* Formatting fix
* PSBT Generators (#1019)
* Introduced non-finalized constructor
* Added property based test comparing finalized and un-finalized construction, it currently fails due to EmptyScriptPubKey not being supported in various ways. Also fixed a ton of bugs!
* Fixed easy bugs relating to EmptyScriptPubKey, one is left
* Fixed the last bugs (multisig order and 0-of-n stuff) and now tests pass!
* Separated out direct finalized PSBT construction from non-finalized PSBT construction
* Translated construction work in tests into PSBT generators
* Added serialization symmetry test
* Made PSBT fee generation dependent on other generators to ensure low enough fees for signing
* Responded to code review
* Added scaladocs
* Fixed arbitraryPSBT to work with older scala versions
* Fixed PSBT compilation breakages from rebase
* Fixed test breakages from rebase
* Validate the PSBT unknown is not a known field
* Increase code coverage on PSBT tests
* Address review
* formatting
* Add error messages, rename function
* Psbt Signer (#1025)
* PSBT Signer
* Create addSignature function for PSBTs
* Use PartialSignature instead of Tuple
* Create extra util functions
* scaladoc and simplify case
* Formatting fix for psbt scaladocs
* Clean up code, add error messages, and scaladoc
* add type hints
* Change InputPSBTMap.fromUTXOSpendingInfo to use BitcoinSingleSigner
* optimize import
* Define separator byte for psbt map
* getRecords simplification
* remove braces, add error message
* PSBT stuff assigned to nkohen (#1035)
* Responded to psbt review assigned to me
* Moved separatorByte to a place where all can use it
* PSBT independent record calls (#1037)
* PSBT independent record calls
* move logic to super class
Co-authored-by: Nadav Kohen <nadavk25@gmail.com>
* Reworked PSBT finalization to return a Try[PSBT] rather than an Option[PSBT] (#1036)
* PSBT signer property based tests (#1038)
* PSBT property based tests
* Address review
* PSBT Combiner property based tests (#1039)
* PSBT Combiner property based tests
* Create pruneGlobal util function
* Switch to arbitrary gen
* Use BaseTransaction instead of Transaction
* Fix transaction witness to work from base transactions
* PSBTs response to Chris's review (#1046)
* PSBT chris's review
* Rename val, give master fingerprint a val
* Formatting fix
* Responded to code review from chris assigned to nkohen
* Split PSBT.scala into many files
* PSBT Docs (#1048)
* PSBT docs
* Fix compiler error, remove nesting
* PSBT output updater tests
* Validate psbt outputs
* Clean up code
* Remove P2WPKHWitnessV0 addition
* format
* PSBT Updater property based tests
* Move code to match block
* EmptyScriptWitness check
* Address nadav review
* Remove unnecessary code, make utxo adder better
* Move require and add error message
* Move require
* PSBT unit tests by Ben
* Change to EmptyScriptPubKey
* Added unit tests to increase code coverage
* responded to review
* Move PSBT unit test to separate file
* Make unknown generator only generate distinct records
* Use groupBy key instead of distinct
* use only first element
* Combine distinctness by key
Co-authored-by: Ben Carman <benthecarman@live.com>
* Create 'fundRawTransaction' and 'fundRawTransactionInternal'
* Move TransactionTestUtil to testkit, begin writing tests for fundrawtransaction
* Add FundTransactionTests
* Move things to 'FundTransactionHandling' trait
* Start segregating things by HDAccount in the wallet, this gives us the ability to query for balances based on account, fund a transaction based on an account, generate a new address based on an account etc. All old api calls are now based on the default account, i.e. getBalance() returns the balance for account 0, fundRawTransaction funds the transaction from account 0 etc.
* Fix compile issue with 2.11.x
* Bump address generation Thread.sleep() to 500ms
* Address ben code review
* Address code review
* Introduced UTXOSpendingInfoSingle
* Implemented constructors for UTXOSpendingInfoSingle
* Introduced BitcoinSignerSingle
* Implemented SingleSigner
* Added property-based test for SingleSigner, it fails
* Fixed SignerTest :)
* Responded to code review
* Added scaladocs
* Fixed p2sh(p2wsh) signing
* Responded to code review and separated UTXOSpendingInfoSingle from UTXOSpendingInfoFull
* Fixed outputGen after rebase
* Fixed txbuilder.md to show this branch's changes
* Create 'fundRawTransaction' and 'fundRawTransactionInternal'
* Move TransactionTestUtil to testkit, begin writing tests for fundrawtransaction
* Add FundTransactionTests
* Move things to 'FundTransactionHandling' trait
* Address code review
* Run scalafmt
* Add more expressive txo states for the wallet
* Rename 'PendingReceived/PendingSpent' -> 'UnconfirmedReceived/UnconfirmedSent'
* Address code review
* Fix posInt implementation to work with older scala versions
* Rename to PendingConfirmationReceived/PendingConfirmationSpent
* Implement abililty to use BIP39 password. This means this password needs to be password through our various projects to be able to correctly generate the key that controls the wallet. This also renames 'CreateKeyManagerApi' -> 'BIP39CreateKeymanagerApi' as the bip39 password is needed when creating the KeyManager
* Add bip39 password to BIP39KeyManager.fromParams(), fix mdocs
* Fix bug in unit test were whe weren't specifying password
* Fix rebase issues
* This creates a subtype BIP39KeyManager and moves all existing KeyManager functionality to that subtype. We leave a empty 'KeyManager' trait for now
* Fix website docs
* Move initialization of wallet entropy into the key manager
* Enable key-manager in ci, add coverage minimum
* Add coverage minimum for key manager, add KeyManagerParams, start adding KeyManager test harness
* Remove success cases on ADTs, start using Either
* fix import
* Get test coverage to 90%
* Add documentation for key-manager.md, add side bar for key-manager
* Fix wallet.md
* Make seedPath default to baseDatadir, so we don't have a unique seed for every network. Add more documentation
* Address code review
* Address code review from Ben
* Add missing import
* Add more tests for coverage
* bitcoind v19 new RPC calls and tests (#863)
* bitcoind v19 new RPC calls and tests
* Code review changes
* Review part 2
* Rename variable to be more descriptive
* Explanitory comment
* Ignore broken test cases
* Add missing signing functions
* Add test to check avoid_reuse flag is on (#870)
* Add test to check avoid_reuse flag is on
* Add test to make sure flags weren't set
* bitcoind v19 Update mempool RPCs and tests (#868)
* Update mempool RPC calls to bitcoind v19 compatibility
* Typo fix
* Add parameter name to calls
* Fix remaining rpc calls
* Formatting
* scaladoc for param
* Change param to correct type
* Clarify on scaladoc
* Add missing fees parmater to mempool rpcs (#875)
* Add weight field to mempool entries after v19 (#876)
* Move DescriptorRpc to be able to be used by future versions of bitcoind (#878)
* Add window_final_block_height to GetChainTxStatsResult (#880)
* Add passphrase argument to createwallet for later versions (#883)
* Add passphrase argument to createwallet for later versions
* Scaladoc + empty passphrase requirement
* Error message
* Add new services names parameter to P2P rpcs (#874)
* Add new services names parameter to P2P rpcs
* Add ServiceIdentifier Reads
* Add fallback case
* Address review
* Change to Try
* Move PsbtRpc to be able to be used by future versions of bitcoind (#877)
* Move PsbtRpc to be able to be used by future versions of bitcoind
* Add test
* Address comment
* Enable bloom filters for v19
* Enable bip 61 for tests
* Change to official binaries
* Force v18 for Spv Tests
* Remove unused config line
* Segregate key manager from wallet
* More consistent naming in build.sbt, run scalafmt
* Add test case making sure we don't overwrite a mnemonic seed
* Add key-manager.md
* Fix compile issues with older scala versions
* Address code review
