* Fix missing super.stop() to shutdown DbAppConfig db connection pool inside of all DbAppConfig subclasses that override stop()
* Fix import
* Remove implicit modifier to NodeUnitTest.getFreshConfig(), make sure we are shutting down chainAppConfig inside NodeUnitTest
* Remove chainTest subclasses implementing ActorSystem, make sure we do correct cleanup in ChainUnitTest
* Decouple CachedChainAppConfig & CachedAppConfig. When using only one sub module in the CachedAppConfig, teardown may fail as the other modules might not be started such as using chainAppConfig in chainTest, BitcoinSAppConfig.stop() requires us to call nodAppConfig.stop() first, which isn't started.
* Push to github to force re-run of CI 4
* Reduce number of rows selected by best filter header/best filter query
* Remove benchmarking code
* use same queries for best height and best filter so we don't repeat ourselves
* join by blockhash
* Apply optimization to compact filter table
Co-authored-by: Ben Carman <benthecarman@live.com>
* Give connection pool example config
* Get hikari logging working on postgres
* fix 2.12.x
* Bump metrics core to a supported version
* Add hikari-logging and hikari-logging-interval configuration options to turn off/on hikari and schedule how often logs appear
* Turn off hikari logging in the db-commons/reference.conf and add it to appserver/reference.conf
* default the logging to off in scala code
* Make sure connection pool is enabled in db-commons, add default database configuration to website
* Address code review, fix things that didn't need to be changed
* Add walletinfo rpc
* Move rootXpub to key manager obj
* Make configuration keys for getinfo the same as the ones in our configuration files
* Cleanup
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* Rename NodeUnitTest.confg -> NodeUnitTest.getFreshConfig()
* Rename CachedBitcoinSAppConfig.config -> CachedBitcoinSAppConfig.cachedConfig
* Make CachedChainAppConfig extend CachedBitcoinSAppConfig
* Make ChainApi.processHeaders() return a failed future in the case we have no valid headers
* Run scalafmt
* Fix test case to check if promise is completed yet
* WIP: Get something working that isn't network specific
Start putting things back in place
Add comment
Revert logback file
Remove BitcoinSLogger object
* Fix unused import
* Get rid of annoying diff
* Fix spacing nit
* WIP
* Get neutrino node with wallet 'receive information about received payments' working again
* Fix compile
* Remove initial sync logic from test case
* Remove sync logic in NeutrinoNodeWithWallet test cases
* Improve logging and rename a few things
* WIP2
* WIP3
* Get NeutrinoNodeWithWallet tests working
* Implement WalletSync, which allows you to sync a wallet from a arbitrary data source
* Get all tests passing again
* Use spv.appConfig in DataMessageHandlerTest rather than caching the config
* Modify cleanup to hopefully get CI passing
* Fix postgres tests by cleaning the table during the destroy phase of the test fixture. This is needed because the same postgres database is shared between tests in the same test suite
* Revert logback-test.xml
* Get sqlite/postgres tests passing pt 2
* syncHeight -> syncDescriptorOpt()
* Add case for genesis block hash in WalletSync
* Fix SpvNodeWithWallet test case to actually test spv functionality
* Remove nodeCallbacks parameters, callbacks should be registered on nodeAppConfig
* Rework BlockHeaderDAO.chainTips into two methods: BlockHeaderDAO.{getBestChainTips,getForkedChainTips}. getForkedChainTips is needed for reorg situations in the case a block header is received that builds off a stale tip
* Deduplicate BlockHeaderDAO.getBlockchains() that are subchains for the best chains
* Multi Wallet support
* Fix Oracle tests
* Fix some CI errors
* Increase KeyManagerAppConfig code coverage
* Add test for multiple seeds in seed folder
* Move things in reference.conf files
* Lower key manager coverage requirement
* Fix postgres test issues
* Create DbAppConfig
* Add some docs, fix hardcoded postgres db name
* Clarify db vs schema
* Add character types
* Fix chaindb in conf
* Always copy default seed
* Remove need for driverName config
* Get Scala 2.13.4 compiling
* Fix broken test case, in general use if() else() statements for Vectors rather than pattern match as I am unsure of how to correctly pattern match on Vector0,Vector1,...,Vector6()
* Bump CI to 2.13.4
* OVerride CurrencyUnit.equals() so that it works on Scala 2.13.4, not sure why there was a regression here between Scala versions 2.13.4/2.13.3
* Add explicit unit tests making sure bitcoin/satoshi currency units are equivalent
* Introduce ChainHandlerCached which behaves like the old ChainHandler. Now Chainhandler.getBestBlockHeader() will read headers from the database
* Remove ChainHandler.blockchains field, now it's only available in ChainHandlerCached
* De-futurify ChainHandler.fromDatabase()
* Adjust logging
* Patch test case
* Use BlockHeaderDAO.chainTips when getting best header rather thean BlockHeaderDAO.getBlockchains(). Implement a helper method ChainHandler.toChainHandlerCached()
* Fix chain.md,wallet.md
* Make ChainHandler.getBestBlockHeader() consider time of header if chainwork is the same. Make test cases less strict on what header is the best header when both chainwork and time are the same on the eader
* Only execute callbacks on headers that are going to be created in the database, not all headers passed into ChainHandler.processHeadersWithChains()
* Turn up log level again
* Small optimizations, check if we have seen a header before before processing it in ChainHandler.processHeadersWithChains(). Fix FilterSyncMarker.toString(). Use ChainHandlerCached in Node
* Remove ChainHandlerCached in appServer, re-add it in Node.scala
* Cleanup a bunch of imports in test
* Use @nowarn annotation for testing things that are deprecated
* Fix base58 test
* Cleanup crypto-test
* Use scala collection compat dependency to get access to the @nowarn annotation on scala 2.12.x
* Rework more scala 2.13.x compile failures in chain-test and key-manager-test
* Optimize imports for entire project
* Fix nits
* Use absolute paths for resolving configurations
Make log level OFF again
WIP: Move resolution of config into AppConfig.start(), things are totally broken
WIP: Make AppConfig.config private
Get all tests passing again but using absolute path in configuration
* Cherry-pick onto master
* Fix BitcoinSAppConfig configuration keys to use the full path
* Apply code review
* Fix log, move invariant for requiredConfirmations back into the lazy val
* Fix DlcOracleAppConfig to use absolute path
* Introduce 'FilterSyncMarker' to ChainApi, make it clearier what exactly the (Int,DoubleSha256Digest) tuple is returned from ChainApi.nextBlockHeaderRange()
* Fix doc
* Add scaladoc to FilterSyncMarker
* Rebase onto master, fix conflicts to use FilterSyncMarker
* Pull over test cases, WIP
* Rework ChainHandler.nextBlockHeaderRange() to use hashes rather than heights to very integrity of header chain, add test case
* Modify test case to make sure we are generating the 'heaviest chain work' header as first in the tuple that is used in reorg test cases
* Refactor to helper method called 'findNextHeader()' to make things simpler
* Add scaladoc to ChainHandler.findNextHeader()
* WIP
* Fix bugs in ChainApi.nextBlockHeaderRange(), now return (startHeight,stopHash) rather than (stopHeight,stopHash)
* Get chain handler tests passing, remove genesis filter header/fitler from default chain project fixture, make it a specialized one to have genesis filter header/filter in the database. Use that specialized fixture in ChainHandlerTest for now
* FilterSync work without having the genesis filter and filter header inserted into the database
* fix bug in DataMessageHandler where we were using sendNextGetCompactFilterHeadersCommand rather than using peerMsgSender.sendGetCompactFilterHeadersMessage
* Refactor ChainHandler.findNextHeader() to centralize where chains are fetched
* Address ben's code review
* Fix maxBy() exception in the case of empty Blockchain in ChainHandler.bestFilterHeaderSearch()
* Add another unit test to CompactFilterHeaderDAO for coverage
* Implement getBestFilterHeader based on a number of block headers that can be passed in as a parameter. These headers can be used to indicate what your current best chain is
* Bring back compiler opts
* Fix compiler error for maxByOption as it isn't in the 2.12 std library
* Implement a context free best filter headers search. The basic strategy is to look at headers in the _future_ of our current best filter header
* Fix bug in sql query were we were doing max chainWork too early on block headers, we needed to filter out headers in our set and _then_ we get the max chain work
* Add more unit tests
* Use database to calculate number of confirmations
* Check block is in best chain
* Use db
* Optimize getNumberOfConfirmations
* Fix compile issue for older versions
* Add better logging messages
* More logs
* Update confs sequentially
* Send GetHeadersMessage using all of our cached headers to prevent reorgs from stalling node
* Add test, improve logging
* Start with correct chains
* Simplify error, change back to createAll
* Optimize getBestFilter functions
* Remove debug statement
* Change varbinary to varchar, create index
* Fix getBlockchainFrom for tests
* Create index for postgres
* Fixed logging so that appenders are always started when logging is happening
* Started rolling policy so that file appenders could start in getLoggerImpl
* Update scalafmt-core to 2.6.0
* Upgrade scalafmt, fix all formatting errors
* Scalafmt on 2.12
* fix test
* More fixes
Co-authored-by: Ben Carman <benthecarman@live.com>
* Have Travis only run tests if it compiles
* Fix warnings + fix compile issue
* Fix compilation issue
* Check formatting in first stage
* Fix name
* Fix formatting error
* Fix Chaindb chainwork Migration
* Move chain work calculation out of node, use the ChainApi returnd by runChainWorkCalc()
Co-authored-by: Ben Carman <benthecarman@live.com>
* Implement best block hash correctly
* Handle no common history
* Fix getBlockProof to be BigInt instead of UInt32
* Fix warnings
* Make get best filter use chainwork
* Remove unused function, test isMissingChainWork
* Fix comparisons for chain work in sql
* Fix migrations from rebase
* Fix CI error
* Fix postgresql
* Added crypto project and decoupled BitcoinSLogger from NetworkElement
Decoupled BitcoinSLogger from Factory
Moved NetworkElement into crypto project
Moved Factory and BitcoinSUtil (renamed to BytesUtil) to crypto project
Moved MaskedToString to crypto project
Added BytesUtil to imports and cleaned up CryptoUtil.recoverPoint
Moved the rest of crypto stuff to the crypto project
Moved crypto tests to crypto-test project
* Added documentation for crypto project
* Add wallet creation time for rescans
* Fix docs
* Clean up and add test
* Remove account bday
* Fix compile issue and docs
* Add more chain handler tests
* Use Instant over Long, add docs
* Fix docs
* Implement Slick's multi database support via our DAOs
WIP2 work on chain project
WIP make JdbcProfieComponent so we have a common JdbcProfile
WIP2
Clean up a few things
WIP got CRUDAutoInc compiling
Get closer to everything compiling in the chain project
Begin moving database code into JdbcProfileComponent
WIP source compiling
Get dbCommonsTest/ test cases passing
Get rest of tests compiling and passing
* Rebase onto master, fix conflicts
* Run scalafmt
* Run scalafmt again
* Cleanup a few compiler warnings by adding type annotations to thigns
* Explicitly cast tables where they are needed in the wallet project where we need TableQuery for foreignkey / joins in slick
* Redo definition of 'table' to use the profile.api.TableQuery in scope
* Rename all *Table.scala files to *Db.scala
* First crack at address ben's code review
* Fix docs compile issue
* Fixed casting (#30)
* Fixed AppConfig casting
* Fixed TableQuery casting in DbManagement subtypes
* Fixed casts in DAOs relating to internal types
* Address nadav's code review, move JdbcProfileComponent into it's own file
* Remove unused listTables() method
Co-authored-by: Nadav Kohen <nadavk25@gmail.com>
* Rename BlockHeaderDAO.getNChildren() to BlockHeaderDAO.getNAncestor() as that is what it actually does
* This changes the behavior of BlockHeaderDAO.getNAncestors from the previous implementation which just fetched headers in between two heights, to actually validating the headers that are fetched from the database with Blockchain.reconstructFromheaders()
* Run scalafmt
* Replace BlockHeader.getNAncestors call to Blockchain.reconstructFromHeaders() with Blockchain.connectWalkBackwards() so that we don't do full POW validation from the database, just make sure the headers connect to avoid reorg problems. Also add some more informative logs
* Address Roman's code review
* Add wallet rescan tests
* Create FilterSync, which gives us an API inside of the chain project to sync filters with
Add another unit test to filter sync
Add more unit tests for ChainSync and FilterSync
Clean up some docs, remove some extra lines of code
Run scalafmt
Add filter-sync.md
Cleanup some nits
Add more information of how FilterSync.syncFilters() works
Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project
Run scalafmt, hide imports in filter-sync.md so code appears cleaner
Move implicits out of invisible block as it seems to cause errors
Make it so FilterSync processes filters in batches rather than fetching them all at once
Fix compile error
* WIP bitcoind implement ChainQueryApi
* rework fixtures to be able to support injecting ChainQueryApi implemented by bitcoind into our fixture infrastructure for creating wallets
* Fix rebase problem
* Implement getFiltersBetweenHeight() with ben's solution
* WIP Start implementing NodeApi against bitcoind
* Actually inject the bitcoind backed nodeApi into our fixture
* Get first rescan test working for rescanning the entire blockchain
* Implement test case for rescanning from a specific height
* Fix NeutrinoNodeWalletTest test case that uses a experimental version of bitcoind, for now i believe our experimental binary is on v18
* Add wallet-rescan.md, add helper method to WalletApi that allows you to clear out the wallet's utxos/addresses
* Add another log to try to debug CI
* Address code review
* Fix wrong ordering of deletion of tables in clearUtxosAndAddresses()
* reset logging level
* Create FilterSync, which gives us an API inside of the chain project to sync filters with
Add another unit test to filter sync
Add more unit tests for ChainSync and FilterSync
Clean up some docs, remove some extra lines of code
Run scalafmt
Add filter-sync.md
Cleanup some nits
Add more information of how FilterSync.syncFilters() works
Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project
Run scalafmt, hide imports in filter-sync.md so code appears cleaner
Move implicits out of invisible block as it seems to cause errors
Make it so FilterSync processes filters in batches rather than fetching them all at once
Fix compile error
* Add comment about trust model
* Run scalafmt
* Renamed size to byteSize in NetworkElement to avoid ambiguity
* Introduced SeqWrapper and MapWrapper to allow for wrapper case classes to have direct access to underlying methods
* Responded to review
* Add flyway migrations
* Make different project's migrations independent of each other
* Rework all AppConfig.initialize() to use migrations rather than what we were doing before
* TXO State migration
* Move to new file, drop old column
* Add block hash column
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
