Commit Graph

114 Commits

Author SHA1 Message Date
Chris Stewart
2048cc0bbb Make ChainApi.processHeaders() return a failed future in the case we … (#2436)
* Rename NodeUnitTest.confg -> NodeUnitTest.getFreshConfig()

* Rename CachedBitcoinSAppConfig.config -> CachedBitcoinSAppConfig.cachedConfig

* Make CachedChainAppConfig extend CachedBitcoinSAppConfig

* Make ChainApi.processHeaders() return a failed future in the case we have no valid headers

* Run scalafmt

* Fix test case to check if promise is completed yet

* WIP: Get something working that isn't network specific

Start putting things back in place

Add comment

Revert logback file

Remove BitcoinSLogger object

* Fix unused import

* Get rid of annoying diff

* Fix spacing nit
2021-01-12 06:11:19 -06:00
Chris Stewart
98ace6f14e 2021 01 02 issue 2457 (#2461)
* WIP

* Get neutrino node with wallet 'receive information about received payments' working again

* Fix compile

* Remove initial sync logic from test case

* Remove sync logic in NeutrinoNodeWithWallet test cases

* Improve logging and rename a few things

* WIP2

* WIP3

* Get NeutrinoNodeWithWallet tests working

* Implement WalletSync, which allows you to sync a wallet from a arbitrary data source

* Get all tests passing again

* Use spv.appConfig in DataMessageHandlerTest rather than caching the config

* Modify cleanup to hopefully get CI passing

* Fix postgres tests by cleaning the table during the destroy phase of the test fixture. This is needed because the same postgres database is shared between tests in the same test suite

* Revert logback-test.xml

* Get sqlite/postgres tests passing pt 2

* syncHeight -> syncDescriptorOpt()

* Add case for genesis block hash in WalletSync

* Fix SpvNodeWithWallet test case to actually test spv functionality

* Remove nodeCallbacks parameters, callbacks should be registered on nodeAppConfig
2021-01-09 09:33:37 -06:00
Chris Stewart
b2560c4606 Rework BlockHeaderDAO.chainTips into two methods: BlockHeaderDAO.{get… (#2443)
* Rework BlockHeaderDAO.chainTips into two methods: BlockHeaderDAO.{getBestChainTips,getForkedChainTips}. getForkedChainTips is needed for reorg situations in the case a block header is received that builds off a stale tip

* Deduplicate BlockHeaderDAO.getBlockchains() that are subchains for the best chains
2021-01-01 09:04:29 -06:00
Chris Stewart
936d4e4911 Remove BlockHeader.getBlockchainsFrom(), rework type signature for Bl… (#2431)
* Remove BlockHeader.getBlockchainsFrom(), rework type signature for BlockHeader.getBlockchainFrom() to return Future[Option[Blockchain]]

* fix nits
2020-12-26 07:17:38 -06:00
Ben Carman
1225584526 Multi Wallet support (#2345)
* Multi Wallet support

* Fix Oracle tests

* Fix some CI errors

* Increase KeyManagerAppConfig code coverage

* Add test for multiple seeds in seed folder

* Move things in reference.conf files

* Lower key manager coverage requirement

* Fix postgres test issues

* Create DbAppConfig

* Add some docs, fix hardcoded postgres db name

* Clarify db vs schema

* Add character types

* Fix chaindb in conf

* Always copy default seed

* Remove need for driverName config
2020-12-16 17:27:56 -06:00
Chris Stewart
f586e25699 Get Scala 2.13.4 compiling (#2294)
* Get Scala 2.13.4 compiling

* Fix broken test case, in general use if() else() statements for Vectors rather than pattern match as I am unsure of how to correctly pattern match on Vector0,Vector1,...,Vector6()

* Bump CI to 2.13.4

* OVerride CurrencyUnit.equals() so that it works on Scala 2.13.4, not sure why there was a regression here between Scala versions 2.13.4/2.13.3

* Add explicit unit tests making sure bitcoin/satoshi currency units are equivalent
2020-11-28 07:36:07 -06:00
Chris Stewart
66ec96b1f5 2020 11 13 issue 2258 (#2260)
* Introduce ChainHandlerCached which behaves like the old ChainHandler. Now Chainhandler.getBestBlockHeader() will read headers from the database

* Remove ChainHandler.blockchains field, now it's only available in ChainHandlerCached

* De-futurify ChainHandler.fromDatabase()

* Adjust logging

* Patch test case

* Use BlockHeaderDAO.chainTips when getting best header rather thean BlockHeaderDAO.getBlockchains(). Implement a helper method ChainHandler.toChainHandlerCached()

* Fix chain.md,wallet.md

* Make ChainHandler.getBestBlockHeader() consider time of header if chainwork is the same. Make test cases less strict on what header is the best header when both chainwork and time are the same on the eader

* Only execute callbacks on headers that are going to be created in the database, not all headers passed into ChainHandler.processHeadersWithChains()

* Turn up log level again

* Small optimizations, check if we have seen a header before before processing it in ChainHandler.processHeadersWithChains(). Fix FilterSyncMarker.toString(). Use ChainHandlerCached in Node

* Remove ChainHandlerCached in appServer, re-add it in Node.scala
2020-11-17 06:19:07 -06:00
Chris Stewart
581d46c2c8 2020 11 02 cleanup (#2233)
* Cleanup a bunch of imports in test

* Use @nowarn annotation for testing things that are deprecated

* Fix base58 test

* Cleanup crypto-test

* Use scala collection compat dependency to get access to the @nowarn annotation on scala 2.12.x

* Rework more scala 2.13.x compile failures in chain-test and key-manager-test

* Optimize imports for entire project

* Fix nits
2020-11-03 11:07:04 -06:00
Chris Stewart
ade2503702 2020 10 05 redo config (#2121)
* Use absolute paths for resolving configurations

Make log level OFF again

WIP: Move resolution of config into AppConfig.start(), things are totally broken

WIP: Make AppConfig.config private

Get all tests passing again but using absolute path in configuration

* Cherry-pick onto master

* Fix BitcoinSAppConfig configuration keys to use the full path

* Apply code review

* Fix log, move invariant for requiredConfirmations back into the lazy val

* Fix DlcOracleAppConfig to use absolute path
2020-10-05 18:00:30 -05:00
Chris Stewart
9b09b04230 Upgrade to scalac 2.13.3 (#2115) 2020-10-04 07:42:44 -05:00
Ben Carman
11e6b7d9eb Initial SigNet support (#2057) 2020-09-25 12:29:13 -05:00
Ben Carman
1e3aee55c3 Merge pull request #2019
* Refactor logging to only use grizzled slf4j

* Fix docs

* test travis config change
2020-09-19 14:09:12 -05:00
Ben Carman
ae6bab857f Increase chain test code coverage even more (#2028) 2020-09-17 16:25:08 -05:00
Ben Carman
16599010c7 Increase chain code coverage (#2023) 2020-09-16 16:24:25 -05:00
Chris Stewart
841230e35c Handle the case where our block headers are synced already (#2021) 2020-09-16 11:45:57 -05:00
Chris Stewart
857b33f31f Make ChainApi.getHeadersBetween() be inclusive on the 'from' parameter (#2009)
* Make ChainApi.getHeadersBetween() be inclusive on the 'from' parameter

* Use ben's suggestion of comparing hashes
2020-09-12 11:13:03 -05:00
Chris Stewart
76b9577181 Introduce 'FilterSyncMarker' to ChainApi, make it clearier what exact… (#2003)
* Introduce 'FilterSyncMarker' to ChainApi, make it clearier what exactly the (Int,DoubleSha256Digest) tuple is returned from ChainApi.nextBlockHeaderRange()

* Fix doc

* Add scaladoc to FilterSyncMarker

* Rebase onto master, fix conflicts to use FilterSyncMarker
2020-09-11 13:48:40 -05:00
Chris Stewart
2980e3fc01 Resolve issues with reorgs and syncing filters (#1969)
* Pull over test cases, WIP

* Rework ChainHandler.nextBlockHeaderRange() to use hashes rather than heights to very integrity of header chain, add test case

* Modify test case to make sure we are generating the 'heaviest chain work' header as first in the tuple that is used in reorg test cases

* Refactor to helper method called 'findNextHeader()' to make things simpler

* Add scaladoc to ChainHandler.findNextHeader()

* WIP

* Fix bugs in ChainApi.nextBlockHeaderRange(), now return (startHeight,stopHash) rather than (stopHeight,stopHash)

* Get chain handler tests passing, remove genesis filter header/fitler from default chain project fixture, make it a specialized one to have genesis filter header/filter in the database. Use that specialized fixture in ChainHandlerTest for now

* FilterSync work without having the genesis filter and filter header inserted into the database

* fix bug in DataMessageHandler where we were using sendNextGetCompactFilterHeadersCommand rather than using peerMsgSender.sendGetCompactFilterHeadersMessage

* Refactor ChainHandler.findNextHeader() to centralize where chains are fetched

* Address ben's code review
2020-09-11 12:41:25 -05:00
rorp
c62be8b5f6 Update callbacks for LN support (#1938)
Update callbacks for LN support
2020-09-09 11:02:31 -07:00
Ben Carman
4c0437e352 Optimize findClosestToTime (#1959)
* Optimize findClosestToTime

* Fix Test

* Rename function, change execption type
2020-09-08 09:23:41 -05:00
Chris Stewart
8b76acd042 Move getBestFilterHeader() use the best chain by chainwork for determining the fitler header (#1964) 2020-09-02 19:41:52 -05:00
Ben Carman
f5fcab9580 Change maxBy and minBys to use maxByOption and minByOption (#1961)
* Change maxBy and minBys to use maxByOption and minByOption

* Fix compile issue
2020-09-02 16:03:57 -05:00
Chris Stewart
98236835a5 Rename ChainApi.nextHeaderBatchRange -> ChainApi.nextBlockHeaderBatchrnage (#1957) 2020-09-02 13:19:20 -05:00
Ben Carman
1670902e75 Fix max by for getBlockCount (#1951)
* Fix max by for getBlockCount

* Add helper func to package
2020-09-01 19:38:15 -05:00
Ben Carman
809ba17891 Fix getBestFilterHeader for headers 2016 or more blocks away from tip (#1943) 2020-08-31 17:00:42 -05:00
Chris Stewart
170242812c Fix maxBy() exception in the case of empty Blockchain in ChainHandler… (#1934)
* Fix maxBy() exception in the case of empty Blockchain in ChainHandler.bestFilterHeaderSearch()

* Add another unit test to CompactFilterHeaderDAO for coverage
2020-08-29 15:29:09 -05:00
Chris Stewart
d3af9c2ccb Implement getBestFilterHeader based on a number of block headers that… (#1926)
* Implement getBestFilterHeader based on a number of block headers that can be passed in as a parameter. These headers can be used to indicate what your current best chain is

* Bring back compiler opts

* Fix compiler error for maxByOption as it isn't in the 2.12 std library

* Implement a context free best filter headers search. The basic strategy is to look at headers in the _future_ of our current best filter header

* Fix bug in sql query were we were doing max chainWork too early on block headers, we needed to filter out headers in our set and _then_ we get the max chain work

* Add more unit tests
2020-08-29 08:13:38 -05:00
Ben Carman
2c3342492c Create DatabaseDriver ADT instead of booleans (#1902) 2020-08-27 12:41:52 -05:00
Ben Carman
3e1c4e2d33 Fix log to output correct function (#1913) 2020-08-27 07:20:58 -05:00
Chris Stewart
3ab280a12b Remove AppConfig.initialize() in favor of AppConfig.start() (#1907)
* Remove AppConfig.initialize() in favor of AppConfig.start()

* Fix docs
2020-08-26 16:24:38 -05:00
rorp
23685f124e Add an ability to one Postgres database for all sub-projects (#1897)
* Add an ability to one Postgres database for all sub-projects
2020-08-26 12:20:18 -07:00
Ben Carman
581851f22c Clean up calls in Node and Chain (#1896)
* Clean up calls in Node and Chain

* Remove parens on non-modifying call
2020-08-26 13:04:34 -05:00
Ben Carman
f468fff7ea Move ChainApi to core (#1888)
* Move Chain Db Reps to core

* Move ChainApi to Core
2020-08-25 07:08:42 -05:00
Scala Steward
bef9c2648e Update scalafmt-core to 2.6.4 (#1686)
* Update scalafmt-core to 2.6.4

* Format everything

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-08-24 14:24:16 -05:00
Ben Carman
2f512364eb Correcly use reference.conf file (#1869)
* Correcly use reference.conf file

* Fix for network specific filter header batch size config option

* Add to other conf files
2020-08-24 06:30:16 -05:00
Ben Carman
f7b9c5722f Use database to calculate number of confirmations (#1789)
* Use database to calculate number of confirmations

* Check block is in best chain

* Use db

* Optimize getNumberOfConfirmations

* Fix compile issue for older versions

* Add better logging messages

* More logs

* Update confs sequentially
2020-08-16 16:15:13 -05:00
Chris Stewart
c9f78a2a40 2020 08 14 issue 1829 (#1833)
* Fix O(n^2) behavior in BaseBlockchain.connectWalkBackwards, cache BlockHeader.hash after the first time it is called

* Revert a few files
2020-08-15 07:31:50 -05:00
Ben Carman
48a7f6f86e Remove default false for useLogbackConf (#1816)
* Remove default false for useLogbackConf

* Fix docs

* Set logging level to WARN for tests
2020-08-14 08:53:07 -05:00
Chris Stewart
ae81dd3bb3 Create more of a project structure in org.bitcoins.core.api, move DbR… (#1799)
* Create more of a project structure in org.bitcoins.core.api, move DbRowAutoInc into the core project

* Get website compiling again
2020-08-12 06:13:23 -05:00
Ben Carman
c3dc52ce90 Send GetHeadersMessage using all of our cached headers to prevent reorgs from stalling node (#1758)
* Send GetHeadersMessage using all of our cached headers to prevent reorgs from stalling node

* Add test, improve logging

* Start with correct chains

* Simplify error, change back to createAll
2020-08-10 13:04:12 -05:00
Ben Carman
deeddf3249 Drop older headers on chain update (#1763) 2020-08-05 14:14:21 -05:00
Ben Carman
34cc2a80ce Add back chain index after creating a temp table (#1753) 2020-08-04 16:57:41 -05:00
Chris Stewart
d39613e9b0 2020 07 25 optimize recalc chainwork (#1697)
* Optimize recalc Chainwork

* Typo and warning fixes

* Rename force recalc cli option

* Always update gensis header

* Add config option

* Fix config option

* Add to example config

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-07-31 10:43:04 -05:00
rorp
8f03551e99 Fix Postgres types (#1723)
* Fix Postgres types

* update DbManagementTest
2020-07-30 16:42:49 -05:00
Ben Carman
870609a406 Optimize getBestFilter functions (#1715)
* Optimize getBestFilter functions

* Remove debug statement

* Change varbinary to varchar, create index

* Fix getBlockchainFrom for tests

* Create index for postgres
2020-07-30 10:47:48 -05:00
Ben Carman
cad6fbeaaf Fix getBlockchainsBetweenHeights (#1710)
* Fix getBlockchainsBetweenHeights

* Fix getBlockchainsBetweenHeights

* Fix compile issue for older version

* Improve test

* Optimizations
2020-07-29 16:36:22 -05:00
Ben Carman
51d35e24e9 Fix Node Startup Issue (#1683)
* Fix Node Startup Issue

* Small fixes

* Add option to force chain work recalc
2020-07-29 09:37:20 -05:00
Ben Carman
f41039b31b Remove need to parse every header in a blockchain on instantiation (#1704)
* Remove need to parse every header in a blockchain on instantiation

* Require height is descending

* Fix invariant
2020-07-29 07:25:24 -05:00
Nadav Kohen
faf2e0148a Fix All Loggers! (#1695)
* Fixed logging so that appenders are always started when logging is happening

* Started rolling policy so that file appenders could start in getLoggerImpl
2020-07-28 07:13:38 -05:00
Ben Carman
b2b9ca7eec Create getBlockchainsBetweenHeights for BlockHeaderDAO (#1703)
* Create getBlockchainsBetweenHeights for BlockHeaderDAO

* Use reconstructFromHeaders instead
2020-07-27 13:14:29 -05:00