Commit graph

70 commits

Author SHA1 Message Date
Ben Carman
728a4a841f
Fix Node Startup Issue (#1683)
* Fix Node Startup Issue

* Small fixes

* Add option to force chain work recalc
2020-07-29 09:37:20 -05:00
Ben Carman
383aaa7639
Remove need to parse every header in a blockchain on instantiation (#1704)
* Remove need to parse every header in a blockchain on instantiation

* Require height is descending

* Fix invariant
2020-07-29 07:25:24 -05:00
Nadav Kohen
436396773d
Fix All Loggers! (#1695)
* Fixed logging so that appenders are always started when logging is happening

* Started rolling policy so that file appenders could start in getLoggerImpl
2020-07-28 07:13:38 -05:00
Ben Carman
4b710dd0ca
Create getBlockchainsBetweenHeights for BlockHeaderDAO (#1703)
* Create getBlockchainsBetweenHeights for BlockHeaderDAO

* Use reconstructFromHeaders instead
2020-07-27 13:14:29 -05:00
Ben Carman
8f3bcbb949
Add extra logs, fix best filter bug (#1624)
* Add extra logs, fix best filter bug

* Fix to work for when filter count != header count

* Put blockheader items baack in place
2020-07-06 11:38:53 -05:00
Ben Carman
102e5775f7
AppConfig Start Interface (#1598)
* AppConfig Start Interface

* Use StartStop interface
2020-06-21 09:33:43 -05:00
Scala Steward
e6d1471af0
Update scalafmt-core to 2.6.0 (#1579)
* Update scalafmt-core to 2.6.0

* Upgrade scalafmt, fix all formatting errors

* Scalafmt on 2.12

* fix test

* More fixes

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-06-19 10:38:12 -05:00
Ben Carman
105052d877
Have travis only run tests if it compiles (#1577)
* Have Travis only run tests if it compiles

* Fix warnings + fix compile issue

* Fix compilation issue

* Check formatting in first stage

* Fix name

* Fix formatting error
2020-06-17 14:05:44 -05:00
Nadav Kohen
2b91c39616
Added compile scope scalafmtCheck to travis runs (#1563) 2020-06-16 15:16:43 -05:00
Chris Stewart
94568aba22
Change ChainApi.getBestFilterHeader() return type to Future[Option[Co… (#1550)
* Change ChainApi.getBestFilterHeader() return type to Future[Option[CompactFilterHeaderDb]] to resolve issue 1549

* Run scalafmt
2020-06-13 17:01:51 -05:00
Ben Carman
4e037b5cc4
Fix compile issue for older versions (#1547) 2020-06-11 15:29:23 -05:00
Ben Carman
9422b325e5
Fix test failure 2020-06-11 13:41:56 -05:00
Ben Carman
7d970eaf13
Attempt to fix memory leak in recalc chain work (#1535) 2020-06-11 13:12:22 -05:00
Chris Stewart
c54ee10809
2020 06 05 mv chainwork calc (#1519)
* Fix Chaindb chainwork Migration

* Move chain work calculation out of node, use the ChainApi returnd by runChainWorkCalc()

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-06-05 11:04:09 -05:00
Ben Carman
43a5c6c05f
Fix Chaindb chainwork Migration (#1518) 2020-06-05 11:01:42 -05:00
Chris Stewart
8c0e64e101
Move slickDbConfig out of JdbcProfileComponent trait into AppConfig (#1510)
* Move slickDbConfig out of JdbcProfileComponent trait into AppConfig

* Cleanup newliens
2020-06-04 12:49:08 -05:00
Ben Carman
31807c9cbd
Implement best block hash correctly (#1452)
* Implement best block hash correctly

* Handle no common history

* Fix getBlockProof to be BigInt instead of UInt32

* Fix warnings

* Make get best filter use chainwork

* Remove unused function, test isMissingChainWork

* Fix comparisons for chain work in sql

* Fix migrations from rebase

* Fix CI error

* Fix postgresql
2020-06-04 12:05:10 -05:00
rorp
a9430c2d5a
PostgreSQL support (#1315) 2020-06-02 16:37:48 -05:00
Chris Stewart
0a35cdb6ef
Implement AppConfigFactory (#1462)
* Implement AppConfigFactory

* Fix docs
2020-05-25 14:43:37 -05:00
Ben Carman
c811ccc62a
Add option to correctly use logback config (#1398)
* Logger now uses correct parent logger for all, names appenders correctly

* Set logback to true for tests, rename file

* Fix docs
2020-05-20 15:46:24 -05:00
Nadav Kohen
4d9692f61e
Crypto Project Refactor (#1380)
* Added crypto project and decoupled BitcoinSLogger from NetworkElement

Decoupled BitcoinSLogger from Factory

Moved NetworkElement into crypto project

Moved Factory and BitcoinSUtil (renamed to BytesUtil) to crypto project

Moved MaskedToString to crypto project

Added BytesUtil to imports and cleaned up CryptoUtil.recoverPoint

Moved the rest of crypto stuff to the crypto project

Moved crypto tests to crypto-test project

* Added documentation for crypto project
2020-04-30 12:34:53 -05:00
Ben Carman
38fe580c06
Neutrino Logging (#1382) 2020-04-30 09:05:26 -05:00
Ben Carman
556f713589
Add wallet creation time for rescans (#1353)
* Add wallet creation time for rescans

* Fix docs

* Clean up and add test

* Remove account bday

* Fix compile issue and docs

* Add more chain handler tests

* Use Instant over Long, add docs

* Fix docs
2020-04-29 09:49:41 -05:00
Chris Stewart
64b13846d9
2020 04 08 multi db dao (#1355)
* Implement Slick's multi database support via our DAOs

WIP2 work on chain project

WIP make JdbcProfieComponent so we have a common JdbcProfile

WIP2

Clean up a few things

WIP got CRUDAutoInc compiling

Get closer to everything compiling in the chain project

Begin moving database code into JdbcProfileComponent

WIP source compiling

Get dbCommonsTest/ test cases passing

Get rest of tests compiling and passing

* Rebase onto master, fix conflicts

* Run scalafmt

* Run scalafmt again

* Cleanup a few compiler warnings by adding type annotations to thigns

* Explicitly cast tables where they are needed in the wallet project where we need TableQuery for foreignkey / joins in slick

* Redo definition of 'table' to use the profile.api.TableQuery in scope

* Rename all *Table.scala files to *Db.scala

* First crack at address ben's code review

* Fix docs compile issue

* Fixed casting (#30)

* Fixed AppConfig casting

* Fixed TableQuery casting in DbManagement subtypes

* Fixed casts in DAOs relating to internal types

* Address nadav's code review, move JdbcProfileComponent into it's own file

* Remove unused listTables() method

Co-authored-by: Nadav Kohen <nadavk25@gmail.com>
2020-04-25 11:28:58 -05:00
Chris Stewart
ce33c57ed9
Bump scala version to 2.13.2 (#1360) 2020-04-23 07:38:20 -05:00
Chris Stewart
efaf457d6a
Make 'maxHeightQuery' vals inside of BlockHeaderDAO, CompactFilterHeaderDAO, CompactFilterDAO (#1325) 2020-04-09 14:56:45 -05:00
Chris Stewart
27dbefd2af
Call getFilterHeaderCount async inside of nextFilterHeaderBatchRange while we are fetching our start height for fetching filters (#1327) 2020-04-09 14:56:03 -05:00
Nadav Kohen
19afc6be2c
Removed SingleSigner abstraction and replaced with a simple signSingle method in SignerUtils (#1308) 2020-04-07 14:26:07 -05:00
Ben Carman
aa2d88f396
Network specific filterHeaderBatchSize (#1286)
* Network specific filterHeaderBatchSize

* Catch correct exceptions

* Add filter-header-batch-size-regtest to example configs
2020-04-03 17:36:29 -05:00
Chris Stewart
4559edde07
Actually add all files for 0.3.0 on the website so they show up (#1256) 2020-03-19 07:32:02 -05:00
Chris Stewart
800fdffffb
2020 03 15 chainhandler getnancestor (#1247)
* Rename BlockHeaderDAO.getNChildren() to BlockHeaderDAO.getNAncestor() as that is what it actually does

* This changes the behavior of BlockHeaderDAO.getNAncestors from the previous implementation which just fetched headers in between two heights, to actually validating the headers that are fetched from the database with Blockchain.reconstructFromheaders()

* Run scalafmt

* Replace BlockHeader.getNAncestors call to Blockchain.reconstructFromHeaders() with Blockchain.connectWalkBackwards() so that we don't do full POW validation from the database, just make sure the headers connect to avoid reorg problems. Also add some more informative logs

* Address Roman's code review
2020-03-18 05:51:31 -05:00
Chris Stewart
5bb31f4951
Fix scala 2.11 compile (#1233) 2020-03-14 08:18:26 -05:00
Chris Stewart
c7f8ab72cf
2020 03 06 wallet rescan test (#1218)
* Add wallet rescan tests

* Create FilterSync, which gives us an API inside of the chain project to sync filters with

Add another unit test to filter sync

Add more unit tests for ChainSync and FilterSync

Clean up some docs, remove some extra lines of code

Run scalafmt

Add filter-sync.md

Cleanup some nits

Add more information of how FilterSync.syncFilters() works

Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project

Run scalafmt, hide imports in filter-sync.md so code appears cleaner

Move implicits out of invisible block as it seems to cause errors

Make it so FilterSync processes filters in batches rather than fetching them all at once

Fix compile error

* WIP bitcoind implement ChainQueryApi

* rework fixtures to be able to support injecting ChainQueryApi implemented by bitcoind into our fixture infrastructure for creating wallets

* Fix rebase problem

* Implement getFiltersBetweenHeight() with ben's solution

* WIP Start implementing NodeApi against bitcoind

* Actually inject the bitcoind backed nodeApi into our fixture

* Get first rescan test working for rescanning the entire blockchain

* Implement test case for rescanning from a specific height

* Fix NeutrinoNodeWalletTest test case that uses a experimental version of bitcoind, for now i believe our experimental binary is on v18

* Add wallet-rescan.md, add helper method to WalletApi that allows you to clear out the wallet's utxos/addresses

* Add another log to try to debug CI

* Address code review

* Fix wrong ordering of deletion of tables in clearUtxosAndAddresses()

* reset logging level
2020-03-13 16:11:02 -05:00
Chris Stewart
6d6b1023c5
2020 03 08 filter sync (#1209)
* Create FilterSync, which gives us an API inside of the chain project to sync filters with

Add another unit test to filter sync

Add more unit tests for ChainSync and FilterSync

Clean up some docs, remove some extra lines of code

Run scalafmt

Add filter-sync.md

Cleanup some nits

Add more information of how FilterSync.syncFilters() works

Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project

Run scalafmt, hide imports in filter-sync.md so code appears cleaner

Move implicits out of invisible block as it seems to cause errors

Make it so FilterSync processes filters in batches rather than fetching them all at once

Fix compile error

* Add comment about trust model

* Run scalafmt
2020-03-10 18:01:14 -05:00
Nadav Kohen
5ccd01ac77
Seq and Map Wrappers (#1131)
* Renamed size to byteSize in NetworkElement to avoid ambiguity

* Introduced SeqWrapper and MapWrapper to allow for wrapper case classes to have direct access to underlying methods

* Responded to review
2020-02-24 14:39:20 -06:00
Nadav Kohen
d215680e74
Got rid of warning messages! (#1133)
* Got rid of warning messages!

* Reverted collection conversion changes because scala 2.11 is dumb
2020-02-19 08:02:33 -06:00
Nadav Kohen
a6e21fe43b
Multiple Logger Fix (#1086)
* An attempt to fix logging to file using MDC

* Responded to review
2020-02-11 11:35:28 -06:00
Ben Carman
c968e79c80 CLI command for filter and filter header counts (#1063) 2020-01-23 14:45:33 -06:00
Ben Carman
c4ade3ba3c Txo state flyway (#1052)
* Add flyway migrations

* Make different project's migrations independent of each other

* Rework all AppConfig.initialize() to use migrations rather than what we were doing before

* TXO State migration

* Move to new file, drop old column

* Add block hash column

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2020-01-22 15:34:36 -06:00
rorp
d776e1c952 Automated wallet recovery (#985)
* Automated wallet recovery

* responded to the PR comments

* some more changes

* fix docs

* cleanup
2020-01-02 13:18:41 -06:00
rorp
a31066d17e
Move rescan logic from node to wallet (#974)
* Move rescan logic from node to wallet
2019-12-23 08:57:55 -08:00
rorp
b92fc1cb8f
Compute confirmations dynamically (#938)
* Compute confirmations dynamically
2019-12-14 12:06:22 -08:00
rorp
7cfd33b753 ChainQueryApi (#926)
* ChainQueryApi

* cleanup
2019-12-09 09:39:35 -06:00
rorp
804f18f47f Refactor compact filter matching (#838)
* Refactor compact filter matching

* cleanup

* some more changes

* addressed the PR comments

* cleanup

* cleanup
2019-11-26 13:40:19 -06:00
rorp
43ec85af39 Rescan RPC (#854)
* Rescan RPC

* cleanup

* fix optional args

* cleanup

* unit tests

* fix build error

* fix unit tests

* scalaMock

* cleanup

* more unit tests

* some more validation checks
2019-11-21 08:03:32 -06:00
rorp
abaa0581c0 Rescan and fetch blocks (#835)
* Rescan and fetch blocks

* unit test

* fix log level

* addessed the PR comments

* fix test timeout

* improve unit tests
2019-10-30 09:51:03 -05:00
rorp
ba91c6107c Create removed Neutrino tables (#796) 2019-10-14 14:42:59 -05:00
rorp
3f734a95db Make ChainApi to scan the block filters in order to find matches (#786)
* Make ChainApi to scan the block filters in order to find matches

* performance improvements, unit test

* some parallelization

* addressed comments

* parallelism level

* fix parallelism computation

* Scala 2.11 compatibility

* increased test coverage

* cleanup

* more checks and cleanup

* change ChainApi.getMatchingBlocks signature

* some more changes
2019-10-11 14:19:42 -05:00
rorp
5cc0b30544 Optimize org.bitcoins.chain.blockchain.BaseBlockChain (#781)
* Optimize org.bitcoins.chain.blockchain.BaseBlockChain

* remove toStream calls

* cleanup

* fix IndexOutOfBoundsException
2019-10-06 09:47:02 -05:00
Chris Stewart
6476e3405d
Add a log at INFO level to indicate we are making progress while syncing (#780)
* Add a log at INFO level to indicate we are making progress while syncing

* Adjust sync log message to INFO in p2p code

* Fix off by one error

* Run scalafmt
2019-10-03 14:57:02 -05:00