Commit Graph

77 Commits

Author SHA1 Message Date
Ben Carman
48a7f6f86e Remove default false for useLogbackConf (#1816)
* Remove default false for useLogbackConf

* Fix docs

* Set logging level to WARN for tests
2020-08-14 08:53:07 -05:00
Chris Stewart
ae81dd3bb3 Create more of a project structure in org.bitcoins.core.api, move DbR… (#1799)
* Create more of a project structure in org.bitcoins.core.api, move DbRowAutoInc into the core project

* Get website compiling again
2020-08-12 06:13:23 -05:00
Ben Carman
c3dc52ce90 Send GetHeadersMessage using all of our cached headers to prevent reorgs from stalling node (#1758)
* Send GetHeadersMessage using all of our cached headers to prevent reorgs from stalling node

* Add test, improve logging

* Start with correct chains

* Simplify error, change back to createAll
2020-08-10 13:04:12 -05:00
Ben Carman
deeddf3249 Drop older headers on chain update (#1763) 2020-08-05 14:14:21 -05:00
Ben Carman
34cc2a80ce Add back chain index after creating a temp table (#1753) 2020-08-04 16:57:41 -05:00
Chris Stewart
d39613e9b0 2020 07 25 optimize recalc chainwork (#1697)
* Optimize recalc Chainwork

* Typo and warning fixes

* Rename force recalc cli option

* Always update gensis header

* Add config option

* Fix config option

* Add to example config

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-07-31 10:43:04 -05:00
rorp
8f03551e99 Fix Postgres types (#1723)
* Fix Postgres types

* update DbManagementTest
2020-07-30 16:42:49 -05:00
Ben Carman
870609a406 Optimize getBestFilter functions (#1715)
* Optimize getBestFilter functions

* Remove debug statement

* Change varbinary to varchar, create index

* Fix getBlockchainFrom for tests

* Create index for postgres
2020-07-30 10:47:48 -05:00
Ben Carman
cad6fbeaaf Fix getBlockchainsBetweenHeights (#1710)
* Fix getBlockchainsBetweenHeights

* Fix getBlockchainsBetweenHeights

* Fix compile issue for older version

* Improve test

* Optimizations
2020-07-29 16:36:22 -05:00
Ben Carman
51d35e24e9 Fix Node Startup Issue (#1683)
* Fix Node Startup Issue

* Small fixes

* Add option to force chain work recalc
2020-07-29 09:37:20 -05:00
Ben Carman
f41039b31b Remove need to parse every header in a blockchain on instantiation (#1704)
* Remove need to parse every header in a blockchain on instantiation

* Require height is descending

* Fix invariant
2020-07-29 07:25:24 -05:00
Nadav Kohen
faf2e0148a Fix All Loggers! (#1695)
* Fixed logging so that appenders are always started when logging is happening

* Started rolling policy so that file appenders could start in getLoggerImpl
2020-07-28 07:13:38 -05:00
Ben Carman
b2b9ca7eec Create getBlockchainsBetweenHeights for BlockHeaderDAO (#1703)
* Create getBlockchainsBetweenHeights for BlockHeaderDAO

* Use reconstructFromHeaders instead
2020-07-27 13:14:29 -05:00
Ben Carman
365d98da10 Add extra logs, fix best filter bug (#1624)
* Add extra logs, fix best filter bug

* Fix to work for when filter count != header count

* Put blockheader items baack in place
2020-07-06 11:38:53 -05:00
Ben Carman
cd594ab069 AppConfig Start Interface (#1598)
* AppConfig Start Interface

* Use StartStop interface
2020-06-21 09:33:43 -05:00
Scala Steward
97a4886c6e Update scalafmt-core to 2.6.0 (#1579)
* Update scalafmt-core to 2.6.0

* Upgrade scalafmt, fix all formatting errors

* Scalafmt on 2.12

* fix test

* More fixes

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-06-19 10:38:12 -05:00
Ben Carman
80930e07c2 Have travis only run tests if it compiles (#1577)
* Have Travis only run tests if it compiles

* Fix warnings + fix compile issue

* Fix compilation issue

* Check formatting in first stage

* Fix name

* Fix formatting error
2020-06-17 14:05:44 -05:00
Nadav Kohen
e5bd6e53a3 Added compile scope scalafmtCheck to travis runs (#1563) 2020-06-16 15:16:43 -05:00
Chris Stewart
6c20a06b45 Change ChainApi.getBestFilterHeader() return type to Future[Option[Co… (#1550)
* Change ChainApi.getBestFilterHeader() return type to Future[Option[CompactFilterHeaderDb]] to resolve issue 1549

* Run scalafmt
2020-06-13 17:01:51 -05:00
Ben Carman
e16ef65b11 Fix compile issue for older versions (#1547) 2020-06-11 15:29:23 -05:00
Ben Carman
137d63d4a4 Fix test failure 2020-06-11 13:41:56 -05:00
Ben Carman
5559475534 Attempt to fix memory leak in recalc chain work (#1535) 2020-06-11 13:12:22 -05:00
Chris Stewart
32c06cbe81 2020 06 05 mv chainwork calc (#1519)
* Fix Chaindb chainwork Migration

* Move chain work calculation out of node, use the ChainApi returnd by runChainWorkCalc()

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-06-05 11:04:09 -05:00
Ben Carman
345e384788 Fix Chaindb chainwork Migration (#1518) 2020-06-05 11:01:42 -05:00
Chris Stewart
3f5b11f3d3 Move slickDbConfig out of JdbcProfileComponent trait into AppConfig (#1510)
* Move slickDbConfig out of JdbcProfileComponent trait into AppConfig

* Cleanup newliens
2020-06-04 12:49:08 -05:00
Ben Carman
61d9f0efba Implement best block hash correctly (#1452)
* Implement best block hash correctly

* Handle no common history

* Fix getBlockProof to be BigInt instead of UInt32

* Fix warnings

* Make get best filter use chainwork

* Remove unused function, test isMissingChainWork

* Fix comparisons for chain work in sql

* Fix migrations from rebase

* Fix CI error

* Fix postgresql
2020-06-04 12:05:10 -05:00
rorp
823a392e98 PostgreSQL support (#1315) 2020-06-02 16:37:48 -05:00
Chris Stewart
d8170f319a Implement AppConfigFactory (#1462)
* Implement AppConfigFactory

* Fix docs
2020-05-25 14:43:37 -05:00
Ben Carman
99db9062ac Add option to correctly use logback config (#1398)
* Logger now uses correct parent logger for all, names appenders correctly

* Set logback to true for tests, rename file

* Fix docs
2020-05-20 15:46:24 -05:00
Nadav Kohen
a503441619 Crypto Project Refactor (#1380)
* Added crypto project and decoupled BitcoinSLogger from NetworkElement

Decoupled BitcoinSLogger from Factory

Moved NetworkElement into crypto project

Moved Factory and BitcoinSUtil (renamed to BytesUtil) to crypto project

Moved MaskedToString to crypto project

Added BytesUtil to imports and cleaned up CryptoUtil.recoverPoint

Moved the rest of crypto stuff to the crypto project

Moved crypto tests to crypto-test project

* Added documentation for crypto project
2020-04-30 12:34:53 -05:00
Ben Carman
fddca91e79 Neutrino Logging (#1382) 2020-04-30 09:05:26 -05:00
Ben Carman
6d7685b76e Add wallet creation time for rescans (#1353)
* Add wallet creation time for rescans

* Fix docs

* Clean up and add test

* Remove account bday

* Fix compile issue and docs

* Add more chain handler tests

* Use Instant over Long, add docs

* Fix docs
2020-04-29 09:49:41 -05:00
Chris Stewart
4c3f6f8e67 2020 04 08 multi db dao (#1355)
* Implement Slick's multi database support via our DAOs

WIP2 work on chain project

WIP make JdbcProfieComponent so we have a common JdbcProfile

WIP2

Clean up a few things

WIP got CRUDAutoInc compiling

Get closer to everything compiling in the chain project

Begin moving database code into JdbcProfileComponent

WIP source compiling

Get dbCommonsTest/ test cases passing

Get rest of tests compiling and passing

* Rebase onto master, fix conflicts

* Run scalafmt

* Run scalafmt again

* Cleanup a few compiler warnings by adding type annotations to thigns

* Explicitly cast tables where they are needed in the wallet project where we need TableQuery for foreignkey / joins in slick

* Redo definition of 'table' to use the profile.api.TableQuery in scope

* Rename all *Table.scala files to *Db.scala

* First crack at address ben's code review

* Fix docs compile issue

* Fixed casting (#30)

* Fixed AppConfig casting

* Fixed TableQuery casting in DbManagement subtypes

* Fixed casts in DAOs relating to internal types

* Address nadav's code review, move JdbcProfileComponent into it's own file

* Remove unused listTables() method

Co-authored-by: Nadav Kohen <nadavk25@gmail.com>
2020-04-25 11:28:58 -05:00
Chris Stewart
cb88b78540 Bump scala version to 2.13.2 (#1360) 2020-04-23 07:38:20 -05:00
Chris Stewart
4f644a2143 Make 'maxHeightQuery' vals inside of BlockHeaderDAO, CompactFilterHeaderDAO, CompactFilterDAO (#1325) 2020-04-09 14:56:45 -05:00
Chris Stewart
bb347890c0 Call getFilterHeaderCount async inside of nextFilterHeaderBatchRange while we are fetching our start height for fetching filters (#1327) 2020-04-09 14:56:03 -05:00
Nadav Kohen
ec0af42ce4 Removed SingleSigner abstraction and replaced with a simple signSingle method in SignerUtils (#1308) 2020-04-07 14:26:07 -05:00
Ben Carman
c2a0051411 Network specific filterHeaderBatchSize (#1286)
* Network specific filterHeaderBatchSize

* Catch correct exceptions

* Add filter-header-batch-size-regtest to example configs
2020-04-03 17:36:29 -05:00
Chris Stewart
de46e74f1a Actually add all files for 0.3.0 on the website so they show up (#1256) 2020-03-19 07:32:02 -05:00
Chris Stewart
44e4ab9a7e 2020 03 15 chainhandler getnancestor (#1247)
* Rename BlockHeaderDAO.getNChildren() to BlockHeaderDAO.getNAncestor() as that is what it actually does

* This changes the behavior of BlockHeaderDAO.getNAncestors from the previous implementation which just fetched headers in between two heights, to actually validating the headers that are fetched from the database with Blockchain.reconstructFromheaders()

* Run scalafmt

* Replace BlockHeader.getNAncestors call to Blockchain.reconstructFromHeaders() with Blockchain.connectWalkBackwards() so that we don't do full POW validation from the database, just make sure the headers connect to avoid reorg problems. Also add some more informative logs

* Address Roman's code review
2020-03-18 05:51:31 -05:00
Chris Stewart
cd84e3483d 2020 03 06 wallet rescan test (#1218)
* Add wallet rescan tests

* Create FilterSync, which gives us an API inside of the chain project to sync filters with

Add another unit test to filter sync

Add more unit tests for ChainSync and FilterSync

Clean up some docs, remove some extra lines of code

Run scalafmt

Add filter-sync.md

Cleanup some nits

Add more information of how FilterSync.syncFilters() works

Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project

Run scalafmt, hide imports in filter-sync.md so code appears cleaner

Move implicits out of invisible block as it seems to cause errors

Make it so FilterSync processes filters in batches rather than fetching them all at once

Fix compile error

* WIP bitcoind implement ChainQueryApi

* rework fixtures to be able to support injecting ChainQueryApi implemented by bitcoind into our fixture infrastructure for creating wallets

* Fix rebase problem

* Implement getFiltersBetweenHeight() with ben's solution

* WIP Start implementing NodeApi against bitcoind

* Actually inject the bitcoind backed nodeApi into our fixture

* Get first rescan test working for rescanning the entire blockchain

* Implement test case for rescanning from a specific height

* Fix NeutrinoNodeWalletTest test case that uses a experimental version of bitcoind, for now i believe our experimental binary is on v18

* Add wallet-rescan.md, add helper method to WalletApi that allows you to clear out the wallet's utxos/addresses

* Add another log to try to debug CI

* Address code review

* Fix wrong ordering of deletion of tables in clearUtxosAndAddresses()

* reset logging level
2020-03-13 16:11:02 -05:00
Chris Stewart
4c9a22f1e1 2020 03 08 filter sync (#1209)
* Create FilterSync, which gives us an API inside of the chain project to sync filters with

Add another unit test to filter sync

Add more unit tests for ChainSync and FilterSync

Clean up some docs, remove some extra lines of code

Run scalafmt

Add filter-sync.md

Cleanup some nits

Add more information of how FilterSync.syncFilters() works

Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project

Run scalafmt, hide imports in filter-sync.md so code appears cleaner

Move implicits out of invisible block as it seems to cause errors

Make it so FilterSync processes filters in batches rather than fetching them all at once

Fix compile error

* Add comment about trust model

* Run scalafmt
2020-03-10 18:01:14 -05:00
Nadav Kohen
052b64b7e2 Seq and Map Wrappers (#1131)
* Renamed size to byteSize in NetworkElement to avoid ambiguity

* Introduced SeqWrapper and MapWrapper to allow for wrapper case classes to have direct access to underlying methods

* Responded to review
2020-02-24 14:39:20 -06:00
Nadav Kohen
7169455eff Got rid of warning messages! (#1133)
* Got rid of warning messages!

* Reverted collection conversion changes because scala 2.11 is dumb
2020-02-19 08:02:33 -06:00
Nadav Kohen
099465f72a Multiple Logger Fix (#1086)
* An attempt to fix logging to file using MDC

* Responded to review
2020-02-11 11:35:28 -06:00
Ben Carman
0c2d324258 CLI command for filter and filter header counts (#1063) 2020-01-23 14:45:33 -06:00
Ben Carman
ed428bd56c Txo state flyway (#1052)
* Add flyway migrations

* Make different project's migrations independent of each other

* Rework all AppConfig.initialize() to use migrations rather than what we were doing before

* TXO State migration

* Move to new file, drop old column

* Add block hash column

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2020-01-22 15:34:36 -06:00
rorp
93c02f02d0 Automated wallet recovery (#985)
* Automated wallet recovery

* responded to the PR comments

* some more changes

* fix docs

* cleanup
2020-01-02 13:18:41 -06:00
rorp
72ead16e5c Move rescan logic from node to wallet (#974)
* Move rescan logic from node to wallet
2019-12-23 08:57:55 -08:00
rorp
2c53a39fd1 Compute confirmations dynamically (#938)
* Compute confirmations dynamically
2019-12-14 12:06:22 -08:00