* Fix maxBy() exception in the case of empty Blockchain in ChainHandler.bestFilterHeaderSearch()
* Add another unit test to CompactFilterHeaderDAO for coverage
* Implement getBestFilterHeader based on a number of block headers that can be passed in as a parameter. These headers can be used to indicate what your current best chain is
* Bring back compiler opts
* Fix compiler error for maxByOption as it isn't in the 2.12 std library
* Implement a context free best filter headers search. The basic strategy is to look at headers in the _future_ of our current best filter header
* Fix bug in sql query were we were doing max chainWork too early on block headers, we needed to filter out headers in our set and _then_ we get the max chain work
* Add more unit tests
* Clean up code for BroadcastDAO test
* Extend StartStopAsync with BitcoinSAppConfig, create 'CachedAppConfig' test trait, clean up P2PClientTest
* Make MerkleBuffersTest use CachedAppConfig
* Start cleaning up after ourselves in the chainTest test suite
* Cancel background runnable for generating blocks after test case, add nodeAppConfig.stop() to tests in NodeAppConfigTest
* Stop mainnet nodeAppConfig in test as per ben's code review
* Refactor ChainUnitTest.withChainFixture() to use makeDependentFixture()
* Call .stop() for appConfig's spun up in ChainAppConfigTest
* Less parallelization on destruction of chainAppConfig
* Revert parallelExecution/fork jvm in CommonSettings
* Clean up code for BroadcastDAO test
* Extend StartStopAsync with BitcoinSAppConfig, create 'CachedAppConfig' test trait, clean up P2PClientTest
* Make MerkleBuffersTest use CachedAppConfig
* Start cleaning up after ourselves in the chainTest test suite
* Cancel background runnable for generating blocks after test case, add nodeAppConfig.stop() to tests in NodeAppConfigTest
* Stop mainnet nodeAppConfig in test as per ben's code review
* Use database to calculate number of confirmations
* Check block is in best chain
* Use db
* Optimize getNumberOfConfirmations
* Fix compile issue for older versions
* Add better logging messages
* More logs
* Update confs sequentially
* Send GetHeadersMessage using all of our cached headers to prevent reorgs from stalling node
* Add test, improve logging
* Start with correct chains
* Simplify error, change back to createAll
* Update scalafmt-core to 2.6.0
* Upgrade scalafmt, fix all formatting errors
* Scalafmt on 2.12
* fix test
* More fixes
Co-authored-by: Ben Carman <benthecarman@live.com>
* Implement best block hash correctly
* Handle no common history
* Fix getBlockProof to be BigInt instead of UInt32
* Fix warnings
* Make get best filter use chainwork
* Remove unused function, test isMissingChainWork
* Fix comparisons for chain work in sql
* Fix migrations from rebase
* Fix CI error
* Fix postgresql
* Added crypto project and decoupled BitcoinSLogger from NetworkElement
Decoupled BitcoinSLogger from Factory
Moved NetworkElement into crypto project
Moved Factory and BitcoinSUtil (renamed to BytesUtil) to crypto project
Moved MaskedToString to crypto project
Added BytesUtil to imports and cleaned up CryptoUtil.recoverPoint
Moved the rest of crypto stuff to the crypto project
Moved crypto tests to crypto-test project
* Added documentation for crypto project
* Add wallet creation time for rescans
* Fix docs
* Clean up and add test
* Remove account bday
* Fix compile issue and docs
* Add more chain handler tests
* Use Instant over Long, add docs
* Fix docs
* Rename BlockHeaderDAO.getNChildren() to BlockHeaderDAO.getNAncestor() as that is what it actually does
* This changes the behavior of BlockHeaderDAO.getNAncestors from the previous implementation which just fetched headers in between two heights, to actually validating the headers that are fetched from the database with Blockchain.reconstructFromheaders()
* Run scalafmt
* Replace BlockHeader.getNAncestors call to Blockchain.reconstructFromHeaders() with Blockchain.connectWalkBackwards() so that we don't do full POW validation from the database, just make sure the headers connect to avoid reorg problems. Also add some more informative logs
* Address Roman's code review
* Create FilterSync, which gives us an API inside of the chain project to sync filters with
Add another unit test to filter sync
Add more unit tests for ChainSync and FilterSync
Clean up some docs, remove some extra lines of code
Run scalafmt
Add filter-sync.md
Cleanup some nits
Add more information of how FilterSync.syncFilters() works
Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project
Run scalafmt, hide imports in filter-sync.md so code appears cleaner
Move implicits out of invisible block as it seems to cause errors
Make it so FilterSync processes filters in batches rather than fetching them all at once
Fix compile error
* Add comment about trust model
* Run scalafmt
* Make ChainApi to scan the block filters in order to find matches
* performance improvements, unit test
* some parallelization
* addressed comments
* parallelism level
* fix parallelism computation
* Scala 2.11 compatibility
* increased test coverage
* cleanup
* more checks and cleanup
* change ChainApi.getMatchingBlocks signature
* some more changes
* Create base traits for async tests called BaseAsyncTest, and then create two subtypes, BitcoinSAsyncTest and BitcoinSAsyncFixtureTest to make it easier to manage resources related to test cases. Now it is easier to shutdown things like an actor system in a structured manner
* Remove two bitcoind tests that don't make sense, 'duration' in the parameter is _really_ an interval (bad naming by usTesting an interval does not make sense inside of TestRpcUtilTest
* Turn log off for ChainAppConfigTest we expect to fail
* Bump timeout for BitcoinSUnitTest, use builtint sbt scalafmtCheck
* Remove hardcoded scalafmt binary
* reduce generator size to attempt to fix#768
* Remove BlockchainBuilder, speed up ChainHandler test that processes mainnet headers, add test for ChainAppConfig
* Add test for BlockHeaderDAO.getAncestorAtHeight()
* Consolidate ChainAppConfigTest, add ScalaTestUtil to help with working with scalatest, speed up ChainHandlerTest a bit more
* WIP: implement processHeaders() in ChainHandler instead of processHeader()
* Create ConnectTipResult which is returned from Blockchain.connectTip(), this decouples 'BlockchainUpdate' and Blockchain.connectTip(). This was needed because it doesn't make sense to have a Vector[BlockHeaderDb] returned from Blockchain.connectTip() as we are validating one header. However, to execute batch inserts, we need to accumulate headers somewhere. I choose to do this in BlockchainUpdate with the 'successfulHeaders' field.
* make chain-verification log level INFO
* Add unit test for benchmarking ChainHandler.processHeaders() that cuts out extraneous noise in another test case
* Address code review
* WIP: De-futurify TipValidatoin.chewNewTip()
* De-futrify POW and TipValidation, now tip connection is done synchronously thanks to our in memory blockchain implementation
* Fix improt issues, unused parameters
* Start the process of refactoring our ChainHandler to be able to avoid database calls on TipValidation
WIP: Begin explicity passing state back and forth in return types of PeerMessageReceiver, P2PClient, , DataMessageHandler. This commit also implements the ability to keep our blockchain completely in memory. Previously when we were updating the tip of the chain, we had to make a database read to figure out what the best tips are. This is suboptimal for performance because a database read needs to be done for every block header we see, now we just keep the chain in memory
Fix bug in DataMessageHandler that pre-emptively sent a getheadersmsg to our peer. Make 'chainApiF' internal to our spvNode (not a parameter). This forces the chainApi to be created from disk everytime a new SpvNode is spun up. This keeps us in sync with the blockchain at disk at the cost of disk access and less modularity of SpvNode
Address torkel code review
Fix rebase issues
Address code review
Address nadav code review
* Rebase onto master, fix api changes
* Add logging to data directory
In this commit we add the ability for the node, chain
and wallet projects (+ the server) to log to the users
data directory instead of whatever directory the
binaries was launched from. This is inherently a bit
more complicated than our previous setup, because
we need to read the user directory before we can create
loggers. As a result of this, some files/methods were
moved around, so the relevant app config could be
found in scope.
We also introduce several logging categories that can be
tuned individually through user configuration. These logggers
are exposed both as traits that give a field `logger`, or
as methods that return the required logger.
* Add datadir configuration to AppConfig
In this commit we add support for AppConfig to pick up
the data directory configuration file. We also add
a section to the contributing guide on how to tune
logging levels.
* Pass data directories explicitly for configuration
* Implement missing branch on POW validation for testnet, implement BlockHeaderDAO.find
* implement second special case for if we are allowing min difficulty blocks, but are at a POW change interval. If we are at a POW change interval, we return that block
* Address code review
* PoC bitcoin-s-cli
* Add CLI, Server sbt projects, remove Ammonite
In this commit we set up sbt configuration for
CLI, Server (in-work-name) and corresponding
test projects.
We also remove Ammonite shell from sbt, as that
isn't really being used. bloop console offers
the same functionality way more ergonimic.
* Move BitcoinSAppConfig into new server project
Server project depends on node, chain wand wallet
so this is a good time for introducing this class
into main sources. We also introduce
BitcoinSTestAppConfig in testkit, to replace the
functionality in BitcoinSAppConfig related to
tests.
* Type chain in blockchainresult
* MVP server setup for node, chain and wallet
* Extremely dirty CLI for interacting with server
* initial attempt at mimicking Bitcoin Core API
* WalletStorage: add method for checking for seed existance
* Check for seed existance on wallet startup
* Fix bug where MnemonicNotFound was not an error
* Segregate confirmed and unconfirmed balance methods
* Add error handling, improve formatting of CLI output
* Tweak build
Bump Sttp version, downgrade to uPickle 2.11 compat,
skip publish in cli-test and server-test
* Add CLI, server and picklers to root project
* Implement 'IndexedSeqLike' on Blockchain data structure to give us access to the scala collection methods we know and love
* Implement ability to handle reorgs in Blockchain.connectTip()
* Add another test case for where block B and C are at the same height as each other, with B being the best block hash. When block D comes in, it is built on top of block C so that means our best block hash should switch to C and then to D after D is connected
* Add more unit tests to Blockchain reorg handling
* Address torkel's code review
This is due to issues with deadlocks with
in-memory based databases. We need DBs
to stay alive between connections, but
not across tests.
In DB intensive chain validation tests we
use in-memory databases instead. This
seems like a reasonable tradeoff between
simplicity and speed.
* Replace AppConfig with more specific types
* Rework database configuration
In this commit we:
1) Introduce new database settings for all submodules,
and at the same time remove the singular datbase setting
for the entire project
2) Introduce a new method .initialize on all configs
that is responsible for creating needed directories,
databases and files
3) Introduce a new class BitcoinSAppConfig that wraps
configs for all three other subproject config. We
also provide implicit conversions that enable this
super-config to be passed in wherever a specialized
config is required.
4) Add more tests for our configuration setup.
* Add Ammonite to Docs deps
