* Create CachedBitcoind, implement it in FundTransactionHandlingTest
* Add BaseWalletTest, extend it with BitcoinSWalletTest & BitcoinSWalletTestCachedBitcoind, add CachedBitcoinV19 and use it RescanHandlingTest
* Make ProcessBlockTest work with cached bitcoind
* Make trait for CachedBitcoindNewest for the newest version of bitcoind
* Make UTXOLifeCycleTest use cached bitcoind
* Add WalletBloom, WalletSyncTest to use cached bitcoinds
* Add WalletIntegrationTest
* Rework beforeAll() and afterAll() into the super trait like BaseWalletTest
* Add standlone BitcoindFixtures, use it in BitcoindBackendTest
* Use new BitcoindFixtures in BitcoindBlockPollingTest
* Introduce BaseNodeTest, start implementing the usage of cached bitcoinds in the nodeTest project
* Use cached bitcoind's with SpvNodeTest & SpvNodeWithWalletTest
* Fix bug on postgres with reusing database, upsert the genesis header rather than create it
* Get NeutrinoNode tests workign with cached bitcoinds
* Fix NeutrinoNodeWithWallet by destroying wallet state for Postgres
* Add teardown helper method for bitcoind
* Teardown chain project when using node fixtures since node is dependent upon the chain project.
* Turn off parallelExecution again
* Switch the parallelExecution flag to only be set on CI, so we can get better performance when running locally
* Start implementing BitcoindFixtures, use BitcoindFixturesCachedTriple on TestUtilRpcTest
* Fix compiler errors, begin implementing NodePair
* Refactor TestRpcUtilTest to use 2 bitcoinds rather than 2
* Reduce the number of bitcoinds that MultiWalletRpcTest needs from 3 -> 1
* Reduce number of bitcoinds used in WalletRpcTest from 3 -> 2
* Add some documentation
* Try to re-add parallelExecution
* Reduce the number of bitcoinds used in PsbtRpcTest from 3 -> 2
* Disable parallelExecution in Test again
* Make BitcoindV21RpcClientTest & BitcoindV20RpcClientTest reduce bitcoind usage from 2 -> 1
* Make BitcoindV19RpcClienttest reduce bitcoind usage from 2 -> 1
* Rework MempoolRpcTest to use fixtures, add BitcoindVersion to CachedBitcoindCollection
* Make sure clientAccumm has to be specified as a paramter now rather than filling in by default
* Begin parameterizing NodePair/NodeTriple to retain type information for the specific version of bitcoind that was used
* Don't implement version in super trait
* Fix docs
* Fix async issue in V21 test suite
* Append to vectors in CachedBitcoinCollection rather than replace
* Fix rebase issues
* Add scaladocs
* Fix BitcoindV18RpcClient address info test
* Implement fixtures in BitcoindV17RpcClientTest fixtures
* Cleanup v17 PsbtRpcTest
* Reduce bitcoind usage from 3 -> 1 in BitcoindV18RpcClientTest
* Remove abandon transaction test, this allows us to reduce the number of bitcoind's used in MempoolRpcTest from 3 -> 2
* Remove the requirement to inject BitcoinSAsyncFixtureTest, add it in the test traits explicitly to make things easier. Also add explicit afterAll() method to tear down both the CachedBitcoind & BitcoinSAsyncFixtureTest
* Fix missing Await.result() in BitcoindRpcTest.afterAll()
* Rework MultiWalletRpcTest to use a NodePair
* Rework BlockchainRpcTest to use fixtures
* Rework Client start()/stop() methods. Now use an AtomicBoolean to indicate when a user has requested a client to start/stop rather than sending pings to bitcoind that can fail because the conneciton pool has been shutdown in test cases
* Try my luck with turning on parallelExecution in CI again
* Revert parallelExecution, now testes do not run in parallel on CI
* Only turn off parallelExecution for bitcoindRpcTest
* Adjust build to only have bitcoindRpcTest NOT in run parallel on mac, reduce number of blocks used in BitcoindRpcTestUtil.createNodeSequence
* Run less tests in the rpc test suite as that takes the longest, move them over to node/wallet/dlc test suite on mac osx CI
* Don't run eclair tests in parallel either
* Remove CachedBitcoind from BitcoinSWalletTest
* Fix async bug in test case
* Push to github to force re-run of CI
* Push to github to force re-run of CI
* Push to github to force re-run of CI
* Refactor starting second bitcoind in MempoolRpcTest, remove Thread.sleep
* Add AkkaUtil.nonBlockingSleep(), replace usages of Thread.sleep() with it where possible
* Remove more Thread.sleep()
* Try bumping threadpool for scheduler
* Reduce scheduler threadpool size to 2
* Push to github to force re-run of CI
* Wrap entire Client.getPayload() into try catch to avoid exceptions leaking
* Use akka's Unmarshal to get response payload
* Catch all non fatal exceptions in BitcoindInstance unit test
* Refactor AsyncUtil to just use an execution context rather than an actor system
* Create testkit-core, start moving data structures over to testkit-core
* WIP, getting rid of the hard dependency on BitcoinSAsyncTest and akka
* Rework test traits to not be dependnent on akka, move BaseAsyncTest into testkitCore, add a test trait that just uses the default scala executionContext
* Get everything compiling
* Move logback-test.xml to testkit-core so it applies to coreTest,cryptoTest as well
* Cleanup rebase issues
* Fix Deps.scala
* Address ben's code review
* Segregate jvm settings and compiler settings
* Filter out -Xfatal-warning for scalajs for now since there is a ton of errors
* Move maintainer settings into CommonSettings.settings
* Move CryptoParams to shared crypto project, make BouncyCastleCryptoParams for bouncy castle specific things
* Rework build structure to work better with scalajs
* Add unit test to make sure CryptoParams & BouncyCastleCryptoParams are consistent
* Refactor AsyncUtil to just use an execution context rather than an actor system
* Create async util project, use it in various projects that depend on it
* Bitcoind v0.21.0rc3 support
* Use v0.21.0 instead of Experimental in node tests
* Bump to rc5
* Use official release
* Fix rebase issues
* Remove DoNotDiscover tag
* Fix count for NeutrinoNodeTest
* Fix bug where test compiler opts were not enabled
* Fix all exsting compiler warnings in test project
* Add scalaCollectionCompat dependency to bitcoindRpcTest so we can use @nowarn annotation on scala 2.12
* Start refactoring testkit to allow for specifying a different binary directory than the sbt default
* Fix BitcoindRpcTestUtil
* Add BitcoindRpcTestClient
* Update scalafmt-core to 2.6.0
* Upgrade scalafmt, fix all formatting errors
* Scalafmt on 2.12
* fix test
* More fixes
Co-authored-by: Ben Carman <benthecarman@live.com>
* Implement best block hash correctly
* Handle no common history
* Fix getBlockProof to be BigInt instead of UInt32
* Fix warnings
* Make get best filter use chainwork
* Remove unused function, test isMissingChainWork
* Fix comparisons for chain work in sql
* Fix migrations from rebase
* Fix CI error
* Fix postgresql
* Try and improve reliability of UpdateBloomFilterTest, also add getMemPoolEntryOpt() that returns an Option indicating if a tx was in the mempool
* Fix nits
* Added crypto project and decoupled BitcoinSLogger from NetworkElement
Decoupled BitcoinSLogger from Factory
Moved NetworkElement into crypto project
Moved Factory and BitcoinSUtil (renamed to BytesUtil) to crypto project
Moved MaskedToString to crypto project
Added BytesUtil to imports and cleaned up CryptoUtil.recoverPoint
Moved the rest of crypto stuff to the crypto project
Moved crypto tests to crypto-test project
* Added documentation for crypto project
* Renamed size to byteSize in NetworkElement to avoid ambiguity
* Introduced SeqWrapper and MapWrapper to allow for wrapper case classes to have direct access to underlying methods
* Responded to review
* bitcoind v19 new RPC calls and tests (#863)
* bitcoind v19 new RPC calls and tests
* Code review changes
* Review part 2
* Rename variable to be more descriptive
* Explanitory comment
* Ignore broken test cases
* Add missing signing functions
* Add test to check avoid_reuse flag is on (#870)
* Add test to check avoid_reuse flag is on
* Add test to make sure flags weren't set
* bitcoind v19 Update mempool RPCs and tests (#868)
* Update mempool RPC calls to bitcoind v19 compatibility
* Typo fix
* Add parameter name to calls
* Fix remaining rpc calls
* Formatting
* scaladoc for param
* Change param to correct type
* Clarify on scaladoc
* Add missing fees parmater to mempool rpcs (#875)
* Add weight field to mempool entries after v19 (#876)
* Move DescriptorRpc to be able to be used by future versions of bitcoind (#878)
* Add window_final_block_height to GetChainTxStatsResult (#880)
* Add passphrase argument to createwallet for later versions (#883)
* Add passphrase argument to createwallet for later versions
* Scaladoc + empty passphrase requirement
* Error message
* Add new services names parameter to P2P rpcs (#874)
* Add new services names parameter to P2P rpcs
* Add ServiceIdentifier Reads
* Add fallback case
* Address review
* Change to Try
* Move PsbtRpc to be able to be used by future versions of bitcoind (#877)
* Move PsbtRpc to be able to be used by future versions of bitcoind
* Add test
* Address comment
* Enable bloom filters for v19
* Enable bip 61 for tests
* Change to official binaries
* Force v18 for Spv Tests
* Remove unused config line
* Create base traits for async tests called BaseAsyncTest, and then create two subtypes, BitcoinSAsyncTest and BitcoinSAsyncFixtureTest to make it easier to manage resources related to test cases. Now it is easier to shutdown things like an actor system in a structured manner
* Remove two bitcoind tests that don't make sense, 'duration' in the parameter is _really_ an interval (bad naming by usTesting an interval does not make sense inside of TestRpcUtilTest
* Turn log off for ChainAppConfigTest we expect to fail
* Bump timeout for BitcoinSUnitTest, use builtint sbt scalafmtCheck
* Remove hardcoded scalafmt binary
* reduce generator size to attempt to fix#768
* Update scalafmt to 2.0.1
* Scalafmt
In this commit we
1) Reconfigure the .scalafmt.conf file
2) Add a scalafmt binary (same version as specified in conf file)
3) Use said binary on CI to check that everything is formatted correctly
* Ran scalafmt
* Update scalafmt ignore
* Add scalafmt plugin to project/plugins.sbt, run it
* run scalafmt
* This is for colin
* Fix remaining errors on V18 RPC calls
* remove unused code final formatting fix
* implemented fixed listwalletdir rpc test
* responding to code review and replacing flatmaps
* Remove BlockchainBuilder, speed up ChainHandler test that processes mainnet headers, add test for ChainAppConfig
* Add test for BlockHeaderDAO.getAncestorAtHeight()
* Consolidate ChainAppConfigTest, add ScalaTestUtil to help with working with scalatest, speed up ChainHandlerTest a bit more
* Build and CI improvements
In this commit we:
1) Parallelize the Travis CI config, by splitting
each project into its own Travis task
2) Download bitcoind binaries through sbt
* Use binaries downloaded by sbt task
* Make BitcoindRpcTestUtil work on Travis without bitcoind on PATH
* Add new downloadEclair task to sbt
* use sbt downloaded binaries in tests
* Fix Eclair and Bitcoind tests
* PoC bitcoin-s-cli
* Add CLI, Server sbt projects, remove Ammonite
In this commit we set up sbt configuration for
CLI, Server (in-work-name) and corresponding
test projects.
We also remove Ammonite shell from sbt, as that
isn't really being used. bloop console offers
the same functionality way more ergonimic.
* Move BitcoinSAppConfig into new server project
Server project depends on node, chain wand wallet
so this is a good time for introducing this class
into main sources. We also introduce
BitcoinSTestAppConfig in testkit, to replace the
functionality in BitcoinSAppConfig related to
tests.
* Type chain in blockchainresult
* MVP server setup for node, chain and wallet
* Extremely dirty CLI for interacting with server
* initial attempt at mimicking Bitcoin Core API
* WalletStorage: add method for checking for seed existance
* Check for seed existance on wallet startup
* Fix bug where MnemonicNotFound was not an error
* Segregate confirmed and unconfirmed balance methods
* Add error handling, improve formatting of CLI output
* Tweak build
Bump Sttp version, downgrade to uPickle 2.11 compat,
skip publish in cli-test and server-test
* Add CLI, server and picklers to root project
* WIP: 2018 12 22 node project (#280)
* Add files from old spv node project
src compiling
test files compiling
ran scalafmt
Fix serializer tests
Get non networking test cases to work
WIP: Debug peermessagehandler
Update CRUD, remove all of the Actor craziness. Add DbManagement trait and unit test db
WIP: Rewroking PeerMessageHandler, create Peer, DataMessageHandler, PeerHandler
Reworking Client to handle all tcp messages and message alignment for bitcoin p2p messages
* Wip: Node refactor
* Create node test project, move all node tests into that project and move all generators for the node project into testkit
* Rework ClientTest to use testkit, start minimizing akka usage, implement connect(),isConnected(), disconnect(), isDisconnected() in PeerMessageReceiver
* Create Peer, PeerHandler, PeerMessageSender and PeerMessageReceiver
* update readme about status of node project (#359)
* Add flyway plugin to manage database schemas (#361)
* Add flyway plugin to manage database schemas
* Switch database driver to sqlite3 to be more portable, rework configs for sqlite3
* Set up sqlite database directories and files if they are not already created
* Add torkel's review
* Add chain, wallet, db-commons projects (#367)
* Add chain, wallet, db-commons projects
* Rework db creation logic if they db does not exist
* Add config logging to try to debug travis ci
* Pass explicit class loader for db config
* Remove duplicate call to dbConfig
* Make DbConfig.dbConfig a lazy val
* Remove noisy log
* Add scaladoc to DbConfig
* Switch dbConfig readme paragraphs
* Fix compile issues introduced during rebase onto master with rpc changes (#394)
* WIP: 2019 03 12 tip validation (#378)
* Implement blockchain handling data structures
Add TipValidation happy path
Add more test cases for TipValidation.checkNewTip for badPrevBlockHash and badPOW
Add overflow check, fix endianness bug for checking proof of work
Add pow transition check, refactor difficultyChangeInterval into chain params, add more tests
fix a few nits
Fix compile error, clean up unused import
Remove redundant files from node project
* Implement GetNextWorkRequrired/CalculateNextWorkRequired, move BlockHeaderDAOTest cases into chain project
* Add full POW change check in TipValidation, address code review nits
* Configure logging in chainTest, turn logging OFF in other test projects
* Address code review pt2
* Add coverage minimum for chain project (#398)
* Add coverage minimum for chain project
* Add first Blockchain.connectTip() unit test, switch to a in memory sqlite database for unit tests, starting using fixtures for BlockHeaderDAO in unit tests
* Add tests for ChainHandler.processNewHeader(), ChainHandler.getHeader(), Blockchain.connectTip(). Refactor redundant configurations being passed around excessivly
* Address code review, fix a flaky test in ClientTest.scala
* Test Fixtures (#403)
* Working test fixtures
* Removed ChainTestFixture trait in main code
* Composing Fixtures (#413)
* Downloaded over 9000 mainnet BlockHeaders into a json file
* Added new fixture with populated blockHeaderDAO
* Split writing to db into batches
* Rebased
* Simplified fixtures with makeFixture abstraction
* Added util functions for composing builders
* Add integration test between bitcoind <-> zmq <-> bitcoin-s-chain project. Test that we can relay a header from bitcoind over zmq into the bitcoin-s chain project. Redo ZmqConfig to use InetSocketAddress
* Address code review
* wip
* A compiling withBitcoindZmqChainHandler fixture
* Tests passing!
* Made blockHeaderDAO private
* Got 9000 new block headers from 562375 to 571375
* Added offset to populated blockHeaderDAO fixture
* Added scaladocs to fixture things
* Initial wallet: import UTXO and spend it (#391)
* Updates ExtKeyVersion with fromChainParams method
* Add equals to Address
* Update BIP44 classes
* Add ScriptType
* Initial work on wallet support
* Add foreign keys pragma for SQLite
* Add UTXO models and DAO
* Add addres P2WPKH generation and WIP for addUTXO
* Add logging config for wallet
* Add change address generation, proper-ish addUtxo and sendToAddress
* Address code review on #391
* Add empty AES passphrase invariant
* Add poor mans test fixtures
* Add listUtxos, listAddresses and getBalance to wallet API
* Use fixtures from chain project
* Fix CI test failures
* Fix broken up package path
* Updates bloop config for new projects (#424)
* Multi fixture file (#419)
* Created FixtureTag and ChainFixture
Used ChainFixture in BitcoinPowTest
Added implicit conversions for nice syntactic sugar
* Added documentation for multi-fixture
* Made defaultTag a val
* add a logback-test.xml to the wallet project (#433)
* Introduce AppConfig that combines ChainParams and DbConfig (#432)
* 2019 04 23 app config per project db config per project (#434)
* Add DB conf file resolution that works across projects
* Create applicatoin configurations for specific projects, rework DbConfig structures for individual projects. Force network to be mixed into DbConfig rather than DbConfig to be mixed into the network
* Add ammonite to db-commons, remove noisy logs
* remove mixin for DbConfig that required a NetworkDb. Now networkDb is just a field on 'DbConfig', this simplifies things downstream type wise when interacting with the projects AppConfig. This commit also removes a parameter from AppConfig, now only a DbConfig needs to be passed in, and we can derive the network and chain params from the DbConfig. The only exemption is UnitTestDbConfig as it is sometimes handy to specify a different network (i.e. mainnet) when testing
* Turn DbConfig objects to case objects, wrap those case objects in their parent type companion object
* remove cast in Wallet.scala
* Add EnhancedEither class for 2.11 compat (#437)
Add implicit conversion from Either to
2.11-compatible Either-wrapper. Also
remove trailing comma in WalletTestUtil
that breaks 2.11 build.
* Fix CI tests hanging (#438)
* Execute wallet tests sequentially to avoid SQLite deadlocks
* Refactor logback config to reduce duplication
* Use in-memory SQLite DB for unit tests
* Debug prints for DatabaseConfig.forConfig
* Fork JVMs in test to ensure proper in-memory DBs
* Pass in Akka config to Eclair tests, avoid cluttering Akka log output
* Don't fork JVM on node tests'
* Move things out of ChainUnitTest (#448)
* Move things out of ChainUnitTest
* Remove printlns
* 2019 04 29 client test (#449)
* Bump timeout on connect to node test
* Change from isConnected -> isInitialized to avoid the error trying to disconnect before we are fully initialized
* Wrote tests for POW difficulty change calculation and header processing (#429)
Fixed BitcoinPowTest
Rebased onto AppConfig code
Rewrote ChainHandler integration test
Made chain handler test synchronous
Fixed a couple test bugs
Implmented a more efficient getAncestorByHeight
Fixed ChainHandler integration test by using the correct starting conditions
Responded to code review
Responded to more code review
Deleted redundant Pow test
Made BlockHeaderDAO.getAncestorAtHeight use a List for its loop to improve performance
* WIP: Create ChainSync, BitcoindChainHandlerViaRpc, add simple ChainSyncTes… (#450)
* Create ChainSync, BitcoindChainHandlerViaRpc, add simple ChainSyncTest to sync one block from a external bitcoind instance via rpc
* Add check for having the best block hash in our chain state already
* Fix prev block hash to be empty hash if genesis block header
* BlockchainBuilder (#439)
* First commit for implementing a BlockchainBuilder
* use Builder rather than ReusableBuilder to be compatible with scala 2.11.x
* Decouple Blockchain & BlockHeaderDAO
* Rebase onto node, incorporate changes in #429
* Add more comments
* Reverse order of headers in builder
* rebase onot node branch, refactor apis
* DB: Add utility method for listing tables in a DB (#447)
* Node rebase (#458)
* Implement BIP32 path diffing
* Rebase node onto newest HD changes in master
* Fix 2.11 compile errors
* 2019 05 01 wallet ammonite scripts pt2 (#452)
* wip -- not finding testkit in doc worksheet
Wip -- classdef not found for create-wallet.sc
zmq bug
Clean up some logs
nest zmq start in bitcoindF
update jeromq to 0.5.2-SNAPSHOT to get rid of annoying log to stdout
Rebase onto node branch with new configs
Successfully running ammonite script create-wallet.sc
2019 05 01 wallet ammonite scripts pt2 (#25)
* Refactor Ammonite dep
* Add basic error handling in AmmoniteBridge
* Add very basic README for doc project
Fix compile issues after rebasing onto master
Add code to sync our wallet code with bitcoind after creating a tx
* refactor ZMQSubscriber to _hopefully_ avoid hanging when we call context.term(). We do this by closing the socket before calling context.term() and using socket.setLinger()
* Update doc/src/main/scala/org/bitcoins/doc/wallet/create-wallet.sc
Co-Authored-By: Christewart <stewart.chris1234@gmail.com>
* 2019 05 05 sync chain (#460)
* Add code to sync our wallet code with bitcoind after creating a tx
Add script to illustrate how the chain persisted and how to sync against a running bitcoind instance on regtest
* fix bug relating to subtraction operator not being communative in Pow.getNextWorkRequired(). This kept us from being able to switch proof of work intervals correctly
* rename script from persist-chain.sc -> sync-chain.sc
* fix 2.11.x compile issues
* Refactor chain, node, wallet config (#463)
* Refactor chain, node, wallet config
Get rid of NetworkDb, DbConfig
Add proper structure to conf system,
moving everything under the bitcoin-s
root key.
* Remove Scalacheck from node project
* Add doc on configuration
* Add override feature to AppConfig
* Address code review in #463
* Throw if default data dir is used in tests, add Scaladoc to AppConfig
* Add explanations for withOverrides, link to configuration.md from AppConfig
* Fix compile error
* Moves chain fixtures to testkit project (#475)
reset node files
* Store encrypted mnemonic to disk (#462)
* Add WalletStorage object
* Add encrypted mnemonic storage, locked wallet
Add lock and unlock operations to wallet.
Separate between locked and unlock wallet.
* Handle non-existant seed file
* Respond to code review from Chris
* Use val instead of import
* Add doc on how mnemonics are encrypted/stored
* 2019 05 15 spv sync headers (#479)
* Implement SpvNode skeleton, create NodeUnitTest and move it to the testkit
* Implement test case to sync a header via spv into bitcoin-s
* Fix compiler errors
* Make node project Main runnable (#26)
* Add logging configuration to node project
* Make default config workable in non-test environments
* Add more logging of config in BH DAO and AppConfig
* Make Peer id optional
* Make node Main.scala runnable
* Implement Main.scala to sync with a locally running bitcoind instance. You can now run with 'bloop run node' and sync the node if you adjust the parameters inside of Main.scala. This also reworks the structure of 'AppConfig'. It turns the *AppConfig into a case class intead of case objects. This allows us to pass custom configs into those case classes
* Address code review from torkel
* Reintroduce withOverrides (#29)
* Turn off chain validation logs
* Make datadir a parameter to bitcoind config rather than having it implicitly written to the bitcoin.conf file. This was a difference that was occurring in the node branch which had a parameter for the datadir and master which was implicitly writing it to bitcoin.conf
* Add ability to overrwrite conf file except in the case of overwriting the DEFAULT_DATADIR & DEFAULT_CONF
* remove extra Bitcoind.stopServers in WalletIntegrationTest
* Add proper parsing of bitcoin.conf, remove datadir
Somewhat recently a new config format was introduced
in Bitcoin Core. Options can now be specified under
section headers ([regtest], [main], [test]), as
well as with network prefixes (regtest.rpcport,
main.prune) and the traditional format. In this commit
we introduce a new type BitcoindConfig that parses
this format. We also make the necessary changes in
the bitcoind/Eclair RPC clients to accomodate this new
type.
We also remove the datadir field from
BitcoindAuthCredentials. This is not strictly necessary
to connect to a bitcoind, so this is a bad separation
of concerns. The datadir field is instead moved into
the BitcoindInstance.
Finally we go over some places in tests and test
utils and assert that we are operating in the
user temporary directory. This is a safety measure
so that other developers/users won't do the same
mistake I did while working on this and accidentally
blow up their $HOME/.bitcoin directory.
* Add BitcoindRpcClient.fromDatadir
* Address code review from Chris
* Put tests for bitcoind-rpc and eclair-rpc into a separate project to get rid of circular dependency
* Modify travis yml file to get rid of all testkit stuff
* rename rpc project to bitcoind-rpc project
* Add bitcoind-rpc test cases that got deleted accidentally
* Add more files missing from bitcoind-rpc-test
* Remove circular deps part on CONTRIBUTING.md
