* Refactor BitcoinSRunner to use StartStop[Async]. This makes it easier to have a uniform interface to write for destruction code in BitcoinSRunnerTest
* Fix other ServerRunTest to cleanup after itself
* Add new app/scripts project which is meant to store useful bitcoin-s scripts
* Add ScanBitcoind with an example of counting segwit txs between a range
* Fix bug of creating a new actor system everytime BitcoindRpcClient.apply(instance) is called
* Add BitcoindRpcClient.fromVersionNoSystem()
* Take ben's suggestions
* Fix compile
* Rework P2SHScriptSignature.isStandardNonP2SH() to account for nesting p2sh script inside of it
* fix compile on java8
* Enable app packaging in scripts project
* Initial implementation of wallet sync with bitcoind with akka streams
* Make BitcoindRpcBackendUtil.startBitcoindBlockPolling use akka streams
* rework BitcoindRpcBackendUtil.syncWalletToBitcoind() to use akka streams
* Wallet Rebroadcast Logic
* Use wallet scheduler for address queue rather than it's own thread
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* Bump default
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* Move blockhash to tx table from spending info table
* Add test, fix spending tx id col name
* Scaladocs, add test
* Add more unit test
* Make id not comparable in process tx test
* Fix tests
* attempt to fix
* Add mempool comment to scaladoc
* Deparallelize process inputs & outputs
* Rework docker configuration to pass in a custom configuration file
* Add default user of bitcoin-s for docker images
* Add runtime docker configuration to website
* App Server docker configuration w/ refactors
Get app server configuration working with docker
Add section for building the appServer for docker
* Remove compile commands
* Give oracle server its own port
* Don't have default cli command port
* Default port in config
* Remove unused server conf
* Oracle own config option
* Give connection pool example config
* Get hikari logging working on postgres
* fix 2.12.x
* Bump metrics core to a supported version
* Add hikari-logging and hikari-logging-interval configuration options to turn off/on hikari and schedule how often logs appear
* Turn off hikari logging in the db-commons/reference.conf and add it to appserver/reference.conf
* default the logging to off in scala code
* Make sure connection pool is enabled in db-commons, add default database configuration to website
* Address code review, fix things that didn't need to be changed
* Add walletinfo rpc
* Move rootXpub to key manager obj
* Make configuration keys for getinfo the same as the ones in our configuration files
* Cleanup
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* Add new sbt-native-image plugin that helps generate a correct native image for the cli
* Initialize everything at build time to try and get cli working again
* Get server working with native image build
* Add '--no-fallback' which will report failures if we fail to build a standalone image
* Revert server project, file issue 2506 to indicate problems with server build
* Add native cli publishing using github actions
* Preprend 'app/' to paths
* Enable http,https protocols on cli
* Remove old graalvm plugin settings related to sbt native packager
* WIP: Get something working that isn't network specific
Start putting things back in place
Add comment
Revert logback file
Remove BitcoinSLogger object
* fix unit test
* Network specific log location
Co-authored-by: benthecarman <benthecarman@live.com>
* Pulled down work from adaptor-dlc onto master
* Reverted some accidental deletions
* Removed unused import
* Added scaladocs
* Responded to Ben's review
* Added some scaladocs and invariants
* Responded to chris' review
* Responded to more review
* Added some comments
* Introduce ChainHandlerCached which behaves like the old ChainHandler. Now Chainhandler.getBestBlockHeader() will read headers from the database
* Remove ChainHandler.blockchains field, now it's only available in ChainHandlerCached
* De-futurify ChainHandler.fromDatabase()
* Adjust logging
* Patch test case
* Use BlockHeaderDAO.chainTips when getting best header rather thean BlockHeaderDAO.getBlockchains(). Implement a helper method ChainHandler.toChainHandlerCached()
* Fix chain.md,wallet.md
* Make ChainHandler.getBestBlockHeader() consider time of header if chainwork is the same. Make test cases less strict on what header is the best header when both chainwork and time are the same on the eader
* Only execute callbacks on headers that are going to be created in the database, not all headers passed into ChainHandler.processHeadersWithChains()
* Turn up log level again
* Small optimizations, check if we have seen a header before before processing it in ChainHandler.processHeadersWithChains(). Fix FilterSyncMarker.toString(). Use ChainHandlerCached in Node
* Remove ChainHandlerCached in appServer, re-add it in Node.scala
* Update Oracle to use new TLVs
* Rename things to use new names, scaladoc, small clean ups
* Add descomposition tests, docs, sign numbers outside of range
* Cleanup a bunch of imports in test
* Use @nowarn annotation for testing things that are deprecated
* Fix base58 test
* Cleanup crypto-test
* Use scala collection compat dependency to get access to the @nowarn annotation on scala 2.12.x
* Rework more scala 2.13.x compile failures in chain-test and key-manager-test
* Optimize imports for entire project
* Fix nits
* Add Wallet State Descriptors
* Sync blocks while offline w/ bitcoind backend
* Add to WalletDbManagement
* Update db management test
* Respond to review
* Rename function
* Remove requirement for ZMQ with bitcoind backend
* Add test, move poller to BitcoindRpcBackendUtil
* Add docs & logs, fix niche case
* Use AtomicReference, reset prevCount on failure
* Use absolute paths for resolving configurations
Make log level OFF again
WIP: Move resolution of config into AppConfig.start(), things are totally broken
WIP: Make AppConfig.config private
Get all tests passing again but using absolute path in configuration
* Cherry-pick onto master
* Fix BitcoinSAppConfig configuration keys to use the full path
* Apply code review
* Fix log, move invariant for requiredConfirmations back into the lazy val
* Fix DlcOracleAppConfig to use absolute path
* Introduce 'FilterSyncMarker' to ChainApi, make it clearier what exactly the (Int,DoubleSha256Digest) tuple is returned from ChainApi.nextBlockHeaderRange()
* Fix doc
* Add scaladoc to FilterSyncMarker
* Rebase onto master, fix conflicts to use FilterSyncMarker
* Apply string factory to a bunch of easy things
* Refactor HDPath, ExtKey, and LnTagPrefix to use StringFactory
* Implemen StringFactory on LnHumanReadablePart, ScriptType, ServiceIdentifier
* Implement StringFactory on AesCrypt, NodeUri, AddressTag, PSBT, TxoState
* Fix failing tests, fix website compile
* Apply StringFactory to all ScriptOperations, ChannelState, Script Parsing
* Clean up code for BroadcastDAO test
* Extend StartStopAsync with BitcoinSAppConfig, create 'CachedAppConfig' test trait, clean up P2PClientTest
* Make MerkleBuffersTest use CachedAppConfig
* Start cleaning up after ourselves in the chainTest test suite
* Cancel background runnable for generating blocks after test case, add nodeAppConfig.stop() to tests in NodeAppConfigTest
* Stop mainnet nodeAppConfig in test as per ben's code review
* Fixed logging so that appenders are always started when logging is happening
* Started rolling policy so that file appenders could start in getLoggerImpl
* Update scalafmt-core to 2.6.0
* Upgrade scalafmt, fix all formatting errors
* Scalafmt on 2.12
* fix test
* More fixes
Co-authored-by: Ben Carman <benthecarman@live.com>
* Pulled down dlc CLI code (returns 400 Bad Request when called)
* Added decodeRawTransaction to server
* Moved decoderawtransaction to CoreRoutes
Co-authored-by: Ben Carman <benthecarman@live.com>
* Break things up in server/Main.scala, allow things to be done asynchronously where they can be
* Rework Main.scala to have small methods that are used to initialize parts of our infra. Now it doesn't depend on global variables in Main either, they are implictly passed to these helper methods
* Fix Chaindb chainwork Migration
* Move chain work calculation out of node, use the ChainApi returnd by runChainWorkCalc()
Co-authored-by: Ben Carman <benthecarman@live.com>
Added OutputReference and introduced public key computation on SPKs
Made InputInfo into an ADT
Made UTXOSpendingInfo use InputInfo
Replaced UTXOSpendingInfo with NewSpendingInfo and got non-test things compiling
Made aliases for (NewSpendingInfo/NewSpendingInfoFull/NewSpendingInfoSingle)[InputInfo]
Got rid of source code mention of UTXOSpendingInfo
sbt compile passes
tests compile
Fixed all coreTest tests
Renamed to UTXOInfo
Some cleanup
Moved redeem script and script witness accessors to companion object
Responded to code review
Added OutputReference and scaladocs
Moved p2pkhPreImageOpt downstream and generalized to hashPreImages
Fixed adding-spks.md
Fixed psbts.md and txbuilder.md
Renamed UTXOInfo
Apply renaming to docs
* Add wallet creation time for rescans
* Fix docs
* Clean up and add test
* Remove account bday
* Fix compile issue and docs
* Add more chain handler tests
* Use Instant over Long, add docs
* Fix docs
* Implement Slick's multi database support via our DAOs
WIP2 work on chain project
WIP make JdbcProfieComponent so we have a common JdbcProfile
WIP2
Clean up a few things
WIP got CRUDAutoInc compiling
Get closer to everything compiling in the chain project
Begin moving database code into JdbcProfileComponent
WIP source compiling
Get dbCommonsTest/ test cases passing
Get rest of tests compiling and passing
* Rebase onto master, fix conflicts
* Run scalafmt
* Run scalafmt again
* Cleanup a few compiler warnings by adding type annotations to thigns
* Explicitly cast tables where they are needed in the wallet project where we need TableQuery for foreignkey / joins in slick
* Redo definition of 'table' to use the profile.api.TableQuery in scope
* Rename all *Table.scala files to *Db.scala
* First crack at address ben's code review
* Fix docs compile issue
* Fixed casting (#30)
* Fixed AppConfig casting
* Fixed TableQuery casting in DbManagement subtypes
* Fixed casts in DAOs relating to internal types
* Address nadav's code review, move JdbcProfileComponent into it's own file
* Remove unused listTables() method
Co-authored-by: Nadav Kohen <nadavk25@gmail.com>
* WIP: Build queue for getnewaddress
* Implement background thread to process requests to getNewAddress so that we fix issue 1009 with it's async issues
* Run scalafmt
* Replace the mutable.ArrayBuffer in AddressHandling with ConcurrentLinkedQueue
* Put FundTransactionhandling.fundRawTransactionInternal's call to getNewChangeAddress() into the for expression so we don't generate an address when the funding of the transaction fails when selecting utxos
* Move thread initialization out of method
* Switch to using ArrayBlockingQueue, which will block thread with .take() so we don't need to loop constantly, add 'addressQueueSize' and 'addressQueueTimeout' configurations
* Update wallet-test/src/test/scala/org/bitcoins/wallet/AddressHandlingTest.scala
Co-Authored-By: rorp <rorp@users.noreply.github.com>
* Add error handling to return the failed future if the queue is full, add a unit test for when the queue is full
* Run scalafmt
* Rebase
* Add scaladoc for throwing an exception
* Run scalafmt again
Co-authored-by: rorp <rorp@users.noreply.github.com>
* Change NodeCallbacks to Future[Unit] s
* Fix compile issue
* Change try to future
* Change Future.sucessful(()) to FutureUtil.unit
* Callback failure handling, callback docs
* Create functions for executing callbacks, make merkleblock a future
* Convert MerkleBuffersTest to BitcoinSAsyncTest
* Fix compiler errors
* Reorder UpdateBloomFilter tests so that txFromWalletP is a success for both calls and the other test does not wait on the promise to complete
* Added server handling for empty account tables while a seed exists
* Add test coverage to key-manager
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* Moved Cli code to an object called ConsoleCli which can be called from sbt console
* Add --allow-incomplete-classpath to cli.sbt
Co-authored-by: Ben Carman <benthecarman@live.com>
* Implement abililty to use BIP39 password. This means this password needs to be password through our various projects to be able to correctly generate the key that controls the wallet. This also renames 'CreateKeyManagerApi' -> 'BIP39CreateKeymanagerApi' as the bip39 password is needed when creating the KeyManager
* Add bip39 password to BIP39KeyManager.fromParams(), fix mdocs
* Fix bug in unit test were whe weren't specifying password
* Fix rebase issues
* This creates a subtype BIP39KeyManager and moves all existing KeyManager functionality to that subtype. We leave a empty 'KeyManager' trait for now
* Fix website docs
* Move initialization of wallet entropy into the key manager
* Enable key-manager in ci, add coverage minimum
* Add coverage minimum for key manager, add KeyManagerParams, start adding KeyManager test harness
* Remove success cases on ADTs, start using Either
* fix import
* Get test coverage to 90%
* Add documentation for key-manager.md, add side bar for key-manager
* Fix wallet.md
* Make seedPath default to baseDatadir, so we don't have a unique seed for every network. Add more documentation
* Address code review
* Address code review from Ben
* Add missing import
* Add more tests for coverage
* Segregate key manager from wallet
* More consistent naming in build.sbt, run scalafmt
* Add test case making sure we don't overwrite a mnemonic seed
* Add key-manager.md
* Fix compile issues with older scala versions
* Address code review
* Add documentation about how the cli can be built with a native image
* Finish rest of documentation on building command line client for bitcoin-s
* Add missing akka actor on walletServer classpath, add plugins to be able to build the walletServer as a standalone jar
* Add server.md
* Rename walletServer -> appServer
* Address code review, add missing main class in server project
* Address code review
* Remove AKKA_CONFIG that was preventing sane logging for akka, move db-commons reference.conf to testkit so we can control testkit applications, and then rename app/server/.../application.conf to reference.conf like good libraries should do
* Update contributing.md
* update a few more links inside of contributing.md
* Run scalafmt
* Update scalafmt to 2.0.1
* Scalafmt
In this commit we
1) Reconfigure the .scalafmt.conf file
2) Add a scalafmt binary (same version as specified in conf file)
3) Use said binary on CI to check that everything is formatted correctly
* Ran scalafmt
* Update scalafmt ignore
* Add scalafmt plugin to project/plugins.sbt, run it
* run scalafmt
* Bump Scala versions
Support Scala 2.12.9
and 2.13.0.
To make this easier, we delete the `scripts` project. Everything
that was in here was covered by content on the website. We also
delete the `doc` folder, as that was a remnant from when `scripts`
was called `doc`.
* Crib uPickle akka-http support while we wait for publish
* Fix compiler warnings
* Add note on test logging to contribution guide
* Reduce duplication in Blockchain implementation
* Use Scala 2.12 for website
* Introduce compat package object for collections converters
* Fix Either compiler warnings
* Add sync-chain and create-wallet docs from deleted scripts
* Fix rebase goofup
* Split app loggers into their corresponding modules
This avoids a bunch of places with awkward implicit resolution.
We also make sure that all logging in node, chain and wallet
happens with the app loggers, and not BitcoinSLogger
* Tune test logging level to WARN
* Start the process of refactoring our ChainHandler to be able to avoid database calls on TipValidation
WIP: Begin explicity passing state back and forth in return types of PeerMessageReceiver, P2PClient, , DataMessageHandler. This commit also implements the ability to keep our blockchain completely in memory. Previously when we were updating the tip of the chain, we had to make a database read to figure out what the best tips are. This is suboptimal for performance because a database read needs to be done for every block header we see, now we just keep the chain in memory
Fix bug in DataMessageHandler that pre-emptively sent a getheadersmsg to our peer. Make 'chainApiF' internal to our spvNode (not a parameter). This forces the chainApi to be created from disk everytime a new SpvNode is spun up. This keeps us in sync with the blockchain at disk at the cost of disk access and less modularity of SpvNode
Address torkel code review
Fix rebase issues
Address code review
Address nadav code review
* Rebase onto master, fix api changes
* Add functionality for updating SPV node bloom filter
* Add SPV node shutdown to exit hook
* Clean up traits/case classes
* Change fixture in WalletBloomTest
* Fix logging bug in TransactionProcessing
* Add MerkleBuffers
In this commit we add MerkleBuffers, which is an object
that lets us aggreagate merkleblocks with their corresponding
transactions before sending them out. This is global, mutable
state (bad!) but it's a working solution for now;
* Use TestAsyncUtil
* Add MerkleBuffers test
* Send getdata if receiving single header
* Change awaitSync to use block count
* Fix UpdateBloomFilterTest
* Add more logging of chain/headers validation
* Send getdata for all blocks
* Nits: Scaladocs, comments toString
* Add logging to data directory
In this commit we add the ability for the node, chain
and wallet projects (+ the server) to log to the users
data directory instead of whatever directory the
binaries was launched from. This is inherently a bit
more complicated than our previous setup, because
we need to read the user directory before we can create
loggers. As a result of this, some files/methods were
moved around, so the relevant app config could be
found in scope.
We also introduce several logging categories that can be
tuned individually through user configuration. These logggers
are exposed both as traits that give a field `logger`, or
as methods that return the required logger.
* Add datadir configuration to AppConfig
In this commit we add support for AppConfig to pick up
the data directory configuration file. We also add
a section to the contributing guide on how to tune
logging levels.
* Pass data directories explicitly for configuration
* Add functionality for broadcasting TXs to node
In this commit we add functionality and tests
for broadcasting a TX from our node. To accomplish
this we introduce a table over broadcastable TXs
that's added to when the externally facing method
broadcastTransaction(tx) withing SpvNode is called.
We send out a inv message for the TX we just added,
and upon receiving a getdata message we search in
the previously mentioned table for entries where
the hashes match up.
* Broadcast TX from server to SPV node
* Perform assertions on the balance of bitcoind after sending a TX
* Remove typeclass from broadcast TX
* Refactor withFundedWalletAndBitcoind
* Match on BitcoindExecption instead of throwable
* Clean up broadcast functionality after code review
* PoC bitcoin-s-cli
* Add CLI, Server sbt projects, remove Ammonite
In this commit we set up sbt configuration for
CLI, Server (in-work-name) and corresponding
test projects.
We also remove Ammonite shell from sbt, as that
isn't really being used. bloop console offers
the same functionality way more ergonimic.
* Move BitcoinSAppConfig into new server project
Server project depends on node, chain wand wallet
so this is a good time for introducing this class
into main sources. We also introduce
BitcoinSTestAppConfig in testkit, to replace the
functionality in BitcoinSAppConfig related to
tests.
* Type chain in blockchainresult
* MVP server setup for node, chain and wallet
* Extremely dirty CLI for interacting with server
* initial attempt at mimicking Bitcoin Core API
* WalletStorage: add method for checking for seed existance
* Check for seed existance on wallet startup
* Fix bug where MnemonicNotFound was not an error
* Segregate confirmed and unconfirmed balance methods
* Add error handling, improve formatting of CLI output
* Tweak build
Bump Sttp version, downgrade to uPickle 2.11 compat,
skip publish in cli-test and server-test
* Add CLI, server and picklers to root project
