Commit Graph

66 Commits

Author SHA1 Message Date
Chris Stewart
c54d6dcbdc 2020 06 09 addr invoice string factory (#1538)
* Implement StringFactory with Address and LnInvoice

* Address code review

* Run scalafmt
2020-06-11 10:44:12 -05:00
Ben Carman
1dd6025b9d Testkit wallet with bitcoind uses bitcoind as api (#1499)
* Testkit wallet with bitcoind uses bitcoind as api

* Fix docs
2020-06-03 14:05:22 -05:00
Ben Carman
62491dad1c Implement Bitcoind as ChainQueryApi & NodeApi (#1481)
* Implement Bitcoind as ChainQueryApi & NodeApi

* Remove default implementations for filter apis

* Fix warnings

* Fix error
2020-06-01 14:03:00 -05:00
Ben Carman
1656b9091f Implement BitcoindRpcClient as a FeeProvider (#1486)
* Implement BitcoindRpcClient as a FeeProvider

* Make FeeRateApi a trait
2020-05-30 09:03:08 -05:00
Chris Stewart
5aeb8ebdd1 Bump akka to 2.6.4, fix deprecated things (#1374)
* Bump akka to 2.6.4, fix deprecated things

* Bump to 2.6.5 akka

* Clean up imports

* Remove materializer from FeeProviders
2020-05-30 08:49:28 -05:00
Nadav Kohen
dfd3353cc4 TxBuilder Refactor (#1426)
* Refactored Transaction

Created RawTxBuilder

Created RawTxFinalizer as layer on top of RawTxBuilder

Introduced RawTxSigner and replaced TxBuilder!

Deleted TxBuilder!

* fixed things after rebase

* Made RawTxBuilder compatible with older versions of scala

* Began responding to review

* Made Finalizer take a Builder rather than the other way around

* Added composition for finalizers

* Ran scalafmt

* Updated txbuilder example documentation

* Moved tests from old TxBuilderTest files to relevant new test files

* Added scaladocs
2020-05-21 09:47:08 -05:00
Chris Stewart
cec29e6894 Try and improve reliability of UpdateBloomFilterTest, also add getMem… (#1434)
* Try and improve reliability of UpdateBloomFilterTest, also add getMemPoolEntryOpt() that returns an Option indicating if a tx was in the mempool

* Fix nits
2020-05-18 13:01:25 -05:00
Nadav Kohen
a503441619 Crypto Project Refactor (#1380)
* Added crypto project and decoupled BitcoinSLogger from NetworkElement

Decoupled BitcoinSLogger from Factory

Moved NetworkElement into crypto project

Moved Factory and BitcoinSUtil (renamed to BytesUtil) to crypto project

Moved MaskedToString to crypto project

Added BytesUtil to imports and cleaned up CryptoUtil.recoverPoint

Moved the rest of crypto stuff to the crypto project

Moved crypto tests to crypto-test project

* Added documentation for crypto project
2020-04-30 12:34:53 -05:00
Chris Stewart
cb88b78540 Bump scala version to 2.13.2 (#1360) 2020-04-23 07:38:20 -05:00
Ben Carman
c696a7306b Require that addresses are the correct network when sending (#1332)
* Require that addresses are the correct network when sending

* Compare network bytes so typed addresses work

* Use isSameNetworkBytes
2020-04-13 13:58:41 -05:00
Chris Stewart
142c5c8a30 Add JsonParseException catch clause in the isStartedF method (#1334) 2020-04-13 09:43:30 -05:00
Ben Carman
aa8a2e14a6 Move Json Reader, Writers, and Serializers to App Commons (#1328)
* Move Json Reader, Writers, and Serializers to App Commons

* Move app-commons outside of app/
2020-04-10 14:33:37 -05:00
Chris Stewart
cd84e3483d 2020 03 06 wallet rescan test (#1218)
* Add wallet rescan tests

* Create FilterSync, which gives us an API inside of the chain project to sync filters with

Add another unit test to filter sync

Add more unit tests for ChainSync and FilterSync

Clean up some docs, remove some extra lines of code

Run scalafmt

Add filter-sync.md

Cleanup some nits

Add more information of how FilterSync.syncFilters() works

Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project

Run scalafmt, hide imports in filter-sync.md so code appears cleaner

Move implicits out of invisible block as it seems to cause errors

Make it so FilterSync processes filters in batches rather than fetching them all at once

Fix compile error

* WIP bitcoind implement ChainQueryApi

* rework fixtures to be able to support injecting ChainQueryApi implemented by bitcoind into our fixture infrastructure for creating wallets

* Fix rebase problem

* Implement getFiltersBetweenHeight() with ben's solution

* WIP Start implementing NodeApi against bitcoind

* Actually inject the bitcoind backed nodeApi into our fixture

* Get first rescan test working for rescanning the entire blockchain

* Implement test case for rescanning from a specific height

* Fix NeutrinoNodeWalletTest test case that uses a experimental version of bitcoind, for now i believe our experimental binary is on v18

* Add wallet-rescan.md, add helper method to WalletApi that allows you to clear out the wallet's utxos/addresses

* Add another log to try to debug CI

* Address code review

* Fix wrong ordering of deletion of tables in clearUtxosAndAddresses()

* reset logging level
2020-03-13 16:11:02 -05:00
rorp
c5f5b8c065 Fix the time based test cases in BitcoindV17RpcClientTest (#1224) 2020-03-13 11:24:43 -07:00
Chris Stewart
4c9a22f1e1 2020 03 08 filter sync (#1209)
* Create FilterSync, which gives us an API inside of the chain project to sync filters with

Add another unit test to filter sync

Add more unit tests for ChainSync and FilterSync

Clean up some docs, remove some extra lines of code

Run scalafmt

Add filter-sync.md

Cleanup some nits

Add more information of how FilterSync.syncFilters() works

Add 'FilterWithHeaderHash' type so that we can actually validate/verify block headers that are being fed into the chain project

Run scalafmt, hide imports in filter-sync.md so code appears cleaner

Move implicits out of invisible block as it seems to cause errors

Make it so FilterSync processes filters in batches rather than fetching them all at once

Fix compile error

* Add comment about trust model

* Run scalafmt
2020-03-10 18:01:14 -05:00
Nadav Kohen
052b64b7e2 Seq and Map Wrappers (#1131)
* Renamed size to byteSize in NetworkElement to avoid ambiguity

* Introduced SeqWrapper and MapWrapper to allow for wrapper case classes to have direct access to underlying methods

* Responded to review
2020-02-24 14:39:20 -06:00
Chris Stewart
6769205a8d Bump 'downloadBitcoind' timeout (#1136) 2020-02-19 09:48:45 -06:00
Chris Stewart
2279313eb9 Move binaries to be stored inside of the HOME/.bitcoin-s/binaries instead of usrdir/binaries. This makes it so different projects that use bitcoin-s can re-use binaries instead of having to download them for every project (#1128) 2020-02-17 12:07:04 -06:00
Chris Stewart
6a5918fa50 Bump binary download timeout for bitcoind (#980) 2019-12-21 17:52:33 -06:00
Ben Carman
5206353a4a Bitcoind v19 RPC (#910)
* bitcoind v19 new RPC calls and tests (#863)

* bitcoind v19 new RPC calls and tests

* Code review changes

* Review part 2

* Rename variable to be more descriptive

* Explanitory comment

* Ignore broken test cases

* Add missing signing functions

* Add test to check avoid_reuse flag is on (#870)

* Add test to check avoid_reuse flag is on

* Add test to make sure flags weren't set

* bitcoind v19 Update mempool RPCs and tests (#868)

* Update mempool RPC calls to bitcoind v19 compatibility

* Typo fix

* Add parameter name to calls

* Fix remaining rpc calls

* Formatting

* scaladoc for param

* Change param to correct type

* Clarify on scaladoc

* Add missing fees parmater to mempool rpcs (#875)

* Add weight field to mempool entries after v19 (#876)

* Move DescriptorRpc to be able to be used by future versions of bitcoind (#878)

* Add window_final_block_height to GetChainTxStatsResult (#880)

* Add passphrase argument to createwallet for later versions (#883)

* Add passphrase argument to createwallet for later versions

* Scaladoc + empty passphrase requirement

* Error message

* Add new services names parameter to P2P rpcs (#874)

* Add new services names parameter to P2P rpcs

* Add ServiceIdentifier Reads

* Add fallback case

* Address review

* Change to Try

* Move PsbtRpc to be able to be used by future versions of bitcoind (#877)

* Move PsbtRpc to be able to be used by future versions of bitcoind

* Add test

* Address comment

* Enable bloom filters for v19

* Enable bip 61 for tests

* Change to official binaries

* Force v18 for Spv Tests

* Remove unused config line
2019-12-04 07:44:44 -06:00
Chris Stewart
588fec14a2 Make 'downloadBitcoind' command async, which reduces task time by 3x (#906) 2019-12-02 09:01:21 -06:00
Nadav Kohen
1578e2642f Nicer Satoshis.apply (#899)
* Replaced Satoshis(Int64(_)) with Satoshis(_)

* Fixed SignerTest
2019-11-26 14:43:14 -06:00
cwaldron97
96476e56b3 (WIP)Changes to client start() (#748)
* changes to client

fixed daemon flag

yielding bitcoindRPCclient

versioned

more implementation

update

fixed yield error and some documentation

docs + compiler error fix

updated

responding to code review

missing bracket

fixed compiler warnings

fixed compiler warning

* scalafmt
2019-10-30 14:59:35 -05:00
rorp
7e19a706de Make tests to not require pre-installed bitcoind (#766)
* Make tests to not require pre-installed bitcoind

* update docs
2019-10-01 06:19:11 -05:00
rorp
74a44d1c4d Initial BIP157 support (#695)
* WIP: Initial BIP157 support

* store block hash and hetgh along with its compact filter header

* download and parse block filters

* getcfilters/cfilter

* cfilter table

* rescan full filter chain

* improved rescan performance

* optimize compact headers download

* addressed the PR comments

* split SVP and Neutrino node implementations

* configurable filter batch sizes

* initial filter sync

* addressed comments

* chage filter table's primary key

* fix Golomb filter deserialization

* batch database inserts

* neutrino node test

* fixed node test

* addressed the PR comments

* serializers tests

* cleanup

* fix compilation errors

* fix unit tests

* increase test coverage

* enable NeutrinoNodeTest

* make scalafmt happy

* don't cache experimental binaries

* inclease test coverage

* fix unit tests

* more granular CI tests

* disable NeutrinoNodeTest

* refactor tests

* addressed comments

* test coveage

* fix formatting

* responded to the comments

* some more changes

* fix the build

* test coverage

* revert testnet3 config parameter

* minor changes

* cleanup
2019-09-25 13:18:51 -05:00
Chris Stewart
67ead0bc7e 2019 09 05 scalafmt update (#735)
* Update scalafmt to 2.0.1

* Scalafmt

In this commit we
1) Reconfigure the .scalafmt.conf file
2) Add a scalafmt binary (same version as specified in conf file)
3) Use said binary on CI to check that everything is formatted correctly

* Ran scalafmt

* Update scalafmt ignore

* Add scalafmt plugin to project/plugins.sbt, run it

* run scalafmt
2019-09-05 20:02:58 -05:00
Chris Stewart
d4ccc2c441 2019 09 05 v18 rpc (#734)
* This is for colin

* Fix remaining errors on V18 RPC calls

* remove unused code final formatting fix

* implemented fixed listwalletdir rpc test

* responding to code review and replacing flatmaps
2019-09-05 09:15:00 -05:00
Torkel Rogstad
e0a5646258 Deprecate generate (#728) 2019-09-01 07:46:38 -05:00
Torkel Rogstad
8a58d7dde8 Make it possible to construct RPC client without ActorSystem (#725) 2019-08-30 15:12:18 -05:00
Torkel Rogstad
f6ed565c14 Refactor sbt and make Eclair tests download bitcoind (#727) 2019-08-30 15:11:52 -05:00
Torkel Rogstad
2f7fb05f96 Build and CI improvements (#710)
* Build and CI improvements

In this commit we:

1) Parallelize the Travis CI config, by splitting
    each project into its own Travis task
2) Download bitcoind binaries through sbt

* Use binaries downloaded by sbt task

* Make BitcoindRpcTestUtil work on Travis without bitcoind on PATH

* Add new downloadEclair task to sbt

* use sbt downloaded binaries in tests

* Fix Eclair and Bitcoind tests
2019-08-27 10:48:48 -05:00
Torkel Rogstad
c2f37335b8 Bump Scala versions (#697)
* Bump Scala versions

Support Scala 2.12.9
and 2.13.0.

To make this easier, we delete the `scripts` project. Everything
that was in here was covered by content on the website. We also
delete the `doc` folder, as that was a remnant from when `scripts`
was called `doc`.

* Crib uPickle akka-http support while we wait for publish

* Fix compiler warnings

* Add note on test logging to contribution guide

* Reduce duplication in Blockchain implementation

* Use Scala 2.12 for website

* Introduce compat package object for collections converters

* Fix Either compiler warnings

* Add sync-chain and create-wallet docs from deleted scripts

* Fix rebase goofup
2019-08-23 13:53:00 -05:00
Torkel Rogstad
2f5d4db1b2 Bump test coverage (#713)
* Bump test coverage

* Lower test coverage requirement for chain
2019-08-23 18:32:55 +02:00
cwaldron97
47644b7910 (WIP) Createwallet rpc from V17 (#712)
* create wallet rpc

* fixed test

* change to createwallet test

* create wallet test
2019-08-21 16:34:11 -05:00
Chris Stewart
ff051ac7a2 2019 08 05 broadcast tx test (#680)
* Start refactoring BroadcastTransactionTest to use new SpvNodeFundedWalletBitcoind fixture

* Double timeout to 10.seconds on spv node disconnect

* Attempt to bump disconnect even more

* Add more logging around starting and stoping spv node to try and debug failures with fixtures on test suites

* move starting of spv node into one place in the fixtures -- createSpvNode() --, also look at ordering of destruction to make sure that the spv node is stopped/destroyed BEFORE bitcoind is stopped/destroyed. If the inverse order happens, our spv node won't receive a response from bitcoind and will hang waiting for the Tcp connection to close gracefully

* Try switching from  to  to get CI to pass

* Add more logging and throws to try and debug failures on ci

* Add logging in the case we send a close command and peer isn't defined

* Match on Tcp.Connect on default receive in P2PClient

* Make SpvNode.disconnect() check to see if we are connected before attempting to disconnect

* Add more logging in fixtures, change behavior of SpvNode.disconnect() to not throw if we are already disconnected

* Switch to preferred way of terminating actor system in tests

* Make PeerMessageSender.sendMsg() return a , make sure a message is not sent to P2PClient until we are fully intialized

* Switch p2p logging to DEBUG

* Complete disconnect promise on peer message recv in the case where we didn't have a peer to begin with

* Make SpvNodeTest more robust, fix bug for when we send 'sendheaders' message. We shouldn't do this until our peer has sent us a verack

* Only send 'sendheaders' message to our peer after we receive a 'sendheaders' message from our peer. Hopefully this solves async issues where we were sending 'sendheaders' before our handshake was fully completed with our peer which means we wouldn't get headers sent to us

* Cleanup pt1
2019-08-14 07:18:36 -05:00
cwaldron97
e6dcd1cfb4 Cleaning up Publish Logs (#634)
* This PR is attempting to remove the publishing errors when publish
locally many of which are incorrectly linked things in scaladocs which
there are some differences in formatting. Hopefully this removes alot
if not all of the "link not found for ..." warning messages that show
up.
2019-07-31 12:28:22 +02:00
cwaldron97
f59d4007c3 Replace Bitcoins params in RPC client with CurrencyUnit (#618)
* implemented currency unit and wrote a writer

* Convert more Bitcoins parameters to CurrencyUnit
2019-07-30 18:52:01 -05:00
cwaldron97
7183296b6b Killing Joda Dependency (#643)
Remove Joda dependency
2019-07-29 12:22:25 +02:00
Torkel Rogstad
400219f74d Tweak error logging of bitcoind responses
In this commit we move the error handling part of bitcoind responses
closer to the source of where they will happen, hopefully
giving better error messages.
2019-07-22 13:33:22 +02:00
Torkel Rogstad
eb3fbbb259 Add types for various bitcoind RPC exceptions
In protocol.h (Bitcoin Core) all RPC errors are enumerated. The
exceptions added here are copied directly from that file.
2019-07-16 12:20:24 +02:00
Torkel Rogstad
56737681ad Fix bug where cookie auth threw on isStartedF 2019-07-12 11:05:17 +02:00
Torkel Rogstad
30e6d7030f Somewhat dirty standalone server and CLI binary (#558)
* PoC bitcoin-s-cli

* Add CLI, Server sbt projects, remove Ammonite

In this commit we set up sbt configuration for
CLI, Server (in-work-name) and corresponding
test projects.

We also remove Ammonite shell from sbt, as that
isn't really being used. bloop console offers
the same functionality way more ergonimic.

* Move BitcoinSAppConfig into new server project

Server project depends on node, chain wand wallet
so this is a good time for introducing this class
into main sources. We also introduce
BitcoinSTestAppConfig in testkit, to replace the
functionality in BitcoinSAppConfig related to
tests.

* Type chain in blockchainresult

* MVP server setup for node, chain and wallet

* Extremely dirty CLI for interacting with server

* initial attempt at mimicking Bitcoin Core API

* WalletStorage: add method for checking for seed existance

* Check for seed existance on wallet startup

* Fix bug where MnemonicNotFound was not an error

* Segregate confirmed and unconfirmed balance methods

* Add error handling, improve formatting of CLI output

* Tweak build

Bump Sttp version, downgrade to uPickle 2.11 compat,
skip publish in cli-test and server-test

* Add CLI, server and picklers to root project
2019-07-10 06:33:17 -05:00
Torkel Rogstad
2632e1a628 Change size_on_disk to a Long (#584) 2019-07-10 05:38:11 -05:00
cwaldron97
1fec083b0b fixed Default datadir 2019-07-09 17:05:55 -05:00
Torkel Rogstad
53ae772cb3 Merge pull request #563 from cwaldron97/2019-07-02-Windows-implementation-for-bitcoind
Add windows compatability to bitcoind RPC client
2019-07-04 10:19:45 +02:00
cwaldron97
8a576f91f6 fixed silly issue with DEFAULT_BITCOIND_LOCATION 2019-07-03 16:10:17 -05:00
cwaldron97
5171ee61ac testing exception message for CI 2019-07-03 15:21:57 -05:00
cwaldron97
dd4e7d5bf0 hopefully fixing RuntimeException 2019-07-03 15:15:20 -05:00
cwaldron97
136b59917c fixed windows issues 2019-07-03 14:27:16 -05:00
cwaldron97
3926030e24 need to figure out DOS regex handling 2019-07-02 15:53:18 -05:00