Commit graph

1727 commits

Author SHA1 Message Date
Chris Stewart
5d28e9143d
Add akka logging documentation to our contributing.md (#764) 2019-09-27 10:56:16 -05:00
Nadav Kohen
4ee36e84e9 Removed the sealed trait and private case class Impl pattern from ScriptPrograms (#759) 2019-09-26 07:31:54 -05:00
Chris Stewart
3f0506f7c0
Run scalafmt (#762) 2019-09-26 07:31:30 -05:00
Chris Stewart
dadd522c6a
Remove hard coded log level in test app config (#757)
* Remove hard coded log level in test app config

* Change log level in default reference.conf to WARN so we have the same log level as before
2019-09-25 15:47:27 -05:00
rorp
933f0fcfd0 Initial BIP157 support (#695)
* WIP: Initial BIP157 support

* store block hash and hetgh along with its compact filter header

* download and parse block filters

* getcfilters/cfilter

* cfilter table

* rescan full filter chain

* improved rescan performance

* optimize compact headers download

* addressed the PR comments

* split SVP and Neutrino node implementations

* configurable filter batch sizes

* initial filter sync

* addressed comments

* chage filter table's primary key

* fix Golomb filter deserialization

* batch database inserts

* neutrino node test

* fixed node test

* addressed the PR comments

* serializers tests

* cleanup

* fix compilation errors

* fix unit tests

* increase test coverage

* enable NeutrinoNodeTest

* make scalafmt happy

* don't cache experimental binaries

* inclease test coverage

* fix unit tests

* more granular CI tests

* disable NeutrinoNodeTest

* refactor tests

* addressed comments

* test coveage

* fix formatting

* responded to the comments

* some more changes

* fix the build

* test coverage

* revert testnet3 config parameter

* minor changes

* cleanup
2019-09-25 13:18:51 -05:00
Chris Stewart
71a136b03d
Fix bug where we were sending messages before we were fully initialized (#755)
* Fix bug where we were sending messages before we were fully initialized

* Move the pattern match out of isInitialized() future in sendMsg()
2019-09-24 09:43:27 -05:00
Scala Steward
c886681c7d Update sbt to 1.3.2 (#752) 2019-09-22 17:21:07 -05:00
Scala Steward
f9ec76009e Update akka-http, akka-http-testkit to 10.1.10 (#751) 2019-09-22 16:11:21 -05:00
Chris Stewart
12dcba42de
bump scalac version to 2.12.10 (#743)
* bump scalac version to 2.12.10

* bump scalac version to 2.12.10 in .travis.yml
2019-09-17 04:08:20 -05:00
Scala Steward
889580a294 Update sbt-scalafmt to 2.0.5 (#744) 2019-09-17 03:45:36 -05:00
Scala Steward
20a84d8d89 Update sbt-ci-release to 1.3.2 (#745) 2019-09-17 03:45:18 -05:00
Scala Steward
de41d63017 Update sttp:core to 1.6.6 (#741) 2019-09-10 01:27:38 -05:00
Scala Steward
8202d1074a Update sbt-scalafmt to 2.0.4 (#738) 2019-09-06 08:13:18 -05:00
Chris Stewart
46681e08d7
2019 09 05 scalafmt update (#735)
* Update scalafmt to 2.0.1

* Scalafmt

In this commit we
1) Reconfigure the .scalafmt.conf file
2) Add a scalafmt binary (same version as specified in conf file)
3) Use said binary on CI to check that everything is formatted correctly

* Ran scalafmt

* Update scalafmt ignore

* Add scalafmt plugin to project/plugins.sbt, run it

* run scalafmt
2019-09-05 20:02:58 -05:00
rorp
7197dd8f24 Fix WalletRpcTest (#736) 2019-09-05 16:09:11 -05:00
Chris Stewart
20a58c32e5
Add receiver param to BitcoindRpcTestUtil.fundBlockChainTransaction() (#737)
* Add receiver param to BitcoindRpcTestUtil.fundBlockChainTransaction()

* Actually implement awaiting for the hash to be seen
2019-09-05 13:18:32 -05:00
Chris Stewart
b7d3ec1225
2019 09 05 v18 rpc (#734)
* This is for colin

* Fix remaining errors on V18 RPC calls

* remove unused code final formatting fix

* implemented fixed listwalletdir rpc test

* responding to code review and replacing flatmaps
2019-09-05 09:15:00 -05:00
Scala Steward
5808807fb9 Update sbt to 1.3.0 (#732) 2019-09-05 07:55:03 -05:00
rorp
7eb883428e Make WalletRpcTest more determenistic (#731) 2019-09-04 06:51:33 -05:00
Torkel Rogstad
69d9c3a3f2 Add rudimentary docs for node, chain and wallet (#726) 2019-09-02 08:16:44 -05:00
Torkel Rogstad
a811f4a6d7 Deprecate generate (#728) 2019-09-01 07:46:38 -05:00
Torkel Rogstad
5d6a71a490 Make it possible to construct RPC client without ActorSystem (#725) 2019-08-30 15:12:18 -05:00
Torkel Rogstad
e18f105abc Refactor sbt and make Eclair tests download bitcoind (#727) 2019-08-30 15:11:52 -05:00
Chris Stewart
404da66cb5
Remove BlockchainBuilder, speed up ChainHandler test that processes m… (#723)
* Remove BlockchainBuilder, speed up ChainHandler test that processes mainnet headers, add test for ChainAppConfig

* Add test for BlockHeaderDAO.getAncestorAtHeight()

* Consolidate ChainAppConfigTest, add ScalaTestUtil to help with working with scalatest, speed up ChainHandlerTest a bit more
2019-08-27 12:29:22 -05:00
Torkel Rogstad
9ce9699853 Build and CI improvements (#710)
* Build and CI improvements

In this commit we:

1) Parallelize the Travis CI config, by splitting
    each project into its own Travis task
2) Download bitcoind binaries through sbt

* Use binaries downloaded by sbt task

* Make BitcoindRpcTestUtil work on Travis without bitcoind on PATH

* Add new downloadEclair task to sbt

* use sbt downloaded binaries in tests

* Fix Eclair and Bitcoind tests
2019-08-27 10:48:48 -05:00
Torkel Rogstad
68f8132070 Remove untyped CryptoUtil methods (#722)
Types are our friends:-)
2019-08-27 06:32:50 -05:00
Torkel Rogstad
c5fc133161
use 2.13 for website publish (#721) 2019-08-26 13:58:55 +02:00
Scala Steward
6170b0cd69 Update sttp:core to 1.6.4 (#664) 2019-08-26 13:54:56 +02:00
Scala Steward
36619cdbfe Update sbt-mdoc to 1.3.2 (#720) 2019-08-26 13:39:02 +02:00
Scala Steward
aa3a5158f1 Update flyway-sbt to 6.0.0 (#719) 2019-08-23 18:44:42 -05:00
Torkel Rogstad
a303818c1e Bump Scala versions (#697)
* Bump Scala versions

Support Scala 2.12.9
and 2.13.0.

To make this easier, we delete the `scripts` project. Everything
that was in here was covered by content on the website. We also
delete the `doc` folder, as that was a remnant from when `scripts`
was called `doc`.

* Crib uPickle akka-http support while we wait for publish

* Fix compiler warnings

* Add note on test logging to contribution guide

* Reduce duplication in Blockchain implementation

* Use Scala 2.12 for website

* Introduce compat package object for collections converters

* Fix Either compiler warnings

* Add sync-chain and create-wallet docs from deleted scripts

* Fix rebase goofup
2019-08-23 13:53:00 -05:00
Scala Steward
f0c9432601 Update flyway-sbt to 5.2.4 (#715) 2019-08-23 12:55:14 -05:00
Chris Stewart
62e959950b
fix monitor invoice timing out in #693 (#718)
* fix monitor invoice timing out in #693

* Bump timeout more
2019-08-23 11:57:53 -05:00
Torkel Rogstad
227b5239e5
Bump test coverage (#713)
* Bump test coverage

* Lower test coverage requirement for chain
2019-08-23 18:32:55 +02:00
Chris Stewart
ab170d05f9
2019 08 16 process header optimization (#701)
* WIP: implement processHeaders() in ChainHandler instead of processHeader()

* Create ConnectTipResult which is returned from Blockchain.connectTip(), this decouples 'BlockchainUpdate' and Blockchain.connectTip(). This was needed because it doesn't make sense to have a Vector[BlockHeaderDb] returned from Blockchain.connectTip() as we are validating one header. However, to execute batch inserts, we need to accumulate headers somewhere. I choose to do this in BlockchainUpdate with the 'successfulHeaders' field.

* make chain-verification log level INFO

* Add unit test for benchmarking ChainHandler.processHeaders() that cuts out extraneous noise in another test case

* Address code review
2019-08-23 05:48:09 -05:00
Scala Steward
f27edd5b16 Update sbt-native-packager to 1.4.0 (#711) 2019-08-22 09:56:34 -05:00
cwaldron97
a2bbffe3ab (WIP) Createwallet rpc from V17 (#712)
* create wallet rpc

* fixed test

* change to createwallet test

* create wallet test
2019-08-21 16:34:11 -05:00
Chris Stewart
1619d5d123
Use transactionally when inserting data into our database with createAll (#708) 2019-08-21 06:28:28 -05:00
Scala Steward
6a69228bae Update akka-actor, akka-stream, ... to 2.5.25 (#700) 2019-08-20 10:46:21 -05:00
Torkel Rogstad
8ba9626500 Tweak module logging (#698)
* Split app loggers into their corresponding modules

This avoids a bunch of places with awkward implicit resolution.

We also make sure that all logging in node, chain and wallet
happens with the app loggers, and not BitcoinSLogger

* Tune test logging level to WARN
2019-08-20 08:11:10 -05:00
Chris Stewart
4afdccb0a9
2019 08 05 broadcast tx test (#680)
* Start refactoring BroadcastTransactionTest to use new SpvNodeFundedWalletBitcoind fixture

* Double timeout to 10.seconds on spv node disconnect

* Attempt to bump disconnect even more

* Add more logging around starting and stoping spv node to try and debug failures with fixtures on test suites

* move starting of spv node into one place in the fixtures -- createSpvNode() --, also look at ordering of destruction to make sure that the spv node is stopped/destroyed BEFORE bitcoind is stopped/destroyed. If the inverse order happens, our spv node won't receive a response from bitcoind and will hang waiting for the Tcp connection to close gracefully

* Try switching from  to  to get CI to pass

* Add more logging and throws to try and debug failures on ci

* Add logging in the case we send a close command and peer isn't defined

* Match on Tcp.Connect on default receive in P2PClient

* Make SpvNode.disconnect() check to see if we are connected before attempting to disconnect

* Add more logging in fixtures, change behavior of SpvNode.disconnect() to not throw if we are already disconnected

* Switch to preferred way of terminating actor system in tests

* Make PeerMessageSender.sendMsg() return a , make sure a message is not sent to P2PClient until we are fully intialized

* Switch p2p logging to DEBUG

* Complete disconnect promise on peer message recv in the case where we didn't have a peer to begin with

* Make SpvNodeTest more robust, fix bug for when we send 'sendheaders' message. We shouldn't do this until our peer has sent us a verack

* Only send 'sendheaders' message to our peer after we receive a 'sendheaders' message from our peer. Hopefully this solves async issues where we were sending 'sendheaders' before our handshake was fully completed with our peer which means we wouldn't get headers sent to us

* Cleanup pt1
2019-08-14 07:18:36 -05:00
Chris Stewart
999d3aeba7
Make all instances of HashDigest extend AnyVal so we do not allocate runtime objects to avoid GCs (#690) 2019-08-14 06:35:10 -05:00
Chris Stewart
c89cfea924
2019 08 09 Don't use BlockHeaderDAO in TipValidation (#688)
* WIP: De-futurify TipValidatoin.chewNewTip()

* De-futrify POW and TipValidation, now tip connection is done synchronously thanks to our in memory blockchain implementation

* Fix improt issues, unused parameters
2019-08-13 10:33:19 -05:00
Scala Steward
c82197934a Update slf4j-api to 1.7.28 (#687) 2019-08-10 10:50:48 -05:00
Scala Steward
6a6011179e Update akka-actor, akka-stream, ... to 2.5.24 (#686) 2019-08-09 16:03:15 -05:00
rorp
81673a4151 Fix Eclair start/shutdown test (#684) 2019-08-09 15:56:33 -05:00
Chris Stewart
762168127c
Change 'height' types from Long -> Int. Int.MaxValue is 2147483647, which is an order of magnitude more than current block heights on mainnet/testnet bitcoin (#685) 2019-08-09 09:22:00 -05:00
Chris Stewart
020a935618
Start the process of refactoring our ChainHandler to be able to avoid… (#655)
* Start the process of refactoring our ChainHandler to be able to avoid database calls on TipValidation

WIP: Begin explicity passing state back and forth in return types of PeerMessageReceiver, P2PClient, , DataMessageHandler. This commit also implements the ability to keep our blockchain completely in memory. Previously when we were updating the tip of the chain, we had to make a database read to figure out what the best tips are. This is suboptimal for performance because a database read needs to be done for every block header we see, now we just keep the chain in memory

Fix bug in DataMessageHandler that pre-emptively sent a getheadersmsg to our peer. Make 'chainApiF' internal to our spvNode (not a parameter). This forces the chainApi to be created from disk everytime a new SpvNode is spun up. This keeps us in sync with the blockchain at disk at the cost of disk access and less modularity of SpvNode

Address torkel code review

Fix rebase issues

Address code review

Address nadav code review

* Rebase onto master, fix api changes
2019-08-06 13:31:54 -05:00
Chris Stewart
6258946f22
Address issue #673, also refactor fixture stuff to be in companion ob… (#676)
* Address issue #673, also refactor fixture stuff to be in companion objects rather than traits so they can be re-used without extending the trait itself. This allows for more modularity with our fixtures

* fix bugs in unit tests, address #675 to make MerkleBufferTests faster

* Refactor test timeout in UpdateBloomFitlerTest

* fix callback logic bug on matching unrelated txs/addresses in UpdateBloomFilterTest, re-order broadcasting of things to avoid async bugs hopefully

* Make  a def in NodeUnitTest, this keeps tests in the same suite from using the same config. This allows us to write multiple tests per suite, instead of just one. This also adds implicit parameters to our fixture constructors/destructors to properly create and destroy this config. The long term goal here also should be getting rid of config.initialize() we are calling everywhere as this is a anti-pattern, fixtures should take care of construction and destruction of things

* Broadcast tx earlier in test case for UpdateBloomFilterTest

* Rework NodeWithWalletTest to use the new fixtures we have in the node project, now use SpvNodeFundedWalletBitcoind
2019-08-06 11:49:17 -05:00
Scala Steward
1471983d27 Update slf4j-api to 1.7.27 (#677) 2019-08-06 07:13:32 -05:00