Commit graph

59 commits

Author SHA1 Message Date
Ben Carman
e247289c34
Make Akka log to file correctly (#1801) 2020-08-12 07:52:51 -05:00
Chris Stewart
fa388c7448
Create more of a project structure in org.bitcoins.core.api, move DbR… (#1799)
* Create more of a project structure in org.bitcoins.core.api, move DbRowAutoInc into the core project

* Get website compiling again
2020-08-12 06:13:23 -05:00
Chris Stewart
d31806f76f
Add synchronous version of StartStop (#1771)
* Add synchronous version of StartStop

* Use ben's suggestion of StartStopAsync extending StartStop
2020-08-06 18:24:12 -05:00
Ben Carman
eb09319c27
Have AppConfig use BitcoinNetwork.fromString (#1748) 2020-08-04 07:51:56 -05:00
Chris Stewart
5e80795c77
2020 07 25 optimize recalc chainwork (#1697)
* Optimize recalc Chainwork

* Typo and warning fixes

* Rename force recalc cli option

* Always update gensis header

* Add config option

* Fix config option

* Add to example config

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-07-31 10:43:04 -05:00
Ben Carman
930f184dca
Optimize getBestFilter functions (#1715)
* Optimize getBestFilter functions

* Remove debug statement

* Change varbinary to varchar, create index

* Fix getBlockchainFrom for tests

* Create index for postgres
2020-07-30 10:47:48 -05:00
Ben Carman
26a2529f50
CRUD Test suite + updateAll improvements (#1618)
* CRUD Tests

* Remove some hard coded values

* DeleteAll test optimization

* Make deleteAll transactional

* Fix deleteAll test
2020-07-30 10:10:56 -05:00
Nadav Kohen
7b8f17ade1
Added locks on start calls for loggers (#1713) 2020-07-29 10:12:49 -05:00
Ben Carman
728a4a841f
Fix Node Startup Issue (#1683)
* Fix Node Startup Issue

* Small fixes

* Add option to force chain work recalc
2020-07-29 09:37:20 -05:00
Nadav Kohen
436396773d
Fix All Loggers! (#1695)
* Fixed logging so that appenders are always started when logging is happening

* Started rolling policy so that file appenders could start in getLoggerImpl
2020-07-28 07:13:38 -05:00
Chris Stewart
6a1e4aea01
Add .transactionally to CRUD.updateAll(), CRUD.upsertAll(), CRUDAutoInc.createAll() (#1698) 2020-07-25 16:47:04 -05:00
Ben Carman
03da22b391
Fix length of BigIntMapper (#1651)
* Fix length of BigIntMapper

* Fix for max value uint256
2020-07-09 14:44:55 -05:00
Ben Carman
c2fa7d7cc8
Address Tagging Attempt 2 (#1320)
* Address & UTXO tagging

* Fix docs

* Remove useless function, improve docs

* Fix rebase errors

* Rebase fixes

* Fix docs

* Fix small test errors

* Fix Postgres migration

* Fix postgres
2020-07-08 14:38:39 -05:00
Ben Carman
102e5775f7
AppConfig Start Interface (#1598)
* AppConfig Start Interface

* Use StartStop interface
2020-06-21 09:33:43 -05:00
Ben Carman
586075e9f8
Temporary fix for update all on CRUDAutoInc Tables (#1596)
* Temporary fix for update all on CRUDAutoInc Tables

* Add comment
2020-06-20 08:46:59 -05:00
Scala Steward
e6d1471af0
Update scalafmt-core to 2.6.0 (#1579)
* Update scalafmt-core to 2.6.0

* Upgrade scalafmt, fix all formatting errors

* Scalafmt on 2.12

* fix test

* More fixes

Co-authored-by: Ben Carman <benthecarman@live.com>
2020-06-19 10:38:12 -05:00
Nadav Kohen
38a73de48f
Lots of misc. improvements from dlc branch (#1583) 2020-06-17 14:34:01 -05:00
Nadav Kohen
2b91c39616
Added compile scope scalafmtCheck to travis runs (#1563) 2020-06-16 15:16:43 -05:00
Nadav Kohen
6b1973264e
Fixed all walletTest threads not being closed (#1552) 2020-06-15 10:50:53 -05:00
Chris Stewart
8c0e64e101
Move slickDbConfig out of JdbcProfileComponent trait into AppConfig (#1510)
* Move slickDbConfig out of JdbcProfileComponent trait into AppConfig

* Cleanup newliens
2020-06-04 12:49:08 -05:00
Ben Carman
31807c9cbd
Implement best block hash correctly (#1452)
* Implement best block hash correctly

* Handle no common history

* Fix getBlockProof to be BigInt instead of UInt32

* Fix warnings

* Make get best filter use chainwork

* Remove unused function, test isMissingChainWork

* Fix comparisons for chain work in sql

* Fix migrations from rebase

* Fix CI error

* Fix postgresql
2020-06-04 12:05:10 -05:00
rorp
a9430c2d5a
PostgreSQL support (#1315) 2020-06-02 16:37:48 -05:00
Chris Stewart
0a35cdb6ef
Implement AppConfigFactory (#1462)
* Implement AppConfigFactory

* Fix docs
2020-05-25 14:43:37 -05:00
Ben Carman
c811ccc62a
Add option to correctly use logback config (#1398)
* Logger now uses correct parent logger for all, names appenders correctly

* Set logback to true for tests, rename file

* Fix docs
2020-05-20 15:46:24 -05:00
Nadav Kohen
4d9692f61e
Crypto Project Refactor (#1380)
* Added crypto project and decoupled BitcoinSLogger from NetworkElement

Decoupled BitcoinSLogger from Factory

Moved NetworkElement into crypto project

Moved Factory and BitcoinSUtil (renamed to BytesUtil) to crypto project

Moved MaskedToString to crypto project

Added BytesUtil to imports and cleaned up CryptoUtil.recoverPoint

Moved the rest of crypto stuff to the crypto project

Moved crypto tests to crypto-test project

* Added documentation for crypto project
2020-04-30 12:34:53 -05:00
Chris Stewart
64b13846d9
2020 04 08 multi db dao (#1355)
* Implement Slick's multi database support via our DAOs

WIP2 work on chain project

WIP make JdbcProfieComponent so we have a common JdbcProfile

WIP2

Clean up a few things

WIP got CRUDAutoInc compiling

Get closer to everything compiling in the chain project

Begin moving database code into JdbcProfileComponent

WIP source compiling

Get dbCommonsTest/ test cases passing

Get rest of tests compiling and passing

* Rebase onto master, fix conflicts

* Run scalafmt

* Run scalafmt again

* Cleanup a few compiler warnings by adding type annotations to thigns

* Explicitly cast tables where they are needed in the wallet project where we need TableQuery for foreignkey / joins in slick

* Redo definition of 'table' to use the profile.api.TableQuery in scope

* Rename all *Table.scala files to *Db.scala

* First crack at address ben's code review

* Fix docs compile issue

* Fixed casting (#30)

* Fixed AppConfig casting

* Fixed TableQuery casting in DbManagement subtypes

* Fixed casts in DAOs relating to internal types

* Address nadav's code review, move JdbcProfileComponent into it's own file

* Remove unused listTables() method

Co-authored-by: Nadav Kohen <nadavk25@gmail.com>
2020-04-25 11:28:58 -05:00
Ben Carman
80882bf649
Wallet Transaction Tracking (#1197)
* Incoming and Outgoing Transaction Tables

* Remove script sigs for witness txs

* Create parent tx_table for incoming and outgoing txs

* Response to review

* Use isCloseEnough

* Fix test

* Fix rebase error

* Test that tx is tracking all sats correctly
2020-04-02 06:55:09 -05:00
Chris Stewart
02d926bb25
Reduce default num threads used for slick being a thread pool of 20 to 5 (#1281) 2020-03-30 12:27:57 -05:00
Chris Stewart
4559edde07
Actually add all files for 0.3.0 on the website so they show up (#1256) 2020-03-19 07:32:02 -05:00
Ben Carman
5c77d549ac
Scalafmt Fix (#1117) 2020-02-12 14:22:55 -06:00
Nadav Kohen
a6e21fe43b
Multiple Logger Fix (#1086)
* An attempt to fix logging to file using MDC

* Responded to review
2020-02-11 11:35:28 -06:00
Chris Stewart
b50f818519
Create migration to drop confirmations column from txo_spending_info … (#1099)
* Create migration to drop confirmations column from txo_spending_info table in the wallet database, also fix a long standing issue in BitcoinSFixture where failure cases in the testing suite were not being handled properly

* Fix scala 2.11.12 compile issue
2020-02-07 07:45:15 -06:00
Chris Stewart
ea555c5705
If migrations fail, attempt to baseline the database and apply migrations again (#1058) 2020-01-23 13:28:50 -06:00
Ben Carman
c4ade3ba3c Txo state flyway (#1052)
* Add flyway migrations

* Make different project's migrations independent of each other

* Rework all AppConfig.initialize() to use migrations rather than what we were doing before

* TXO State migration

* Move to new file, drop old column

* Add block hash column

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2020-01-22 15:34:36 -06:00
Chris Stewart
1ad402ea56
Add more expressive txo states for the wallet (#1001)
* Add more expressive txo states for the wallet

* Rename 'PendingReceived/PendingSpent' -> 'UnconfirmedReceived/UnconfirmedSent'

* Address code review

* Fix posInt implementation to work with older scala versions

* Rename to PendingConfirmationReceived/PendingConfirmationSpent
2020-01-05 18:42:20 -06:00
rorp
d776e1c952 Automated wallet recovery (#985)
* Automated wallet recovery

* responded to the PR comments

* some more changes

* fix docs

* cleanup
2020-01-02 13:18:41 -06:00
Nadav Kohen
15ec7b89b6 Nicer Satoshis.apply (#899)
* Replaced Satoshis(Int64(_)) with Satoshis(_)

* Fixed SignerTest
2019-11-26 14:43:14 -06:00
rorp
ba91c6107c Create removed Neutrino tables (#796) 2019-10-14 14:42:59 -05:00
Chris Stewart
81dcdbeb57
2019 09 27 logging refactor pt2 (#765)
* Remove AKKA_CONFIG that was preventing sane logging for akka, move db-commons reference.conf to testkit so we can control testkit applications, and then rename app/server/.../application.conf to reference.conf like good libraries should do

* Update contributing.md

* update a few more links inside of contributing.md

* Run scalafmt
2019-09-28 09:50:42 -05:00
Chris Stewart
dadd522c6a
Remove hard coded log level in test app config (#757)
* Remove hard coded log level in test app config

* Change log level in default reference.conf to WARN so we have the same log level as before
2019-09-25 15:47:27 -05:00
rorp
933f0fcfd0 Initial BIP157 support (#695)
* WIP: Initial BIP157 support

* store block hash and hetgh along with its compact filter header

* download and parse block filters

* getcfilters/cfilter

* cfilter table

* rescan full filter chain

* improved rescan performance

* optimize compact headers download

* addressed the PR comments

* split SVP and Neutrino node implementations

* configurable filter batch sizes

* initial filter sync

* addressed comments

* chage filter table's primary key

* fix Golomb filter deserialization

* batch database inserts

* neutrino node test

* fixed node test

* addressed the PR comments

* serializers tests

* cleanup

* fix compilation errors

* fix unit tests

* increase test coverage

* enable NeutrinoNodeTest

* make scalafmt happy

* don't cache experimental binaries

* inclease test coverage

* fix unit tests

* more granular CI tests

* disable NeutrinoNodeTest

* refactor tests

* addressed comments

* test coveage

* fix formatting

* responded to the comments

* some more changes

* fix the build

* test coverage

* revert testnet3 config parameter

* minor changes

* cleanup
2019-09-25 13:18:51 -05:00
Torkel Rogstad
a303818c1e Bump Scala versions (#697)
* Bump Scala versions

Support Scala 2.12.9
and 2.13.0.

To make this easier, we delete the `scripts` project. Everything
that was in here was covered by content on the website. We also
delete the `doc` folder, as that was a remnant from when `scripts`
was called `doc`.

* Crib uPickle akka-http support while we wait for publish

* Fix compiler warnings

* Add note on test logging to contribution guide

* Reduce duplication in Blockchain implementation

* Use Scala 2.12 for website

* Introduce compat package object for collections converters

* Fix Either compiler warnings

* Add sync-chain and create-wallet docs from deleted scripts

* Fix rebase goofup
2019-08-23 13:53:00 -05:00
Chris Stewart
1619d5d123
Use transactionally when inserting data into our database with createAll (#708) 2019-08-21 06:28:28 -05:00
Torkel Rogstad
8ba9626500 Tweak module logging (#698)
* Split app loggers into their corresponding modules

This avoids a bunch of places with awkward implicit resolution.

We also make sure that all logging in node, chain and wallet
happens with the app loggers, and not BitcoinSLogger

* Tune test logging level to WARN
2019-08-20 08:11:10 -05:00
Chris Stewart
4afdccb0a9
2019 08 05 broadcast tx test (#680)
* Start refactoring BroadcastTransactionTest to use new SpvNodeFundedWalletBitcoind fixture

* Double timeout to 10.seconds on spv node disconnect

* Attempt to bump disconnect even more

* Add more logging around starting and stoping spv node to try and debug failures with fixtures on test suites

* move starting of spv node into one place in the fixtures -- createSpvNode() --, also look at ordering of destruction to make sure that the spv node is stopped/destroyed BEFORE bitcoind is stopped/destroyed. If the inverse order happens, our spv node won't receive a response from bitcoind and will hang waiting for the Tcp connection to close gracefully

* Try switching from  to  to get CI to pass

* Add more logging and throws to try and debug failures on ci

* Add logging in the case we send a close command and peer isn't defined

* Match on Tcp.Connect on default receive in P2PClient

* Make SpvNode.disconnect() check to see if we are connected before attempting to disconnect

* Add more logging in fixtures, change behavior of SpvNode.disconnect() to not throw if we are already disconnected

* Switch to preferred way of terminating actor system in tests

* Make PeerMessageSender.sendMsg() return a , make sure a message is not sent to P2PClient until we are fully intialized

* Switch p2p logging to DEBUG

* Complete disconnect promise on peer message recv in the case where we didn't have a peer to begin with

* Make SpvNodeTest more robust, fix bug for when we send 'sendheaders' message. We shouldn't do this until our peer has sent us a verack

* Only send 'sendheaders' message to our peer after we receive a 'sendheaders' message from our peer. Hopefully this solves async issues where we were sending 'sendheaders' before our handshake was fully completed with our peer which means we wouldn't get headers sent to us

* Cleanup pt1
2019-08-14 07:18:36 -05:00
Chris Stewart
c89cfea924
2019 08 09 Don't use BlockHeaderDAO in TipValidation (#688)
* WIP: De-futurify TipValidatoin.chewNewTip()

* De-futrify POW and TipValidation, now tip connection is done synchronously thanks to our in memory blockchain implementation

* Fix improt issues, unused parameters
2019-08-13 10:33:19 -05:00
Torkel Rogstad
31642aff44 Add configurable logging to data directory (#640)
* Add logging to data directory

In this commit we add the ability for the node, chain
and wallet projects (+ the server) to log to the users
data directory instead of whatever directory the
binaries was launched from. This is inherently a bit
more complicated than our previous setup, because
we need to read the user directory before we can create
loggers. As a result of this, some files/methods were
moved around, so the relevant app config could be
found in scope.

We also  introduce several logging categories that can be
tuned individually through user configuration. These logggers
are exposed both as traits that give a field `logger`, or
as methods that return the required logger.

* Add datadir configuration to AppConfig

In this commit we add support for AppConfig to pick up
the data directory configuration file. We also add
a section to the contributing guide on how to tune
logging levels.

* Pass data directories explicitly for configuration
2019-08-01 06:01:56 -05:00
cwaldron97
b6a5ffea4d Cleaning up Publish Logs (#634)
* This PR is attempting to remove the publishing errors when publish
locally many of which are incorrectly linked things in scaladocs which
there are some differences in formatting. Hopefully this removes alot
if not all of the "link not found for ..." warning messages that show
up.
2019-07-31 12:28:22 +02:00
Torkel Rogstad
8125bca7c2 Add functionality for broadcasting TXs to node (#577)
* Add functionality for broadcasting TXs to node

In this commit we add functionality and tests
for broadcasting a TX from our node. To accomplish
this we introduce a table over broadcastable TXs
that's added to when the externally facing method
broadcastTransaction(tx) withing SpvNode is called.
We send out a inv message for the TX we just added,
and upon receiving a getdata message we search in
the previously mentioned table for entries where
the hashes match up.

* Broadcast TX from server to SPV node

* Perform assertions on the balance of bitcoind after sending a TX

* Remove typeclass from broadcast TX

* Refactor withFundedWalletAndBitcoind

* Match on BitcoindExecption instead of throwable

* Clean up broadcast functionality after code review
2019-07-17 07:32:05 -05:00
Torkel Rogstad
53af971b10 Add all wallet outpoints to bloom filter
This is a workaround for Bloom filters never being
updated for SegWit. See comment in
LockedWallet#getBloomFilter for more context.
2019-07-16 14:18:29 +02:00