Commit Graph

539 Commits

Author SHA1 Message Date
Chris Stewart
d162242a39
2024 03 24 v21 rpc refactor (#5494)
* WIP: Consolidate V19FilterRpc into BlockchainRpc

* Implement ChainApi.getFilterHeader() with bitcoind

* Make {BitcoindV22RpcClient,BitcoindV23RpcClient,BitcoindV24RpcClient} extend BitcoindRpcClient directly

* Fix MiningRpcTest

* Remove cast in BitcoindBackendTest
2024-03-26 13:04:26 -05:00
Chris Stewart
177542a13c
Remove grizzled.sl4fj was its no longer maintained (#5482) 2024-03-18 16:48:58 -05:00
Chris Stewart
708ce41f24
Re-implement bitcoin-s.node.query-wait-time (#5470)
* Add sentQuery Instant to SyncNodeState subclasses, add logic in DataMessageHandler.handlePayload() to detect if a query has timed out

* Refactor to use NodeTestUtil.disconnectNode()

* Cleanup comments
2024-03-11 09:54:44 -05:00
Chris Stewart
55e4dab86e
Rework NodeApi.downloadBlocks() to use DoubleSha256DigestBE (#5456)
* Rework NodeApi.downloadBlocks() to use DoubleSha256DigestBE rather than DoubleSha256Digest

* Fix test cases

* Fix docs

* Fix compile
2024-03-05 15:19:32 -06:00
Chris Stewart
ae3a634703
2024 03 01 Fix syncPeer exceptions (#5446)
* Add NodeRunningState.removePeer(), add helper method onDisconnectNodeStateUpdate() to consolidate disconnect logic

* Remove duplicate logic

* Add bitcoin-s.node.maxConnectedPeers test

* Clean up resources in test case

* Remove unecessary replacePeers()

* Add NodeState.addPeer()

* Add NodeRunningState.connectedPeerCount, separate handling of connection slots and filter slots
2024-03-04 09:22:16 -06:00
Chris Stewart
8b23b1f4f6
Add delay to wait for zmq to get setup in chainTest (#5427)
* Add delay to wait for zmq to get setup in chainTest

* Add println

* only run zmq tests on CI

* Bump Thread.sleep from 1ms -> 10ms

* Empty commit to re-run CI

* Revert things

* try making thread names more unique

* try no sleep

* Remove println

* Put logging back

* Try removing ZMQ.NOBLOCK

* turn off parallelExecution to see if it matters

* Add more logging

* Re-enable parallelExecution

* Try catching all NonFatal exns

* Bump timeout

* Bump maxTries

* Try downgrading jeromq back to 0.5.4

* Revert things

* scalafmt
2024-03-03 11:54:50 -06:00
Chris Stewart
8f812c644f
Fix Promise already completed errors in chainTest (#5440) 2024-02-29 14:53:55 -06:00
Chris Stewart
d1ecd35e63
Fix PeerDb.lastSeen race condition in unit tests (#5435)
* Disconnect from bitcoind side rather than bitcoin-s side to make sure we don't run into a race condition for updating lastSeen

* Remove usage of CachedAppConfig, switch implicit param args for PeerDAO to be consistent with rest of code base

* Fix bug in getNodeURIFromBitcoind(), disconnect from bitcoind side in PeerManagerTest

* Add println to see why still fialing

* Add PeerConnection.getLocalAddress, add parameter localAddressBitcoinS  to NodeTestUtil.getNodeURIFromBitcoind() to make sure we are getting the correct URI

* Empty commit to run CI
2024-02-28 10:33:01 -06:00
Chris Stewart
783ed1903d
Rename bitcoin-s.node.inactivity-timeout -> bitcoin-s.node.health-check-interval (#5418) 2024-02-23 08:31:22 -06:00
Chris Stewart
1dff918848
2023 02 21 pekko (#5413)
* Get dependencies resolving, get tor module compiling

* Get everything compiling with pekko

* Rework configs/logging to use pekko namespace

* Rename AkkaUtil -> PekkoUtil, BitcoinSAkkaAsyncTest -> BitcoinSPekkoAsyncTest
2024-02-22 10:26:21 -06:00
Chris Stewart
3177ee405f
Move away from suredbits provided peers by default (#5390)
* Move away from suredbits provided peers by default

* Remove setting bitcoin-s.node.use-default-peers

* Don't allow peer to stay connected if it doesn't support compact filters

* Move disconnection logic into managePeerAfterInitialization()

* Remove reconnect() when we have no other peers, send getaddr message after we know we are keeping the peer connection

* Remove hasCf flag check, revert log

* Reduce inactivity-timeout to 5 minutes by default, shuffle peers in PeerFinder beforing pushing onto stack

* Add logic to disconnect AttemptToConnect peers after addr mesage is received

* Create PeerFinder.queryForPeerConnections(), call it directly from PeerManager on health checks rather than restarting PeerFinder

* scalafmt

* Move more logic into managePeerAfterInitalization()

* Don't set isPersistent for db peers by default

* Add call to queryForPeerConnections() inside of PeerManager.onDisconnect() to try to attempt to more peers when we have no more connections, fix MisBehavingPeer logic to not start sync, starting sync should happen in onDisconnect()

* Revert logback-test.xml

* Sort peers we are attempting to connect to by lastSeen parameter

* Refactor DoneSyncing.toHeaderSync() to return Option[HeaderSync] to represent the case where we don't have a peer to sync with

* scalafmt

* Remove duplicate PeerFinder.buildPeerData()
2024-02-20 12:12:57 -06:00
Chris Stewart
e4fbfe73e6
2024 02 16 fix get filter sync marker from stop block header bug (#5401)
* Fix bug in getFilterSyncMarkerFromStopBlockHeader where we were previously querying for children rather than the block hashes at the given height for building the FilterSyncMarker for getcfilter request. Also introduce delay in sending getcfheader because of https://github.com/bitcoin/bitcoin/issues/27085

* Only fallback to reorg scenario if we find no headers at height + 1

* Remove the best filter's block header from the candidate headers in reorg scenarios
2024-02-17 06:26:55 -06:00
Chris Stewart
1685c84eea
2024 01 31 Refactor BitcoindRpcTestUtil.{startSevers(),stopSevers()} to use Future.traverse() (#5382)
* Refactor BitcoindRpcTestUtil.startServers() to use Future.traverse()

* Refactor BitcoindRpcTestUtil.stopServers() to use Future.traverse()
2024-01-31 15:20:36 -06:00
Chris Stewart
018631dad4
Create NodeTestUtil.awaitConnectionCount() (#5378)
Refactor ReConnectionTest to use NodeTestUtil.awaitConnectionCount()

Refactor more of codebase to use NodeTestUtil.awaitConnectionCount()

Refactor PeerManagerTest to use NodeTestUtil.awaitConnectionCount()

Refactor more nodeTest to use NodeTestUtil.awaitConnectionCount()

use bitcoinds.size for expectedConnectionCount
2024-01-31 07:39:20 -06:00
Chris Stewart
76e468c5c4
2024 01 28 Refactor PeerFinder.start() to avoid initializing connections (#5376)
* Conslidate PeerFinder.start() logic into peerConnectionScheduler, add bitcoins-s.node.try-peers-start-delay config settig to indicate how long until we start attempting to connect to peers in PeerFinder

* Cleanup

* Revert logback-test.xml

* Add documentation

* Empty commit to run CI

* Rework NodeAppConfig.peers to return Vector[Peer] rather than Vector[String]

* Add higher priority to paramPeers
2024-01-29 09:46:31 -06:00
Chris Stewart
3a927edee1
Fix bug when requesting FilterSyncMarker for filter headers (#5344)
* Fix bug when requesting FilterSyncMarker for filter headers outside of our in memory blockchain's range

* Rework buildNHeaders to be faster, move it to ChainUnitTest
2024-01-06 09:59:26 -06:00
Chris Stewart
b39736fb8d
Rework NodeTestUtil to use a specific bestBlockHash (#5332)
* Rework NodeTestUtil to use a specific bestBlockHash, this is useful in reorg situations

Use param in reorg test, modify scaladoc

WIP

* Cleanup

* Try to fix usage of stopHashBE

* Cleanup

* WIP: Fix getCompactFilterStartHeight()

* Revert logback-test.xml

* Fix bug with isFiltersSynced() in reorg situations

* scalafmt, fix compile

* Fix another bug with isFiltersSynced()

* Fix compile
2023-12-31 17:38:12 -06:00
Chris Stewart
5e81ec5ed2
Add ChainApi.getBestChainTips() (#5338)
* Add ChainApi.getBestChainTips()

* Implement BitcoindRpcClient.getBestChainTips()

* Revert NodeTestUtil

* Use ChainApi.getBestChainTips() throughout codebase rather than blockHeaderDAO.getBestChainTips()
2023-12-31 16:54:40 -06:00
Chris Stewart
d983a1bac4
2023 12 24 Make filter headers / filters sync work with block header reorgs (#5333)
* Rework ChainApi.nextBlockHeaderBatchRange() to take stopHash parameter, get chainTest/test passing

* WIP: Refactor ChainApi.nextFilterHeaderBatchRange()

* Finish ChainApi.nextFilterHeaderBatchRange() refactor, get all tests passing except reorg related tests in nodeTest

* Get NeutrinoNodeTest reorg test case working

* Improve flaky test

* Cleanup

* Switch sync check to bitcoinds(1)

* Empty commit to run CI

* Implement batchSize tests in ChainHandlerTest

* Rework ChainHandlerTest to not assume we have the genesis filter header / filter inserted into the database already

* Cleanup println

* Fix bug with nextFilterHeaderBatchRange() wrt to startHeightOpt parameter

* Fix off by one bug with compact filter sync

* Add check for connectionCount

* Add longer timeout

* scalafmt
2023-12-28 10:43:01 -06:00
Chris Stewart
16fb5d2dad
2023 12 19 compiler opts (#5330)
* Enable more compiler options

* enable -Xlint:valpattern, fix bug with out Test / scalacOptions were set with incorrect source code compiler flags

* Enable -Xlint:eta-sam

* Fix cryptoTestJVM

* Fix cryptoTestJS/test

* Fix asyncUtilTest

* Fix asyncUtilTest
2023-12-20 05:42:01 -06:00
Chris Stewart
01537185bd
2023 12 13 mv connectioncount check fixtures (#5324)
* Move peer connection count checks into test fixtures

* Add connection count check in another test fixture

* scalafmt
2023-12-14 06:19:47 -06:00
Chris Stewart
b252c2d6a2
2023 10 16 Implement WalletCallbackStreamManager, DLCWalletCallbackStreamManager (#5263)
* WIP: Implement WalletCallbackStreamManager

* Implement stopping of WalletCallbackStreamManager in WalletAppConfig

* WIP: DLCWalletCallbackStreamManager

* Refactor DLCWalletCallbacks to be a trait

* Integrate DLCWalletCallbackStreamManager into WebsocketUtil

* Use  java.util.concurrent.atomic.AtomicBoolean rather than monix

* Remove logger param from dlc callbacks

* fix docs

* Call super.stop() in DLCappConfig

* Empty commit to run CI

* Try to debug

* Empty commit to run CI

* Revert things
2023-10-19 11:52:07 -05:00
Chris Stewart
531c909597
Use BitcoindRpcBackendUtil.buildBitcoindNodeApi() in wallet test fixtures, re-implement getConfirmationsForBlocks() to use akka streams to avoid max-open-requests limit inside of akka (#5259) 2023-10-12 18:02:06 -05:00
Chris Stewart
12e5dfc021
Fix chain fixtures to use ChainHandler rather than always using ChainHandlerCached (#5243)
* Fix chain fixtures to use CHainHandler rather than always using ChainHandlerCached

* Remove unecessary ChainAppConfig.start()
2023-09-25 11:13:59 -05:00
Scala Steward
d7037ede28
Update scala-library to 2.13.12 (#5235)
* Update scala-library to 2.13.12

* Fix compiler errors on scala 2.13.12

---------

Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
2023-09-12 10:45:17 -05:00
Chris Stewart
b13e0565af
Remove PeerMessageReceiver (#5182)
* Remove PeerMessageReceiver

* Dont use QueueOfferResult, use akka.Done
2023-08-06 13:10:11 -05:00
Chris Stewart
c238191209
2023 08 04 rm peermsgrecvstate (#5177)
* WIP: Remove PeerMessageReceiverState from PeerMessageReceiver

* Get tests passing
2023-08-06 11:57:04 -05:00
Chris Stewart
2032b16620
Fix bug where we just need to awaitSyncAndIBD() rather than attempt to sync() (#5158)
* Fix bug where we just ned to awaitAllSync() rather than attempt to sync()

* Use awaitSyncAndIBD, mv awaitSyncAndIBD to NodeTestUtil

* Replace more NodeUnitTest.syncNeutrinoNode() with NodeTestUtil.awaitSyncAndIBD

* Replace more NodeUnitTest.syncNeutrinoNode() with NodeTestUtil.awaitSyncAndIBD
2023-07-21 17:08:01 -05:00
Chris Stewart
1ccc6a9230
2023 07 19 cleanup nodetest (#5156)
* Remove explicit call to sync()

* Remove extra call to NodeUnitTest.syncNeutrinoNode() in test fixture

* Add NeutrinoNodeNotConnnectedWithBitcoinds

* Don't switch sync peer if its the same peer we are currently syncing from it

* Fix another test

* Remove another sync() call

* Add carve out with MisBehavingPeers for the case where the badPeer was our last peer

* Remove more .sync() calls
2023-07-21 09:18:48 -05:00
Chris Stewart
e08469901b
Make PeerMessageSender.reconnect() return a Future that is completed when the connection is established (#5155)
* Make PeerMessageSender.reconnect() return a Future that is completed when the connection is established

* Replace hard require() with AsyncUtil.retryUntilSatisfiedF()
2023-07-19 13:10:15 -05:00
Chris Stewart
44190a535c
Defensively try to sync everytime a peer is a initialized (#5154)
* Defensively try to sync everytime a peer is a initialized

* Turn logging OFF

* Fix DataMessagehandlerTest, remove calls to NeutrinoNode.sync() in test fixtures
2023-07-19 09:43:23 -05:00
Chris Stewart
e66c078863
First attempt at implementing inactivityChecks() (#5144)
* First attempt at implementing inactivityChecks()

* Move lastParsedMessage logic into PeerMessageSender

* Add bitcoin-s.node.inactivity-timeout config option

* implement unit test

* Fix CommonSettings

* scalafmt

* scalafmt

* Rename lastSuccessfulParsedMsg -> lastSuccessfulParsedMsgOpt

* make sure we disconnect in the case the stream terminates with an error

* Reduce log level
2023-07-16 16:03:16 -05:00
Chris Stewart
b1e6488bb7
Move Socks5ProxyParams, Credentials into core (#5138) 2023-07-08 13:57:44 -05:00
Chris Stewart
1d82ed04a7
2023 06 18 implement tcp connection using akka streams (#5111)
* WIP: Try to move byte streaming/parsing of p2p messages out of P2PClient

* WIP2: Work on killing the actor, replace it with a steram

* Get basic ability to send/receive version message working

* Transition PeerMessageReceiverState to Initializing inside of PeerMessagesender.connect()

* Refactor things out of PeerMessageSender.connect(), add some flow logs

* Get NeutrinoNodeTest be able to sync passing

* Fix some bugs, create ConnectionGraph helper class

* Use killswitch rather than Source.maybe to disconnect peer

* WIP: Debug

* Switch halfClose to false on Tcp.outgoingConnection() to not keep lingering connections

* Delete P2PClientActorTest

* Delete all P2PClient stuff

* Attempt implementing reconnection logic in PeerMessageSender

* remove supervisor

* Empty commit to re-run CI

* Small cleanups

* Implement sendResponseTimeout()

* Restore logback-test.xml

* Add callback to log error message on reconnect

* Increase queueSize/maxConcurrentOffers size
2023-06-24 07:53:44 -05:00
Chris Stewart
5ae4993bed
Move PeerMessageReceiverState into PeerMessageReceiver (#5110)
* Move PeerMessageReceiverState into PeerMessageReceiver

* Delete PeerMessageReceiverTest
2023-06-19 15:04:35 -05:00
Chris Stewart
458a80d854
Destroy wallet config when tearing down node (#5105) 2023-06-15 13:39:37 -05:00
Chris Stewart
bed670fb6f
Remove chainApi parameter from NeutrinoNode (#5101)
* Remove chainApi parameter from NeutrinoNode

* Fix doc
2023-06-13 12:07:04 -05:00
Chris Stewart
ae8c97a4d1
Rework Node.sync() to return Future[Peer] rather than Future[Option[Peer]] (#5096) 2023-06-08 10:30:26 -05:00
Chris Stewart
ebe98959ed
Upgrade to scalac 2.13.11 (#5097)
* Upgrade to scalac 2.13.11

* Fix compile issues with scalac 2.13.11
2023-06-08 10:27:39 -05:00
Chris Stewart
295be36d63
Get PeerMessageSenderApi using akka streams for outbound p2p messages (#5069)
* Get PeerMessageSenderApi using akka streams for outbound p2p messages

* Use offer method rather than accessing queue directly

* Fix flaky unit test

* Empty commit to re-run CI

* Move methods for requesting filterheaders/filters into PeerManager, now use akka stream for those outbound p2p message

* Move sendInventoryMessage to PeerManager

* Move sendGetHeadersMessage() methods to PeerManager

* WIP: move more methods to PeerMessageSenderApi

* WIP2

* initialize stream before calling PeerFinder.start() so oubound messages get processed

* Rebase

* Make queue buffer size dependent on maxConnectedPeers

* Change state to HeaderSync() if we are re-querying for block headers

* Empty commit to re-run CI

* Remove PeerMessageSender from handleDataPayload()

* Limit access to PeerManager.peerMessageSenders

* revert a few things

* Fix rebase issues

* Fix rebase

* Turn down logging

* Rebase

* remove guard that checks peer size before labeling as MisBehavingPeer

* Fix small bug where we needed to switch syncPeer and we weren't

* Empty commit to run CI

* Empty commit to re-run CI

* Fix test case where we weren't awaiting for Node.sync() to return Some(peer)

* Empty commit to re-run CI

* Fix another reference where we were calling Node.sync() too soon after Node.start()

* scalafmt

* Add another retryUntilSatisfied() on NeutrinoNode.sync()
2023-06-08 08:47:24 -05:00
Chris Stewart
abeaaa05de
Remove awaitPeerWithServices() (#5093)
* Remove awaitPeerWithServices()

* Empty commit to run CI

* Rework Node.sync() to return Future[Option[Peer]] rather than Future[Unit]. This returns the peer we are syncing with, if we could find one to sync with

* Turn logging OFF again

* Empty commit to re-run CI

* Use AsyncUtil.retryUntilSatisfied() when calling node.sync() after starting node to make sure we have a peer to sync from in a test case

* Await on re-started node not stale reference in NeutrinoNodeWithWalletTest

* Fix second reference

* Empty commit to re-run CI
2023-06-07 04:47:39 -05:00
Chris Stewart
e74af8bf24
Move P2PClientCallbacks.onStop() into disconnection logic rather than Actor.postStop() (#5089)
* Move P2PClientCallbacks.onStop() into disconnection logic rather than Actor.postStop()

* Rename onStop -> onDisconnect

* Add reconnect flag to P2PCallbacks.onDisconnect so we don't attempt to reconnect when not necessary

* Rename flag to forceReconnect, check getPeerConnectionCount in PeerManager.onP2PClientDisconnect to see if we have 0 connections. If we do reconnect

* Add PeerManager.isStarted flag, guard reconnection logic with this flag in onP2PClientDisconnected()

* Clear PeerFinder._peerData when PeerFinder.stop() is called

* WIP: Move disconnection logic into stream

* Rework ordering of PeerManager.stop() to shutdown queue after peers are removed

* Empty commit to re-run CI

* Await for getConnectionCount async in test case

* Try increasing queue size

* Bump queue size to 16

* Put initialization, initialization timeout logic in queue rather than callbacks

* Make messages that are sent to the queue rather than callbacks for various control messages

* Empty commit to re-run CI

* Empty commit to re-run CI

* Empty commit to re-run CI

* Remove P2PCallbacks all together

* Re-add PeerMessageReceiverTest
2023-06-05 10:39:02 -05:00
Chris Stewart
61e142a631
2023 05 29 peermanager peerfinder refactor (#5086)
* WIP

* Reset PeerManager.dataMessageHandler to None on PeerManager.stop()

* WIP2

* Always set dataMessageHandler.peerData we process a new message in the stream

* remove PeerManager from PeerDatta

* Make PeerFinder mutable inside of PeerManager, pass the queue into PeerFinder to decouple PeerManager and PeerFinder

* Don't verify actor system shutdown for now
2023-05-31 13:35:46 -05:00
Chris Stewart
9202e63c90
Create P2PClientCallbacks to encapsulate callbacks (#5084) 2023-05-30 07:40:52 -05:00
Chris Stewart
34df4ccbb1
Make ControlMessageHandler take PeerManager rather than Node as a param (#5081)
* Make ControlMessageHandler take PeerManager rather than Node as a param

* refactor PeerData to not take a reference to Node

* Move ControlMessageHandler out of {Node,NeutrinoNode}
2023-05-29 09:01:06 -05:00
Chris Stewart
777743989a
Remove PeerHandler as it isn't used in src (#5075)
* Remove PeerHandler as it isn't used in src

* Fix rebase
2023-05-22 11:11:03 -05:00
Chris Stewart
ce6d2212c1
2023 04 22 peermanager dmh refactor (#5057)
* WIP: Move DataMessageHandler into PeerManager

* Get things compiling

* Turn off logging
2023-04-24 08:14:02 -05:00
Chris Stewart
18482c7e44
2023 04 20 decouple node (#5049)
* Remove node parameter to PeerMessageReceiver, pass the ControlMessageHandler and DataMessageHandler as parameters

* Remove node reference in DataMessageHandler, just pass PeerManager as parameter

* Fix pattern match to be exhaustive

* Move fetchCompactFilterHeaders, sendFirstGetCompactFilterHeadersCommand out of DataMessagehandler to decouple DataMessageHandler, PeerManager

* scalafmt
2023-04-20 18:39:30 -05:00
Chris Stewart
447c6d03de
Implement getconnectioncount rpc (#5048)
* Implement getconnectioncount rpc

* Reduce waits in test, add documentation

* Fix docs

* Empty commit to re-run CI
2023-04-19 16:47:54 -05:00
Chris Stewart
51429a7d68
2023 04 17 refactor peer message receiver (#5045)
* WIP: Refactor PeerMessageReceiver methods into PeerMessageReceiverState

* Remove state helper methods from PeerMessageReceiver

* WIP: Remove PeerMessageReceiverState from PeerMessageReceiver

* Get things compiling and tests passing

* Remove currentPeerMessageHandlerReceiver

* Refactor PeerMessageReceiverTest to not use bitcoind
2023-04-18 10:25:24 -05:00