Commit graph

4380 commits

Author SHA1 Message Date
Chris Stewart
878dce7945
Use Await.result() in DataMessageHandlerTest to avoid test cases that never complete (#5159) 2023-07-24 07:57:54 -05:00
Chris Stewart
2032b16620
Fix bug where we just need to awaitSyncAndIBD() rather than attempt to sync() (#5158)
* Fix bug where we just ned to awaitAllSync() rather than attempt to sync()

* Use awaitSyncAndIBD, mv awaitSyncAndIBD to NodeTestUtil

* Replace more NodeUnitTest.syncNeutrinoNode() with NodeTestUtil.awaitSyncAndIBD

* Replace more NodeUnitTest.syncNeutrinoNode() with NodeTestUtil.awaitSyncAndIBD
2023-07-21 17:08:01 -05:00
Chris Stewart
1ccc6a9230
2023 07 19 cleanup nodetest (#5156)
* Remove explicit call to sync()

* Remove extra call to NodeUnitTest.syncNeutrinoNode() in test fixture

* Add NeutrinoNodeNotConnnectedWithBitcoinds

* Don't switch sync peer if its the same peer we are currently syncing from it

* Fix another test

* Remove another sync() call

* Add carve out with MisBehavingPeers for the case where the badPeer was our last peer

* Remove more .sync() calls
2023-07-21 09:18:48 -05:00
Chris Stewart
4a344f0187
Fix bug in ChainApi.processFilterHeaders() (#5157)
* Fix bug in ChainApi.processFilterHeaders()

* Reduce log level for case where we are processing filter headers we have already seen before
2023-07-19 17:09:04 -05:00
Chris Stewart
e08469901b
Make PeerMessageSender.reconnect() return a Future that is completed when the connection is established (#5155)
* Make PeerMessageSender.reconnect() return a Future that is completed when the connection is established

* Replace hard require() with AsyncUtil.retryUntilSatisfiedF()
2023-07-19 13:10:15 -05:00
Chris Stewart
44190a535c
Defensively try to sync everytime a peer is a initialized (#5154)
* Defensively try to sync everytime a peer is a initialized

* Turn logging OFF

* Fix DataMessagehandlerTest, remove calls to NeutrinoNode.sync() in test fixtures
2023-07-19 09:43:23 -05:00
Chris Stewart
f034435430
Move setSyncing into PeerManager.syncHelper() (#5153) 2023-07-18 15:23:01 -05:00
Chris Stewart
ba8a0bf75a
2023 07 17 coretest script refactor (#5150)
* Refactor coreTest script test cases to consolidate

* Refactor blockchain tests in coreTest

* Refactor transaction tests in coreTest

* Refactor address tests in coreTest

* Fix missing asserts

* Fix missing asserts
2023-07-17 12:15:47 -05:00
Chris Stewart
e66c078863
First attempt at implementing inactivityChecks() (#5144)
* First attempt at implementing inactivityChecks()

* Move lastParsedMessage logic into PeerMessageSender

* Add bitcoin-s.node.inactivity-timeout config option

* implement unit test

* Fix CommonSettings

* scalafmt

* scalafmt

* Rename lastSuccessfulParsedMsg -> lastSuccessfulParsedMsgOpt

* make sure we disconnect in the case the stream terminates with an error

* Reduce log level
2023-07-16 16:03:16 -05:00
dependabot[bot]
516bfdd007
Bump semver from 5.7.1 to 5.7.2 in /website (#5149)
Bumps [semver](https://github.com/npm/node-semver) from 5.7.1 to 5.7.2.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/v5.7.2/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v5.7.1...v5.7.2)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-16 07:19:32 -05:00
dependabot[bot]
4622080b5b
Bump json5 from 2.2.1 to 2.2.3 in /website (#4943)
Bumps [json5](https://github.com/json5/json5) from 2.2.1 to 2.2.3.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v2.2.1...v2.2.3)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-15 15:58:21 -05:00
Chris Stewart
4bd5616e67
Make sure PeerData.stop() is called when PeerFinder.removePeer() is called (#5147)
* Make sure PeerData.stop() is called  when PeerFinder.removePeer() is called

* Fix log, add a new log
2023-07-15 09:22:14 -05:00
Scala Steward
115da94229
Update sbt to 1.9.2 (#5142) 2023-07-10 12:11:13 -05:00
Chris Stewart
ebe79287af
Implement socks5 proxy in PeerMessageSender (#5136)
* WIP: Implement socks5 proxy in PeerMessageSender

* Get something working

* Refactor to use Either when passing a socks5 message or non socks5 ByteString downstream

* Socks5Message -> Socks5MessageResponse

* Revert things

* More cleanup

* Fix rebase

* Move socks5Handler() Flow into Socks5Connection

* Revert NeutrinoNode

* Implement auth for socks5 proxy in stream

* Cleanups
2023-07-10 06:56:15 -05:00
Chris Stewart
2b117d349a
Remove duplicate Peer class (#5141) 2023-07-09 16:10:49 -05:00
Scala Steward
cde3977312
Update sbt-scalajs-crossproject to 1.3.2 (#5137) 2023-07-08 13:57:56 -05:00
Chris Stewart
b1e6488bb7
Move Socks5ProxyParams, Credentials into core (#5138) 2023-07-08 13:57:44 -05:00
Chris Stewart
ee619051a3
2023 07 04 node refactors (#5130)
* Make PeerData.peerMessageSender not wrapped in Future

* Remove peerMessageSenderApi param

* Small cleanups

* Rename DataMessageHandlerState -> NodeState

* Cleanups in DataMessageHandler

* Refactor StreamDataMessageWrapper -> NodeStreamMessage

* More small fixes

* Make handleDataPayload() take PeerData rather than Peer

* Move peers into NodeState and remove as param in DataMessageHandler

* replacePeers whenever PeerManager.{onP2PClientDisconnected,onInitialization} is called

* rework ValidatingHeaders.toString()

* Empty commit to run CI
2023-07-07 08:11:39 -05:00
Scala Steward
51f5f53322
Update client3:core to 3.8.16 (#5134) 2023-07-07 08:11:09 -05:00
Chris Stewart
9b6bca06c0
Fix duplicate filter header sync by adding delay before attempting to sync filter headers (#5132)
* Fix duplicate filter header sync by adding delay before attempting to sync filter headers

* Fix bug where we don't wait for AsyncUtil.nonBlockingSleep()
2023-07-06 12:05:14 -05:00
Chris Stewart
fc99087c89
Ignore messages in queue that were queued before we disconnected peer (#5131) 2023-07-05 15:31:27 -05:00
Scala Steward
cd812ac7a3
Update janino to 3.1.10 (#5128) 2023-07-05 14:38:22 -05:00
Chris Stewart
5e6caa866f
Implement call to Node.stop() when shutting down BitcoinSServerMain (#5124) 2023-06-29 15:57:46 -05:00
Scala Steward
aff86c7679
Update sbt to 1.9.1 (#5123) 2023-06-29 10:12:57 -05:00
Chris Stewart
f522ffec72
2023 06 28 connection fail logs (#5121)
* Rework logs for the case where our tcp connection fails. This makes the logs simpler

* Revert initialDelay

* Remove redundant setting
2023-06-29 09:01:53 -05:00
Chris Stewart
aa8da307b7
Remove batchAndParallelExecute test case (#5122)
* Remove batchAndParallelExecute test case

* scalafmt
2023-06-29 08:24:38 -05:00
Chris Stewart
b0cf3d8146
Fix race condition between NodeAppConfig.migrate() calls when bitcoin-s.tor.enabled=false (#5120)
* Fix race condition between NodeAppConfig.migrate() calls

* Use Future.traverse()
2023-06-28 14:13:26 -05:00
Chris Stewart
6befad2dd3
Remove offer(SendToPeer) from PeerManager queue (#5119)
* Remove offer(SendToPeer) from PeerManager queue

* scalafmt

* Remove comments
2023-06-28 07:58:34 -05:00
Chris Stewart
08a76fb040
Fix bug in DataMessageHandler.isFiltersSynced() (#5118)
* Fix bug in DataMessageHandler.isFiltersSynced()

* Try alternative implementation to fix bug

* Fix valid states for CompactFilterMessage, revert PeerFinder delay
2023-06-27 14:24:26 -05:00
Chris Stewart
dd77eb0566
Add jdk.management.agent module on dev builds (#5115) 2023-06-26 10:56:49 -05:00
Scala Steward
2023e3cfb7
Update bcprov-jdk18on to 1.75 (#5112) 2023-06-26 10:37:46 -05:00
Scala Steward
3df12aa929
Update sbt-scalajs, scalajs-compiler, ... to 1.13.2 (#5113) 2023-06-26 10:30:32 -05:00
Chris Stewart
1d82ed04a7
2023 06 18 implement tcp connection using akka streams (#5111)
* WIP: Try to move byte streaming/parsing of p2p messages out of P2PClient

* WIP2: Work on killing the actor, replace it with a steram

* Get basic ability to send/receive version message working

* Transition PeerMessageReceiverState to Initializing inside of PeerMessagesender.connect()

* Refactor things out of PeerMessageSender.connect(), add some flow logs

* Get NeutrinoNodeTest be able to sync passing

* Fix some bugs, create ConnectionGraph helper class

* Use killswitch rather than Source.maybe to disconnect peer

* WIP: Debug

* Switch halfClose to false on Tcp.outgoingConnection() to not keep lingering connections

* Delete P2PClientActorTest

* Delete all P2PClient stuff

* Attempt implementing reconnection logic in PeerMessageSender

* remove supervisor

* Empty commit to re-run CI

* Small cleanups

* Implement sendResponseTimeout()

* Restore logback-test.xml

* Add callback to log error message on reconnect

* Increase queueSize/maxConcurrentOffers size
2023-06-24 07:53:44 -05:00
Chris Stewart
5ae4993bed
Move PeerMessageReceiverState into PeerMessageReceiver (#5110)
* Move PeerMessageReceiverState into PeerMessageReceiver

* Delete PeerMessageReceiverTest
2023-06-19 15:04:35 -05:00
Chris Stewart
e0c9500770
Reduce p2pclient timeout to 10 seconds (#5109) 2023-06-17 08:00:29 -05:00
Scala Steward
2fe80bd52b
Update scala-collection-compat to 2.11.0 (#5107) 2023-06-16 13:34:54 -05:00
Chris Stewart
30b06b1131
Simplify DataMessageHandlerTest so we don't have duplicate callback execution (#5108)
* Simplify DataMessageHandlerTest so we don't have duplicate callback execution

* Turn logs OFF again
2023-06-16 13:34:45 -05:00
Scala Steward
2a966a6916
Update bcprov-jdk18on to 1.74 (#5106) 2023-06-16 10:26:45 -05:00
Chris Stewart
c3eed1e92b
2023 05 05 Make DataMessageHandler be accummulated in our akka stream (#5098)
* WIP

* WIP2

* Rebase with P2PClientCallbacks

* scalafmt

* Rework stream materialization in PeerManager to have DataMessageHandler encapsulated in the stream

* WIP: Fix compile

* Get everything compiling, ignore Uncached tests for now

* Increase queue size

* WIP: Make queue re-usable based on PeerManager.{start()/stop()}

* Get things compiling after rebase

* Try to handle case where we have SendToPeer in queue with peer that has been disconnected

* Empty commit to re-run CI

* Add sleep to let version/verack handshake complete in P2PClientActorTest

* Empty commit to re-run CI

* Reduce usage of bitcoind in P2PClientActorTest from 3 bitcoinds -> 2 bitcoinds

* Add error message to PeerFinder.stop() so we know what peer was not getting removed

* Cleanup error message

* Fix scalafmt, add state to log message

* Fix bug PeerMessageReceiverState.stopReconnect() which didn't send DisconnectedPeer() to queue

* Empty commit to re-run CI

* Empty commit to re-run CI

* Reduce log level of onP2PClientDisconnected

* Empty commit to re-run CI

* Small cleanup

* scalafmt

* Get new reference to ChainHandler in more places node.syncFromNewPeer() is called

* Fix rebase

* Commit to run on CI

* Empty commit to run CI

* Empty commit to run CI

* Empty commit to re-run CI

* Empty commit to re-run CI

* Try to reproduce with logs on CI

* Empty commit to re-run CI

* WIP

* Rework onP2PClientDisconnected to return new DataMessagehandlerState

* Save comment about bug

* Add a helper method switchSyncToPeer to take into account the previous DataMessagehandlerState if we need to start a new sync because of disconnection

* Empty commit to re-run CI

* Empty commit to re-run CI

* Cleanup

* Fix case where we weren't sending getheaders to new peer when old peer was disconnected when in state DoneSyncing

* Revert logback-test.xml

* remove comment

* Try using syncHelper() rather than getHeaderSyncHelper() to make sure we sync filters as well if needed

* Re-add log

* Fix bug where we weren't starting to sync filter headers

* Tighten dmhState type to SyncDataMessageHandler on syncFilters(), clean up uncessary code

* Empty commit to re-run CI

* Empty commit to re-run CI
2023-06-16 08:08:11 -05:00
Chris Stewart
458a80d854
Destroy wallet config when tearing down node (#5105) 2023-06-15 13:39:37 -05:00
Scala Steward
2363ec4d4e
Update logback-classic to 1.4.8 (#5103) 2023-06-14 15:50:24 -05:00
Chris Stewart
dae8c0fc9c
Don't execute onResponseTimeout() when disconnecting (#5104) 2023-06-14 15:49:20 -05:00
Chris Stewart
f4f45a1cad
Move {syncFromNewPeer(), syncHelper()} into PeerManager, remove reference to Node inside of PeerManager (#5102)
* Move {syncFromNewPeer(), syncHelper()} into PeerManager, remove reference to Node inside of PeerManager

* Empty commit to re-run CI
2023-06-13 14:18:22 -05:00
Chris Stewart
bed670fb6f
Remove chainApi parameter from NeutrinoNode (#5101)
* Remove chainApi parameter from NeutrinoNode

* Fix doc
2023-06-13 12:07:04 -05:00
Chris Stewart
e3f8eb2cc6
Add PeerMessageSenderApi.gossipGetHeadersMessage(), use it in Node.sync() (#5100)
* Add PeerMessageSenderApi.gossipGetHeadersMessage(), use it in Node.sync()

* Rework invalid headers test case to not need to reach into internals of akka stream

* Rework re-query headers test case to not need to reach into internals of akka stream

* Rework switch peers test case to not need to reach into internals of akka stream

* Use peerManager.offer() rather than reaching into DataMessageHandler to send messages to stream

* use gossipGetHeadersMessage() after getting done with IBD to query all peers instead of just one

* Empty commit to re-run CI

* Empty commit to re-run CI

* Empty commit to re-run CI
2023-06-09 15:34:29 -05:00
Chris Stewart
a5778948cb
Remove downloadBlocksBasedOnIBD() as it is redundant (#5099) 2023-06-08 16:10:04 -05:00
Chris Stewart
ae8c97a4d1
Rework Node.sync() to return Future[Peer] rather than Future[Option[Peer]] (#5096) 2023-06-08 10:30:26 -05:00
Chris Stewart
ebe98959ed
Upgrade to scalac 2.13.11 (#5097)
* Upgrade to scalac 2.13.11

* Fix compile issues with scalac 2.13.11
2023-06-08 10:27:39 -05:00
Chris Stewart
295be36d63
Get PeerMessageSenderApi using akka streams for outbound p2p messages (#5069)
* Get PeerMessageSenderApi using akka streams for outbound p2p messages

* Use offer method rather than accessing queue directly

* Fix flaky unit test

* Empty commit to re-run CI

* Move methods for requesting filterheaders/filters into PeerManager, now use akka stream for those outbound p2p message

* Move sendInventoryMessage to PeerManager

* Move sendGetHeadersMessage() methods to PeerManager

* WIP: move more methods to PeerMessageSenderApi

* WIP2

* initialize stream before calling PeerFinder.start() so oubound messages get processed

* Rebase

* Make queue buffer size dependent on maxConnectedPeers

* Change state to HeaderSync() if we are re-querying for block headers

* Empty commit to re-run CI

* Remove PeerMessageSender from handleDataPayload()

* Limit access to PeerManager.peerMessageSenders

* revert a few things

* Fix rebase issues

* Fix rebase

* Turn down logging

* Rebase

* remove guard that checks peer size before labeling as MisBehavingPeer

* Fix small bug where we needed to switch syncPeer and we weren't

* Empty commit to run CI

* Empty commit to re-run CI

* Fix test case where we weren't awaiting for Node.sync() to return Some(peer)

* Empty commit to re-run CI

* Fix another reference where we were calling Node.sync() too soon after Node.start()

* scalafmt

* Add another retryUntilSatisfied() on NeutrinoNode.sync()
2023-06-08 08:47:24 -05:00
Chris Stewart
abeaaa05de
Remove awaitPeerWithServices() (#5093)
* Remove awaitPeerWithServices()

* Empty commit to run CI

* Rework Node.sync() to return Future[Option[Peer]] rather than Future[Unit]. This returns the peer we are syncing with, if we could find one to sync with

* Turn logging OFF again

* Empty commit to re-run CI

* Use AsyncUtil.retryUntilSatisfied() when calling node.sync() after starting node to make sure we have a peer to sync from in a test case

* Await on re-started node not stale reference in NeutrinoNodeWithWalletTest

* Fix second reference

* Empty commit to re-run CI
2023-06-07 04:47:39 -05:00