* Fix bug in DataMessageHandler.isFiltersSynced()
* Try alternative implementation to fix bug
* Fix valid states for CompactFilterMessage, revert PeerFinder delay
* WIP: Try to move byte streaming/parsing of p2p messages out of P2PClient
* WIP2: Work on killing the actor, replace it with a steram
* Get basic ability to send/receive version message working
* Transition PeerMessageReceiverState to Initializing inside of PeerMessagesender.connect()
* Refactor things out of PeerMessageSender.connect(), add some flow logs
* Get NeutrinoNodeTest be able to sync passing
* Fix some bugs, create ConnectionGraph helper class
* Use killswitch rather than Source.maybe to disconnect peer
* WIP: Debug
* Switch halfClose to false on Tcp.outgoingConnection() to not keep lingering connections
* Delete P2PClientActorTest
* Delete all P2PClient stuff
* Attempt implementing reconnection logic in PeerMessageSender
* remove supervisor
* Empty commit to re-run CI
* Small cleanups
* Implement sendResponseTimeout()
* Restore logback-test.xml
* Add callback to log error message on reconnect
* Increase queueSize/maxConcurrentOffers size
* WIP
* WIP2
* Rebase with P2PClientCallbacks
* scalafmt
* Rework stream materialization in PeerManager to have DataMessageHandler encapsulated in the stream
* WIP: Fix compile
* Get everything compiling, ignore Uncached tests for now
* Increase queue size
* WIP: Make queue re-usable based on PeerManager.{start()/stop()}
* Get things compiling after rebase
* Try to handle case where we have SendToPeer in queue with peer that has been disconnected
* Empty commit to re-run CI
* Add sleep to let version/verack handshake complete in P2PClientActorTest
* Empty commit to re-run CI
* Reduce usage of bitcoind in P2PClientActorTest from 3 bitcoinds -> 2 bitcoinds
* Add error message to PeerFinder.stop() so we know what peer was not getting removed
* Cleanup error message
* Fix scalafmt, add state to log message
* Fix bug PeerMessageReceiverState.stopReconnect() which didn't send DisconnectedPeer() to queue
* Empty commit to re-run CI
* Empty commit to re-run CI
* Reduce log level of onP2PClientDisconnected
* Empty commit to re-run CI
* Small cleanup
* scalafmt
* Get new reference to ChainHandler in more places node.syncFromNewPeer() is called
* Fix rebase
* Commit to run on CI
* Empty commit to run CI
* Empty commit to run CI
* Empty commit to re-run CI
* Empty commit to re-run CI
* Try to reproduce with logs on CI
* Empty commit to re-run CI
* WIP
* Rework onP2PClientDisconnected to return new DataMessagehandlerState
* Save comment about bug
* Add a helper method switchSyncToPeer to take into account the previous DataMessagehandlerState if we need to start a new sync because of disconnection
* Empty commit to re-run CI
* Empty commit to re-run CI
* Cleanup
* Fix case where we weren't sending getheaders to new peer when old peer was disconnected when in state DoneSyncing
* Revert logback-test.xml
* remove comment
* Try using syncHelper() rather than getHeaderSyncHelper() to make sure we sync filters as well if needed
* Re-add log
* Fix bug where we weren't starting to sync filter headers
* Tighten dmhState type to SyncDataMessageHandler on syncFilters(), clean up uncessary code
* Empty commit to re-run CI
* Empty commit to re-run CI
* Add PeerMessageSenderApi.gossipGetHeadersMessage(), use it in Node.sync()
* Rework invalid headers test case to not need to reach into internals of akka stream
* Rework re-query headers test case to not need to reach into internals of akka stream
* Rework switch peers test case to not need to reach into internals of akka stream
* Use peerManager.offer() rather than reaching into DataMessageHandler to send messages to stream
* use gossipGetHeadersMessage() after getting done with IBD to query all peers instead of just one
* Empty commit to re-run CI
* Empty commit to re-run CI
* Empty commit to re-run CI
* Get PeerMessageSenderApi using akka streams for outbound p2p messages
* Use offer method rather than accessing queue directly
* Fix flaky unit test
* Empty commit to re-run CI
* Move methods for requesting filterheaders/filters into PeerManager, now use akka stream for those outbound p2p message
* Move sendInventoryMessage to PeerManager
* Move sendGetHeadersMessage() methods to PeerManager
* WIP: move more methods to PeerMessageSenderApi
* WIP2
* initialize stream before calling PeerFinder.start() so oubound messages get processed
* Rebase
* Make queue buffer size dependent on maxConnectedPeers
* Change state to HeaderSync() if we are re-querying for block headers
* Empty commit to re-run CI
* Remove PeerMessageSender from handleDataPayload()
* Limit access to PeerManager.peerMessageSenders
* revert a few things
* Fix rebase issues
* Fix rebase
* Turn down logging
* Rebase
* remove guard that checks peer size before labeling as MisBehavingPeer
* Fix small bug where we needed to switch syncPeer and we weren't
* Empty commit to run CI
* Empty commit to re-run CI
* Fix test case where we weren't awaiting for Node.sync() to return Some(peer)
* Empty commit to re-run CI
* Fix another reference where we were calling Node.sync() too soon after Node.start()
* scalafmt
* Add another retryUntilSatisfied() on NeutrinoNode.sync()
* Remove awaitPeerWithServices()
* Empty commit to run CI
* Rework Node.sync() to return Future[Option[Peer]] rather than Future[Unit]. This returns the peer we are syncing with, if we could find one to sync with
* Turn logging OFF again
* Empty commit to re-run CI
* Use AsyncUtil.retryUntilSatisfied() when calling node.sync() after starting node to make sure we have a peer to sync from in a test case
* Await on re-started node not stale reference in NeutrinoNodeWithWalletTest
* Fix second reference
* Empty commit to re-run CI
* Move P2PClientCallbacks.onStop() into disconnection logic rather than Actor.postStop()
* Rename onStop -> onDisconnect
* Add reconnect flag to P2PCallbacks.onDisconnect so we don't attempt to reconnect when not necessary
* Rename flag to forceReconnect, check getPeerConnectionCount in PeerManager.onP2PClientDisconnect to see if we have 0 connections. If we do reconnect
* Add PeerManager.isStarted flag, guard reconnection logic with this flag in onP2PClientDisconnected()
* Clear PeerFinder._peerData when PeerFinder.stop() is called
* WIP: Move disconnection logic into stream
* Rework ordering of PeerManager.stop() to shutdown queue after peers are removed
* Empty commit to re-run CI
* Await for getConnectionCount async in test case
* Try increasing queue size
* Bump queue size to 16
* Put initialization, initialization timeout logic in queue rather than callbacks
* Make messages that are sent to the queue rather than callbacks for various control messages
* Empty commit to re-run CI
* Empty commit to re-run CI
* Empty commit to re-run CI
* Remove P2PCallbacks all together
* Re-add PeerMessageReceiverTest
* WIP
* Reset PeerManager.dataMessageHandler to None on PeerManager.stop()
* WIP2
* Always set dataMessageHandler.peerData we process a new message in the stream
* remove PeerManager from PeerDatta
* Make PeerFinder mutable inside of PeerManager, pass the queue into PeerFinder to decouple PeerManager and PeerFinder
* Don't verify actor system shutdown for now
* Drain data message stream before PeerManager.stop() is complete
* Try to fix race condition where peers.length gets mutated as peers get disconnected
* Adjust logic to check if we have a _new_ peer, not a specific peer count
* rework PeerManager.stop() ordering
* Rework PeerFinder.stop() to use Future.traverse()
* Make ControlMessageHandler take PeerManager rather than Node as a param
* refactor PeerData to not take a reference to Node
* Move ControlMessageHandler out of {Node,NeutrinoNode}
* Fix ReConnectionTest
* Cleanup
* Revert logback-test.xml
* Fix connectioncount test case
* Get P2PClientTest passing consistently
* Empty commit to re-run CI
* Make PeerManager.peerDataMap private
* Remove PeerMessageSender as param to DataMessageHandler.addToStream()
* Remove PeerMessageSender parameter from DataMessageWrapper
* Add PeerMessageSenderApi
* Try adding a supervision strategy to the stream
* Empty commit to re-run CI
* Adjust log level down to try and get a better idea of whats happening on IC
* Add commandName to exception when we cannot find peerMessageSender in stream
* Try decreasing queue size to reduce async processing between being stream processing and actor receiving external p2p message
* Empty commit to re-run CI
* Increase max concurrent offers to nodeAppConfig.maxConnectedPeers
* Revert logging
* WIP: Add new states for FilterHeaderSync, FilterSync, DoneSyncing
* Get DataMessageHandler compiling
* Get src compiling
* Get tests compiling
* Remove syncPeer from DoneSyncing
* WIP: Get first NeutrinoNodeTest working
* Get first NeutrinoNodeTest working for basic sync
* Fix small bug
* Fix bug where we transition to HeaderSync when we receive HeadersMessage with 0 headers. Also reduce conflicting states in DataMessageHandler.getHeaders()
* Add caveats for empty HeadersMessage
* WIP: Get NeutrinoNodeTest syncing compact filters during IBD passing consistently
* Remove generate block delay
* Empty commit to re-run CI
* Fix bug in onP2PClientStopped()
* Fix handling of DoneSyncing state when we receive headers messages
* Remove impossible case
* Remove node parameter to PeerMessageReceiver, pass the ControlMessageHandler and DataMessageHandler as parameters
* Remove node reference in DataMessageHandler, just pass PeerManager as parameter
* Fix pattern match to be exhaustive
* Move fetchCompactFilterHeaders, sendFirstGetCompactFilterHeadersCommand out of DataMessagehandler to decouple DataMessageHandler, PeerManager
* scalafmt
* WIP: Refactor PeerMessageReceiver methods into PeerMessageReceiverState
* Remove state helper methods from PeerMessageReceiver
* WIP: Remove PeerMessageReceiverState from PeerMessageReceiver
* Get things compiling and tests passing
* Remove currentPeerMessageHandlerReceiver
* Refactor PeerMessageReceiverTest to not use bitcoind
* Fix bug where compact filters weren't being processed in order of block height during IBD
* Use sorted compact filter messages in chainApi.processFilters()
* Add logic to fetch block headers after compact filters are synced in IBD to avoid a stale tip
* Restore logging, remove println
* Add comment linking to issue
* Cleanup logging
* Only send getheaders message after filter sync if we are in IBD
* WIP
* Modify DataMessageHandler to use Set for filter batch, fix test case
* revert log level
* Remove duplicate filters test
* Re-add test to make sure we don't throw an exception when processing a filter we've seen before
* Empty commit to re-run CI
* Empty commit to re-run CI
* Empty commit to re-run CI
* Fix NeutrinoNodeTest
* Empty commit to re-run CI
* Empty commit to re-run CI
* Segregate P2PClientTest and P2PClientActorTest to make it easier to test the actor
* Fix cleanup code
* Clean up tables for postgres test case
* Fix BitcoinSAppConfigBitcoinFixtureStarted.afterAll()
* Empty commit
* Add forceNamedWallet parameter to for postgres test cases where we need multiple wallets. This prevents them from writing to the same postgres schema by making unique wallet names
* Cleanup:
* Empty commit
* Make PeerManager.finder private
* Make PeerManager.supervisor private
* Write unit test to see if default peer is added to PeerManager
* Use withNeutrinoNodeUnstarted fixture
* Get unit tests passing
* Add println to try and debug CI
* Add more println
* more println
* Adjust nodeTest / parallelExecution = false
* Try to make error logged to figure out why failure is happening on CI
* Adjust log level to ERROR
* Add catch to try and log failure messages
* Add log for sending messages to peers
* Try to increase threadpool size to see if we are deadlocking
* Add more logs
* Add another log to detect where failure is
* Try using Actor.tell() with explicit ActorRef.noSender
* Very detailed logging
* Remove duplicate method from rebase
* Revert things
* Revert build.sbt
* Empty commit
* Bump timeout
* Empty commit to run CI
* Revert more files
* Move handle network received message into awaitNetworkRequest() context
* Fix bug where I wasn't wrapping in NetworkMessageReceived
* revert logging
* remove more uncessary logs
* Empty commit
* Pull things over from 4950 that shouldn't cause issues to reduce size of 4950
* Reduce timeout for processing a batch of messages from 1000 seconds -> 60 seconds
* scalafmt
* Empty commit
* WIP
* WIP2
* Get all DataMessageHandlerTest passing
* Get NeutrinoNodeTest passing
* Fix NeutrinoNodeWithUncachedBitcoindTest unit test
* Use chainApi to detect if we are in IBD
* Empty commit
* Revert logging levels
* add header sync validation
* fix docs, minor fixes
* Refactor to us InvalidBlockHeader ChainException, also refactor recovery to private helper method
* changes from comments
Co-authored-by: Chris Stewart <stewart.chris1234@gmail.com>
* Implement isinitialblockdownload flag in databaes
* Change to IsInitialBlockDownload to be less confusing
* Fix missing refactors, add ChainApi.isIBD()
* Implement logic for ChainApi.isIBD(), add configuration flag to toggle whether to emit websocket events while IBD is ongoing or not, add isinitialblockdownload field to getinfo endpoint
* Remove confusing ChainApi.isIBD() that didn't accurately account for the case where IBD wasn't set in the database
* Fix bug around emitting the websocket events
* Add some documentation
* removed logger parameter from where it's used with callbacks
* removed logger from onTXRecived... and onBlockRecived...
* removed unused imports
* removed logger from walletCallbacks
* Replace BoundedSourceQueueWithComplete with SourceQueueWithComplete so we can use the returned Future
* Recover the expected exceptions in CallBackUtilTest
* add P2PClientSupervisor
* changes from comments: made P2PClient Future
* empty commit to see if mac failures are consistent on ci
* changes from comments
* changes from comments
* Move pollBitcoind out of startBitcoindBlockPolling
* Rework startBitcoindBlockPolling to not return Future[Cancellable]
* Stop NodeCallbacksStreamManager inside of NodeAppConfig.stop()
* Make BitcoindSyncState which encapsulates syncing with bitcoind and polling of mempool
* Fix bug where processingBitcoindBlocks cannot be top level val as that method may be used by different parts of the codebase
* Rename isPolling -> processingBitcoindBlocks
* Cleanup more resources in appServerTest/test
* Complete polling stream regardless if we synced blocks are not to complete the Future returned by pollBitcoind
* Empty commit to run CI
* Move MockChainQueryApi/NodeApi out of BaseWalletTest
* refactor BitcoinSServerMain.start() to return WalletHolder
* Add walletConfig to WalletWithBitcoind
* Move findOutputsBeingSpent into WalletApi
* Add WalletApi.clearAllAddresses(), HDWalletApi.{findAccount, getnewaddress(account)}
* Add HDWalletApi.fundRawTransaction() with an account as parameter
* Add WalletApi.findByScriptPubKey()
* Fix lots of tests in WalletIntegrationTest
* Create WalletApi.processOurTransaction()
* Get things compiling
* Fix tag integration test
* Refactor AnyDLCHDWalletApi -> DLCNeutrinoHDWalletApi
* Fix docs
* Get postgres tests passing locally
* Move initBalance map before calling callback in CallBackUtilTest
* Get compile working, rename to destroyOnlyWalletWithBitcoindCached
* Fix docs
* Fix missing destroyWalletAppConfig
* Fix scalafmt
* Fix bug in dlcWalletTest where wallet db thread pools weren't being shutdown after unit test completes
* Empty commit
* Implement akka stream proxy for nodecallbacks
Add killswitch to createBitcoindNodeCallbacksForWallet
Add unit test for killswitch in createBitcoindNodeCallbacksForWallet
Add delays to make sure callbacks are executed
Fix rebase
Move killswitch out of methods into class level val
* Bump timeout
* Attempt to implement NodeCallbacks proxy with akka streams
* Refactor CallbackUtil to use NodeCallbackStreamManager
* Rebase & remove killswitch
* Implement NodeCallbackStreamManager.+ method
* Add `sync` JSON field to the `getinfo` endpoint output
* improve test coverage
* update docs
* rename sync flag
* WebSocket notifications
* fix unit tests
* fix unit tests
* increase test timout
* add support to find and switch peers
* fix compile on 2.12
* allow empty config peers in regtest
* fix test
* minor fixes
changes timeouts, fix issue with ipv6 dns seeds, initDisconnect when disconnected bug, dbPeers order fix
* fix: not removing peers on initialization timeout
* fix: query again when previous failed
* fix: wrong condition for deferred peers
* restore log levels
* clean up
* add PeerStack to allow trying peers based on priority values
* fix migrations
* changes from comments
* use StartStopAsync
* changes from comments
* fix switch if peer down test
* changes from comments
