Introduce a wtclient `Manager` which handles tower clients. It indexes
clients by the policy used. The policy field is thus removed from the
`Config` struct which configures the Manager and is instead added to a
new `towerClientCfg` which configures a specific client managed by the
manager. For now, only the `NewClient` method is added to the Manager.
It can be used to construct a new `TowerClient`. The Manager currently
does notthing with the clients added to it.
This commit also adds tests for the DB changes made in the previous
commit since we can now read the new field with the FetchChanInfos
method.
The commit following this one does the backfill migration.
The watchtower client test framework currently uses a mock version of
the tower client DB. This can lead to bugs if the mock DB works slightly
differently to the actual bbolt DB. So this commit ensures that we only
use the bbolt db for the tower client tests. We also increment the
`waitTime` used in the tests a bit to account for the slightly longer DB
read and write times. Doing this switch resulted in one bug being
caught: we were not removing sessions from the in-memory set on deletion
of the session and so that is fixed here too.
In this commit, the bugs demonstrated in prior commits are fixed. In the
case where an session has persisted a CommittedUpdate and the tower is
being removed, the session will now replay that update on to the main
task pipeline so that it can be backed up using a different session.
This commit does a few things:
- First, it gives the sessionQueue access to the TowerClient task
pipeline so that it can replay backup tasks onto the pipeline on Stop.
- Given that the above is done, the ForceQuit functionality of the
sessionQueue and TowerClient can be removed.
- The bug demonstrated in a prior commit is now fixed due to the above
changes.
In preparation for an upcoming commit where multiple threads will have
access to the TowerClient sessionQueueSet, we turn it into a thread safe
struct.
In this commit, we add an Identifier method to the blob.Type struct
which returns a unique identifier for a given blob type. This identifier
is then used for initialising the disk overflow queue of the given
client.
In this commit, a new generic DiskOverflowQueue implementation is added.
This allows a user to specify a maximum number of items that the queue
can hold in-memory. Any new items will then overflow to disk. The
producer and consumer of the queue items will interact with the queue
just like a normal in-memory queue.
Lock the `backupMu` when accessing `c.chanCommitHeights` in the `New`
function. It is not strictly necessary right now but good to add it so
that there is no accidental oversight if the `perUpdate` method is ever
extracted and reused in future.
Since the retrubution info of a backup task is now only constructed at
the time that the task is being bound to a session, the in-memory queue
only needs to carry the BackupID of the task.
Since the TowerClient now has a callback that it can use to retrieve the
retribution for a certain channel and commit height, let it use this
call back instead of requiring the info to be passed to it through
BackupState.
In this commit, a new BuildBreachRetribution callback is added to the
tower client's Config struct. The main LND server provides the client
with an implementation of the callback.
In this commit, a new `ExhaustedSessionFilter` function is added and
used as a PostEvalFilterFn used when loading sessions from the DB. It
allows us to not unnecessarily load exhausted sessions into memory for
areas of the code where they will not be needed.
In this commit, a PreEvaluateFilterFn option is added to the
wtdb.ClientSessionListCfg and it is used instead of a separate
ClientSessionFilterFn parameter. This neatens quiet a few function
signatures.
This commit adds a deleteSessionFromTower method which can be used to
dial the tower that we created a given session with and then sends that
tower the DeleteSession method.
In this commit, a thread-safe min-heap is implemented. It will carry
sessionCloseItems which carry a sessionID and a block height at which
the session should be closed.
In this commit, a new ClientSessionFilterFn parameter is added to the
DB's ListClientSession method which can be used to allow the caller to
specify a filter function for filtering sessions read from the DB.
Currently all filtering of sessions are done after the sessions have
been read from the DB, so adding this option should provide some
efficiency.
In this commit, a migration is done that takes all the AckedUpdates of
all sessions and stores them in the RangeIndex pattern instead and
deletes the session's old AckedUpdates bucket. All the logic in the code
is also updates in order to write and read from this new structure.
In this commit, the bug demonstrated in the previous commit is fixed.
The locking capabilities of the AddressIterator are used to lock
addresses if they are being used for session negotiation. So now, when a
request comes through to remove a tower address then a check is first
done to ensure that the address is not currently in use. If it is not,
then the request can go through.
This commit upgrades the wtclient package to make use of the new
`AddressIterator`. It does so by first creating new `Tower` and
`ClientSession` types. The new `Tower` type has an `AddressIterator`
instead of a list of addresses. The `ClientSession` type contains a
`Tower`.
In this commit, the new ListClientSession functional options and new
FetchSessionCommittedUpdates function are utilised in order to allow us
to completely remove the CommittedUpdates member from the ClientSession
struct.
In this commit, we start making use of the new ListClientSession
functional options added in the previous commit. We use the functional
options in order to calculate the max commit heights per channel on the
construction of the tower client. We also use the options to count the
total number of acked and committed updates. With this commit, we are
also able to completely remove the AckedUpdates member of the
ClientSession since it is no longer used anywhere in the code.
This commit adds functional options to the ListClientSessions call that
can be used to perform a variety of extra operations during the DB
query. These functional options are not yet used in this commit.
In this commit, the functions used to fetch candidate sessions and
towers on creation of the watchtower Client are changed to make use of
the more efficient lookup functions. Previously, all sessions were
listed from the DB and then these were used to collect the active
towers which in certain situations lead to some users getting the
"tower not found" error on start up. With this commit, we instead first
list all Towers in the DB and then we fetch the sessions for each of
those towers.
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
Currently the ForceQuit call is scheduled after trying to stop the
backup queue. In certain cases, the call to stop the queue never
finishes, which means the force quit is never scheduled. We rememdy by
scheduling this call before any other operations to ensure we can always
exit ungracefully if necessary.
This addresses a potential panic when a tower has one of its candidate
sessions chosen, but its only reachable address was removed by a
user-initiated RPC before the fact.
