In this commit, we update an existing migration which at the time of
writing has not been included in a release. We update it so that it
converts the format used for MissionControl result encoding to use pure
TLV instead. The 3 structs that have been updated are: `mcHop`,
`mcRoute` and `paymentResult`.
So that `missionControlStore` can be unaware of the backing DB structure
it is writing to. In an upcoming commit when we change mission control
to write to namespaced buckets instead, we then only need to update the
`namespacedDB` implementation.
Add a new mcRoute type that houses the data about a route that MC
actually uses. Then add a migration (channeldb/migration32) that
migrates the existing store from its current serialisation to the new,
more minimal serialisation.
This removes duplication of in-memory data during the periodic flushing
stage of the mission control store.
The existing code entirely duplicates the in-memory cache of the store,
which is very wasteful when only a few additional results are being
rotated into the store.
This has a significant performance penalty specially for wallets that
remain online for a long time with a low volume of payments. The worst
case scenario are wallets that see at most 1 new payment a second, where
the entire in-memory cache is recreated every second.
This commit improves the situation by determining what will be the
actual changes that need to be committed before initiating the db
transaction and only keeping track of these to update the in-memory
cache if the db tx is successful.
This modifies the mission control store to avoid running the one second
ticker for flushing data when there is no work to be done.
This improves performance of a quiscent LN node by avoiding a one second
interval busy loop that does nothing when there are no payments flowing
through the node.
This modifies the mission control store to avoid doing any work when no
new payment result entries are in the queue to be processed.
The mission control store maintains keeps the latest N (in production:
1000) entries in its DB, evicting older entries when new ones are added.
Currently, its implementation is somewhat less performant than it could
be.
This commit adds an early return to the storeResults function to avoid
doing any DB or memory operations when its outstanding queue is empty,
improving the performance during quiescent periods of the LN node's
execution.
Since bbolt returns references to internally stored data when storing
locally it's best to copy the byte slices returned or alternatively
convert them to string (which also makes a copy) to avoid crashes casued
by memory corruption.
This commit changes missioncontrol's store update from per payment to
every second. Updating the missioncontrol store on every payment caused
gradual slowdown when using etcd.
We also completely eliminate the use of the cursor, further reducing
the performance bottleneck.
Similarly as with kvdb.View this commits adds a reset closure to the
kvdb.Update call in order to be able to reset external state if the
underlying db backend needs to retry the transaction.
This commit adds a reset() closure to the kvdb.View function which will
be called before each retry (including the first) of the view
transaction. The reset() closure can be used to reset external state
(eg slices or maps) where the view closure puts intermediate results.
