Commit graph

913 commits

Author SHA1 Message Date
Olaoluwa Osuntokun
37fe29af88
channeldb: add new TestSettleIndexAmpPayments test case 2021-10-28 15:50:25 -07:00
Olaoluwa Osuntokun
7dc5efbebf
channeldb: add new test to exercise cancellation of multiple AMP setIDs 2021-10-28 15:50:22 -07:00
Olaoluwa Osuntokun
a4f8842831
channeldb: optimize updateInvoice for AMP by only reading relevant HTLCs
In this commit, we update the logic in `updateInvoice` to allow callers
to pass in either a hint, or the setID in the update callback. This
makes things more efficient for AMP invoices with thousands of recurring
payments, as we no longer need to read out _all_ the invoices each time
we go to update the state of a few HTLCs.
2021-10-28 15:50:15 -07:00
Olaoluwa Osuntokun
f969d81e1a
channeldb: for AMP, store set ID w/ invoice num in settle index
This change allows us to deliver notifications to a user of all the
settled recurring payments to the same payment_addr in the order that
they occurred.
2021-10-28 15:50:12 -07:00
Olaoluwa Osuntokun
3b28ad6d72
channeldb: for AMP don't update the main invoice state in updateInvoice
In this commit, we modify the way we handle state updates for AMP
invoices. With the current logic, once an invoice is settled, we'll
update the primary invoice state. This means that a user can't take the
same payment_addr, w/ a new set_id and pay the invoice again.

To remedy this, we'll move to instead _not_ updating the main invoice
state each time a new htlc Set (group of HTLCs according to setID) is
added. Instead, given that each HTLC stores an individual state, we'll
instead just use that directly from now on.

We also update TestSetIDIndex to account for new repeated settle AMP
logic.
2021-10-28 15:50:10 -07:00
Olaoluwa Osuntokun
c0f934e363
channeldb: don't cancel other HTLCs w/ diff setID once once is settled
With this change, we allow multiple users to concurrently pay the same
AMP invoice with distinct set IDs.
2021-10-28 15:50:07 -07:00
Olaoluwa Osuntokun
700bae16a3
channeldb: store AMP invoice htlc sets in new prefix key near invoice
In this commit, we modify the HTLC storage for AMP invoice only to be
stored within a new key prefix next to the main invoice data. We do this
as otherwise each time an AMP invoice is settled (enabled by the
following commits), we need to continually encode+decode the _entire_
set of invoices.

Instead, we store the AMP invoices within the main invoice bucket, using
the invoice number as a key prefix, with the final key being:
`invoiceNum || setID`
2021-10-28 15:50:04 -07:00
Olaoluwa Osuntokun
65cca8dd1c
channeldb: add new AMPInvoiceState field to store AMP sub-invoice metadata
In this commit, we add a new type `AMPInvoiceState` that's used to store
AMP sub-invoice meta data alongside the main invoice. This will be used
to allow changes to be made to an AMP invoices without reading out all
the HTLCs. In addition, callers can use this metadata to look up
information about the current sub-invoice state of AMP HTLCs.
2021-10-28 15:50:02 -07:00
Oliver Gugger
0a2ccfc52b
multi: use single read transaction for path finding
This commit partially reverts bf27d05a.
To avoid creating multiple database transactions during a single path
finding operation, we create an explicit transaction when the cached
graph is instantiated.
We cache the source node to avoid needing to look that up for every path
finding session.
The database transaction will be nil in case of the in-memory graph.
2021-10-22 09:32:49 +02:00
Oliver Gugger
1fef2970cf
channeldb: implement DB fallback for path finding
If the in-memory graph cache is disabled, we fall back to querying the
database.
2021-10-22 09:32:48 +02:00
Oliver Gugger
f216da32f3
lnd+channeldb: add graph cache option to channeldb
With this commit we forward the config option for disabling the channel
graph cache as a boolean to the channeldb. But we invert its meaning to
make the flag easier to understand.
2021-10-22 09:32:48 +02:00
Wilmer Paulino
f38c6d6662
channeldb: add new ChannelType bit for script enforced leased channels 2021-10-19 18:30:19 -07:00
Oliver Gugger
77bf6cfed3
channeldb: fix graph cache feature handling [skip ci]
Fixes #5830.
When a channel for a node is announced before the node itself is
announced on the network, it's possible that we have channels for a node
but no features defined yet. This was previously logged as a warning
which spammed the log unnecessarily.
2021-10-08 10:02:11 +02:00
Oliver Gugger
6240851f93
channeldb: optimize memory usage of initial cache fill
With this commit we use an optimized version of the node iteration that
causes fewer memory allocations by only loading the part of the graph
node that we actually need to know for the cache.
2021-09-29 17:00:06 +02:00
Oliver Gugger
a95a3728b5
lnd+channeldb: pre-allocate cache size
To avoid the channel map needing to be re-grown while we fill the cache
initially, we might as well pre-allocate it with a somewhat sane value
to decrease the number of grow events.
2021-09-29 17:00:05 +02:00
Oliver Gugger
1d1c42f9ba
multi: use minimal policy in cache 2021-09-29 17:00:04 +02:00
Oliver Gugger
15d3f62d5e
multi: use cache for source channels 2021-09-29 17:00:04 +02:00
Joost Jager
369c09be61
channeldb+routing: add in-memory graph
Adds an in-memory channel graph cache for faster pathfinding.

Original PoC by: Joost Jager
Co-Authored by: Oliver Gugger
2021-09-29 17:00:04 +02:00
Oliver Gugger
d6fa912188
multi: further decouple graph
To further separate the channel graph from the channel state, we
refactor the AddrsForNode method to use the graphs's public methods
instead of directly accessing any buckets. This makes sure that we can
have the channel state cached with just its buckets while not using a
kvdb level cache for the graph.
At the same time we refactor the graph's test to also be less dependent
upon the channel state DB.
2021-09-29 17:00:03 +02:00
Andras Banki-Horvath
11cf4216e4
multi: move all channelstate operations to ChannelStateDB 2021-09-29 17:00:03 +02:00
Oliver Gugger
c1f686f860
channeldb+funding: move opening channel state to DB
The funding manager doesn't need to know the details of the underlying
storage of the opening channel state, so we move the actual store and
retrieval into the channel database.
2021-09-29 17:00:02 +02:00
Andras Banki-Horvath
60cccf8409
multi: carve out LinkNodeDB from channeldb.DB for cleaner separation 2021-09-29 17:00:02 +02:00
Andras Banki-Horvath
292b8e1ce6
channeldb: fix dangerous type casting hack 2021-09-29 17:00:02 +02:00
Andras Banki-Horvath
639faeed6d
channeldb: use kvdb.Backend instead of channeldb.DB for the Graph 2021-09-29 17:00:01 +02:00
yyforyongyu
995b9a711f
trivial: typo fix 2021-09-23 13:24:15 +08:00
yyforyongyu
556b038398
channeldb: attach DB to channel in FetchHistoricalChannel 2021-09-23 13:22:34 +08:00
yyforyongyu
c74c1d0f51
channeldb/migration: remove old forwarding packages 2021-09-23 13:22:34 +08:00
yyforyongyu
d8c48fa3a5
channeldb: wipe all forwarding pkgs when close channel 2021-09-23 13:22:34 +08:00
yyforyongyu
3d50edf9f8
channeldb: add method to wipe all forwarding packages
This commit adds a method, Wipe, to remove all forwarding packages on
disk for a given channel.
2021-09-23 13:22:33 +08:00
yyforyongyu
0fff613f61
channeldb: add docs for fwd pkg bucket 2021-09-23 13:22:33 +08:00
Andras Banki-Horvath
3648f2fa4b
payments: prefetch payment on hot paths 2021-09-14 13:12:23 +02:00
Bjarne Magnussen
a5641c5351
channeldb: add ability to delete a single payment from its ID
Adds `DeletePayment` to the channeldb, which allows to delete a single payment. If only failed HTLCs for this payment should be deleted it can be specified by the bool `failedHtlcsOnly`.
2021-09-13 12:50:40 +02:00
Bjarne Magnussen
d5a87f8b5b
channeldb: refactor fetching failed HTLC IDs inside DeletePayments 2021-09-13 12:16:17 +02:00
Andras Banki-Horvath
204966db40
channeldb: migration to flatten the htlc attempts bucket 2021-09-07 10:46:49 +02:00
Andras Banki-Horvath
63d7f5fca7
channeldb: flatten the payment-htlcs-bucket 2021-09-07 10:46:46 +02:00
Joost Jager
3f775778c3
channeldb+routing: add tx parameter
Adds an optional tx parameter to ForAllOutgoingChannels and FetchChannel
so that data can be queried within the context of an existing database
transaction.
2021-08-24 13:43:24 +02:00
yyforyongyu
aa1561c60d
lnwire: use write buffer in Encode methods
This commit changes the WriteElement and WriteElements methods to take a
write buffer instead of io.Writer. The corresponding Encode methods are
changed to use the write buffer.
2021-08-10 05:36:08 +08:00
Oliver Gugger
fe931d179f
etcd+channeldb: fix linter issues, rename receiver
With this commit we address some issues the linter picked up after
touching older code.
2021-08-04 14:55:56 +02:00
Oliver Gugger
482f76a0f4
mod+kvdb+channeldb: use btcwallet new DB interface
Depends on btcsuite/btcwallet#757.
Pulls in the updated version of btcwallet and walletdb that have the DB
interface enhanced by their own View() and Update() methods with the
reset callback/closure supported out of the box. That way the global
package-level View() and Update() functions now become pure redirects.
2021-08-04 14:55:55 +02:00
Andras Banki-Horvath
f33b5a4057
payments: allocate payment sequences in blocks
The bucket sequence we use as payment sequence makes the DB update in
InitPayment conflict and therefore queue up when many concurrent
payments happen. This change allocates payment sequences in blocks
of 1000 to avoid these conflicts.
2021-07-21 16:36:10 +02:00
Olaoluwa Osuntokun
079ab1c2bf
channeldb: fix typo in PaymentControl.Fail 2021-07-07 15:31:27 -07:00
carla
64333c0303
channeldb: cache htlc attempt session key 2021-05-19 09:08:18 +02:00
carla
8071ebb16a
channeldb: read raw htlc attempt session key 2021-05-19 09:03:52 +02:00
carla
eb068bf666
multi: unexport session key and add constructor for htlc attempt info 2021-05-19 09:03:46 +02:00
Olaoluwa Osuntokun
7ab5906093
Merge pull request #5245 from bhandras/kvdb_module
kvdb: make kvdb a top level submodule to allow dependency in other projects
2021-05-13 15:59:20 -07:00
Conner Fromknecht
8402e346f5
channeldb/invoice: fail extra HTLC sets immeidately 2021-05-10 16:55:18 -07:00
Conner Fromknecht
d93c3298b7
channeldb+invoice: add state filter to HTLCSet 2021-05-10 16:55:17 -07:00
Olaoluwa Osuntokun
1868c7dec1
Merge pull request #5231 from Crypt-iQ/unsigned_ack_patch_04212021
lnwallet: use tail not tip in getUnsignedAckedUpdates
2021-05-07 12:46:48 -07:00
Andras Banki-Horvath
14c851c8fc
kvdb: move channeldb/kvdb to top level 2021-05-07 14:18:56 +02:00
Olaoluwa Osuntokun
4685341dcb
Merge pull request #5227 from Roasbeef/spent-channs-zombie-index
routing: add chans rejected due to failed chain validation to zombie index
2021-05-05 18:32:13 -07:00