Commit Graph

937 Commits

Author SHA1 Message Date
eugene
4373faa818
channeldb: change to ErrTypeForDecodingErr for amp state
Was instead returning ErrTypeForEncodingErr.
2022-03-23 12:05:11 -04:00
Oliver Gugger
7dfe4018ce
multi: use btcd's btcec/v2 and btcutil modules
This commit was previously split into the following parts to ease
review:
 - 2d746f68: replace imports
 - 4008f0fd: use ecdsa.Signature
 - 849e33d1: remove btcec.S256()
 - b8f6ebbd: use v2 library correctly
 - fa80bca9: bump go modules
2022-03-09 19:02:37 +01:00
Oliver Gugger
13f187046b
multi: fix issues reported by whitespace linter 2022-02-15 16:26:08 +01:00
Oliver Gugger
895a2e497b
multi: formatting and comment fixes 2022-02-10 11:02:02 +01:00
Oliver Gugger
ffee7d1bcf
multi: fix newly detected linter issues 2022-02-10 11:02:02 +01:00
Oliver Gugger
dfdc2bff8b
multi: run gosimports 2022-02-10 11:02:01 +01:00
Rafe
62dc1b5323 multi: Update Licenses to 2022 [skip ci] 2022-02-09 00:25:20 +02:00
Dimitris Apostolou
530a2059e5
multi: Fix typos [skip ci] 2022-01-24 12:19:02 +02:00
Olaoluwa Osuntokun
d67e6d5414
Merge pull request #6111 from bottlepay/cache-loading
kvdb+channeldb: speed up graph cache
2022-01-20 17:35:04 -08:00
Olaoluwa Osuntokun
91f32ad270
contractcourt: catch error when no historical bucket exists
For older nodes, this bucket was never created, so we'll get an error if
we try and query it. In this commit, we catch this error like we do when
a given channel doesn't have the information (but the bucket actually
exists).

Fixes #6155
2022-01-18 09:11:13 -08:00
Joost Jager
352008a0c2
channeldb: speed up graph cache loading
Use the optimized ForEachChannel method to reduce the graph cache
loading time.
2022-01-14 14:57:29 +01:00
Joost Jager
2e2229a2e7
channeldb: reallocate node in ForEachNodeCacheable
Allows cacheableNode to be used outside of the callback. This is a
preparation for optimization of the graph cache population.
2022-01-14 14:57:27 +01:00
Joost Jager
cb4703638f
channeldb: optimize ForEachChannel
In this commit, we modify the implementation of ForEachChannel to
utilize the new kvdb method ForAll. This greatly reduces the number of
round-trips to the database needed to iterate over all channels
in the graph.
2022-01-14 14:57:25 +01:00
Joost Jager
54324d554a
channeldb: extract deserializeChanEdgePolicyRaw
Allows for pure deserialization without depending on a database connection.
2022-01-12 19:11:56 +01:00
Joost Jager
aff9685a61
channeldb: extract AddNodeFeatures
Preparation for efficient graph cache loading where we don't iterate
over the channels of each node in the network individually.
2022-01-12 19:11:54 +01:00
naveen
87a0e52464
multi: Removed deprecated interfacer linter
Removed the deprecated interfacer linter from being called for linting.

Also removed the `nolint:interfacer` within the code.

https://github.com/lightningnetwork/lnd/issues/5741
2022-01-11 14:08:18 +01:00
naveen
545e84f4ce
multi: formatting changes 2022-01-11 14:08:17 +01:00
Oliver Gugger
b147d589c1
channeldb: add NoMigration mode
In case the channeldb package is used as a library in external tools, it
can be useful to allow read-only access to a DB. This allows such a
tool to access a DB even if not all migrations were executed, which can
be useful for recovery purposes.
To make it possible to even start the DB with a read-only backend, we
need to disable the automatic migration step.
2021-12-13 12:31:49 +01:00
ueno
86425091e9 channeldb: check fetchOpenChannel() error 2021-11-23 17:31:57 +09:00
Andras Banki-Horvath
3df216f6c3
channeldb: avoid locking the graph cache while iterating channels
It may happen that we do pathfinding while also attempt to change the
graph. In this case changing the database, channel state, graph while
reading from the same sources may create deadlocks. To resolve this we
change locking policy in the graph cache when pathfinding.
2021-11-08 12:40:14 +01:00
Olaoluwa Osuntokun
fae470293f
channeldb: add new ForEachNode method using the channel graph cache
This commit, adds a new ForEachNode method to the channel graph cache
that assumes the contents won't be modified. This is generally useful,
and will be used in a later commit to optimize some heavy RPC calls.
2021-11-04 15:30:25 -07:00
Olaoluwa Osuntokun
fc2a29f717
channeldb: add failing test to demonstrate panic w/o graph cache
This test panics as is, since we assume the inbound edge is always
there.
2021-11-02 20:19:01 -07:00
Andras Banki-Horvath
ea5d43a2a8
channeldb: fix crash when inbound policy is unset 2021-11-02 20:18:58 -07:00
Joost Jager
d997bbf6b3
channeldb/test: test with postgres 2021-10-29 10:54:44 +02:00
Olaoluwa Osuntokun
37fe29af88
channeldb: add new TestSettleIndexAmpPayments test case 2021-10-28 15:50:25 -07:00
Olaoluwa Osuntokun
7dc5efbebf
channeldb: add new test to exercise cancellation of multiple AMP setIDs 2021-10-28 15:50:22 -07:00
Olaoluwa Osuntokun
a4f8842831
channeldb: optimize updateInvoice for AMP by only reading relevant HTLCs
In this commit, we update the logic in `updateInvoice` to allow callers
to pass in either a hint, or the setID in the update callback. This
makes things more efficient for AMP invoices with thousands of recurring
payments, as we no longer need to read out _all_ the invoices each time
we go to update the state of a few HTLCs.
2021-10-28 15:50:15 -07:00
Olaoluwa Osuntokun
f969d81e1a
channeldb: for AMP, store set ID w/ invoice num in settle index
This change allows us to deliver notifications to a user of all the
settled recurring payments to the same payment_addr in the order that
they occurred.
2021-10-28 15:50:12 -07:00
Olaoluwa Osuntokun
3b28ad6d72
channeldb: for AMP don't update the main invoice state in updateInvoice
In this commit, we modify the way we handle state updates for AMP
invoices. With the current logic, once an invoice is settled, we'll
update the primary invoice state. This means that a user can't take the
same payment_addr, w/ a new set_id and pay the invoice again.

To remedy this, we'll move to instead _not_ updating the main invoice
state each time a new htlc Set (group of HTLCs according to setID) is
added. Instead, given that each HTLC stores an individual state, we'll
instead just use that directly from now on.

We also update TestSetIDIndex to account for new repeated settle AMP
logic.
2021-10-28 15:50:10 -07:00
Olaoluwa Osuntokun
c0f934e363
channeldb: don't cancel other HTLCs w/ diff setID once once is settled
With this change, we allow multiple users to concurrently pay the same
AMP invoice with distinct set IDs.
2021-10-28 15:50:07 -07:00
Olaoluwa Osuntokun
700bae16a3
channeldb: store AMP invoice htlc sets in new prefix key near invoice
In this commit, we modify the HTLC storage for AMP invoice only to be
stored within a new key prefix next to the main invoice data. We do this
as otherwise each time an AMP invoice is settled (enabled by the
following commits), we need to continually encode+decode the _entire_
set of invoices.

Instead, we store the AMP invoices within the main invoice bucket, using
the invoice number as a key prefix, with the final key being:
`invoiceNum || setID`
2021-10-28 15:50:04 -07:00
Olaoluwa Osuntokun
65cca8dd1c
channeldb: add new AMPInvoiceState field to store AMP sub-invoice metadata
In this commit, we add a new type `AMPInvoiceState` that's used to store
AMP sub-invoice meta data alongside the main invoice. This will be used
to allow changes to be made to an AMP invoices without reading out all
the HTLCs. In addition, callers can use this metadata to look up
information about the current sub-invoice state of AMP HTLCs.
2021-10-28 15:50:02 -07:00
Oliver Gugger
0a2ccfc52b
multi: use single read transaction for path finding
This commit partially reverts bf27d05a.
To avoid creating multiple database transactions during a single path
finding operation, we create an explicit transaction when the cached
graph is instantiated.
We cache the source node to avoid needing to look that up for every path
finding session.
The database transaction will be nil in case of the in-memory graph.
2021-10-22 09:32:49 +02:00
Oliver Gugger
1fef2970cf
channeldb: implement DB fallback for path finding
If the in-memory graph cache is disabled, we fall back to querying the
database.
2021-10-22 09:32:48 +02:00
Oliver Gugger
f216da32f3
lnd+channeldb: add graph cache option to channeldb
With this commit we forward the config option for disabling the channel
graph cache as a boolean to the channeldb. But we invert its meaning to
make the flag easier to understand.
2021-10-22 09:32:48 +02:00
Wilmer Paulino
f38c6d6662
channeldb: add new ChannelType bit for script enforced leased channels 2021-10-19 18:30:19 -07:00
Oliver Gugger
77bf6cfed3
channeldb: fix graph cache feature handling [skip ci]
Fixes #5830.
When a channel for a node is announced before the node itself is
announced on the network, it's possible that we have channels for a node
but no features defined yet. This was previously logged as a warning
which spammed the log unnecessarily.
2021-10-08 10:02:11 +02:00
Oliver Gugger
6240851f93
channeldb: optimize memory usage of initial cache fill
With this commit we use an optimized version of the node iteration that
causes fewer memory allocations by only loading the part of the graph
node that we actually need to know for the cache.
2021-09-29 17:00:06 +02:00
Oliver Gugger
a95a3728b5
lnd+channeldb: pre-allocate cache size
To avoid the channel map needing to be re-grown while we fill the cache
initially, we might as well pre-allocate it with a somewhat sane value
to decrease the number of grow events.
2021-09-29 17:00:05 +02:00
Oliver Gugger
1d1c42f9ba
multi: use minimal policy in cache 2021-09-29 17:00:04 +02:00
Oliver Gugger
15d3f62d5e
multi: use cache for source channels 2021-09-29 17:00:04 +02:00
Joost Jager
369c09be61
channeldb+routing: add in-memory graph
Adds an in-memory channel graph cache for faster pathfinding.

Original PoC by: Joost Jager
Co-Authored by: Oliver Gugger
2021-09-29 17:00:04 +02:00
Oliver Gugger
d6fa912188
multi: further decouple graph
To further separate the channel graph from the channel state, we
refactor the AddrsForNode method to use the graphs's public methods
instead of directly accessing any buckets. This makes sure that we can
have the channel state cached with just its buckets while not using a
kvdb level cache for the graph.
At the same time we refactor the graph's test to also be less dependent
upon the channel state DB.
2021-09-29 17:00:03 +02:00
Andras Banki-Horvath
11cf4216e4
multi: move all channelstate operations to ChannelStateDB 2021-09-29 17:00:03 +02:00
Oliver Gugger
c1f686f860
channeldb+funding: move opening channel state to DB
The funding manager doesn't need to know the details of the underlying
storage of the opening channel state, so we move the actual store and
retrieval into the channel database.
2021-09-29 17:00:02 +02:00
Andras Banki-Horvath
60cccf8409
multi: carve out LinkNodeDB from channeldb.DB for cleaner separation 2021-09-29 17:00:02 +02:00
Andras Banki-Horvath
292b8e1ce6
channeldb: fix dangerous type casting hack 2021-09-29 17:00:02 +02:00
Andras Banki-Horvath
639faeed6d
channeldb: use kvdb.Backend instead of channeldb.DB for the Graph 2021-09-29 17:00:01 +02:00
yyforyongyu
995b9a711f
trivial: typo fix 2021-09-23 13:24:15 +08:00
yyforyongyu
556b038398
channeldb: attach DB to channel in FetchHistoricalChannel 2021-09-23 13:22:34 +08:00