This commit fixes a potential blocking when notifying invoice updates.
When a new subscription client is created followed by an immediate
cancel, it's likely the client will be removed from the registry's
map(noop) and then added to its map again. This subscription will then
be kept in registry until lnd is restarted. Another more serious issue
is when multiple subscriptions are made for the same invoice, when the
above case happens, other subscriptions may never send invoice updates
because a previous client has a stopped notification queue that blocks
following notifications.
In this commit, we add the setID to the invoiceEvent struct as it will
be useful when we need to be able to distinguish a new open invoice,
from an AMP invoice that's being settled for the first time.
we then update the logic during notification dispatch to utilize the new
field to allow it to detect the repeated settles of AMP invoices.
In this commit, we update the logic in `updateInvoice` to allow callers
to pass in either a hint, or the setID in the update callback. This
makes things more efficient for AMP invoices with thousands of recurring
payments, as we no longer need to read out _all_ the invoices each time
we go to update the state of a few HTLCs.
In this commit, we add a new `LookupInvoiceV2` method attached to the
invoice RPC sub-server. Compared to the existing version, this version
allows an invoice to be looked up by an invoice ref. This enables users
to query an AMP invoice based on a specific _set ID_, so they can get
the information related to the set of cumulative settles to that
invoices.
To make it possible to use a remote lnrpc server as a signer for our
wallet, we need to change our main interface to sign the message instead
of the message's digest. Otherwise we'd need to alter the
lnrpc.SignMessage RPC to accept a digest instead of only the message
which has security implications.
This commit adds height-based invoice expiry for hodl invoices
that have active htlcs. This allows us to cancel our intentionally
held htlcs before channels are force closed. We only add this for
hodl invoices because we expect regular invoices to automatically
be resolved.
We still keep hodl invoices in the time-based expiry queue,
because we want to expire open invoices that reach their timeout
before any htlcs are added. Since htlcs are added after the
invoice is created, we add new htlcs as they arrive in the
invoice registry. In this commit, we allow adding of duplicate
entries for an invoice to be added to the expiry queue as each
htlc arrives to keep implementation simple. Our cancellation
logic can already handle the case where an entry is already
canceled, so this is ok.
In order to be consistent with other sub systems an error is now
returned from the Stop functions.
This also allows writing a generic cleanup mechanism to stop all
sub systems in case of a failure.
Adds a set of test cases that exercise the spontaneous AMP payment flow
with valid and invalid reconstructions, as well as with single and
multiple HTLCs. This also asserts that spontaneous AMP is gated behind
the existing AcceptKeysend flag.
In this commit, we move to start rejecting any normal payments that
aren't keysend, if they don't also include the MPP invoice payload. With
this change, we require that some sort of e2e secret (either the payment
addr or the keysend pre-image) is present in a payload before we'll
accept the payment.
The second portion of the commit also updates all current tests in the
package. We kept the base `TestSettleInvoice` test in-tact as it still
exercises some useful behavior. However, we've removed all cases that
allow an overpayment, as the new MPP logic doesn't allow overpayment for
various reasons. In addition to this, some of the returned errors are
slightly different, tho the actual behavior is equivalent.
This commit extends invoice garbage collection to also remove invoices
which are canceled when LND is already up and running. When the option
GcCanceledInvoicesOnTheFly is false (default) then invoices are kept and
the behavior is unchanged.
This commit adds channeldb.ScanInvoices to scan through all invoices in
the database. The new call will also replace the already existing
channeldb.FetchAllInvoicesWithPaymentHash call in preparation to collect
invoices we'd like to delete and watch for expiry in one scan in later
commits.
In this commit, we add a String() method to the failure resolution
outcome. Without this, logs aren't very useful as the integer version of
the outcome is printed rather than the description.
Adds a new configuration flag to lnd that will keep keysend payments in
the accepted state. An application can then inspect the payment
parameters and decide whether to settle or cancel.
The on-the-fly inserted keysend invoices get a configurable expiry time.
This is a safeguard in case the application that should decide on the
keysend payments isn't active.
SettleHodlInvoice and CancelInvoice both notifyClients after
notifyHodlSubscribers. This commit changes UpdateInvoice to follow the
same pattern so that we are consistent.
Previously it wasn't possible to store a preimage in the invoice
database and signal that a payment should not be settled right away. The
only way to hold a payment was to insert the magic UnknownPreimage value
in the invoice database. This commit introduces a distinct flag to
signal that an invoice is a hold invoice and thereby allows the preimage
to be present in the database already.
Preparation for (key send) hodl invoices for which we already know the
preimage.
This commit moves the db calls for retrieving add and settle backlogs
outide of the main event loop. All other db operations are performed
outside of the event loop and synchronized via the invoice registry's
mutex, which also synchronizes the order in which events submitted to be
processed.
This resolves various concurrency issues where notifications can be
missed of inconsistent reads against the databse. This is especially
important in this case because we are actually making two separate
database calls.
This commit adds LinkErrors with failure details to htlcs which fail on
our incoming link. This change is made with the intention of notifying
detailed htlc failure reasons in sendHTLCError. The FailureDetail
interface is implemented on FailureResolutionResults so that they can
directly be used to enrich LinkErrors. sendHtlcError is updated to
take a LinkError in preparation for the addition of a htlcnotifier
which will notify the detail of the error.
This commit splits the resolution result enum into results divided
by outcome (settled, failed or accepted). This allows us to more
strictly control which resolution results can be used with which
HtlcResolution structs, to prevent the combination of a settle
resolution result with a failure resolution result, for example.
This commit repalces the htlcResolution struct with an interface.
This interface is implemeted by failure, settle and accept resolution
structs. Only settles and fails are exported because the existing
code that handles htlc resolutions uses a nil resolution to indicate
that a htlc was accepted. The accept resolution is used internally
to report on the resolution result of the accepted htlc, but a nil
resolution is surfaced. Further refactoring of all the functions
that call NotifyExitHopHtlc to handle a htlc accept case (rather than
having a nil check) is required.
This commit intends to fix slow first startup time when there are many
invoices that need to be canceled. The slowdown is caused by a combination
of adding invoices to the expiry watcher one-by-one and slow
cancellation. Due to slow cancellation and the unbuffered channel which
we use to pass invoices to the expiry watcher blocks the registry.
With this fix we'll instead batch add invoices to the expiry watcher and
thereby won't block the registry startup.
This commit adds handling code for the key send custom record. If this
record is present and its hash matches the payment hash, invoice
registry will insert a new invoice into the database "just in time". The
subsequent settle flow is unchanged. The newly inserted invoice is
picked up and settled. Notifications will be broadcast as usual.
This commit moves handling of invoice not found
errors into NotifyExitHopHtlc and exposes a
resolution result to the calling functions. The
intention of this change is to make calling
functions as naive of the invoice registry's
mechanics as possible.
When NotifyExitHopHtlc is called and an invoice
is not found, calling functions can take action
based on the HtlcResolution's InvoiceNotFound
outcome rather than having to add a special error
check on every call to handle the error.
This commit adds the resolution result obtained
while updating an invoice in the registry to
htlcResolution. The field can be used by calling
functions to determine the outcome of the
update and act appropriately.
This commit adds a constructor for HtlcResolution creation
to enforce provision of all relevant values when an
event is created. A custom construstor which also takes
a preimage is added for settle events.
This commit renames HodlEvent to HtlcResolution
to better reflect the fact that the struct is
only used for htlc settles and cancels, and that
it is not specifically used for hodl invoices.
This commit exports UpdateResult so that
calling functions can interpret the outcome
of an invoice update. This is useful for
determining the wire failure required
(fail invalid details or mpp_timeout once
implemented) and for notifying specific
htlc failure details. The enum is renamed
to ResolutionResult.
This commit adds InvoiceExpryWatcher which is a separate class that
receives new invoices (and existing ones upon restart) from InvoiceRegistry
and actively watches their expiry. When an invoice is expired
InvoiceExpiryWatcher will call into InvoiceRegistry to cancel the
invoice and by that notify all subscribers about the state change.
This commit adds Clock and DefaultClock and moves the private
invoices.testClock under the clock package while adding basic
unit tests for it.
Clock is an interface currently encapsulating Now() and TickAfter().
It can be added as an external dependency to any class. This way
tests can stub out time.Now() or time.After().
The DefaultClock class simply returns the real time.Now() and
time.After().
Previously the cancel and add actions were combined in a single map.
Nil values implictly signaled cancel actions. This wasn't very obvious.
Furthermore this split prepares for processing the adds and cancels
separately, which is more efficient if there are already two maps.
