The htlcswitch tests were creating temporary database files
but failing to clean them up.
We fix this by making it obvious when temporary db files are
created, and cleaning up those resources where necessary in
the tests.
Previously, the Switch would not check waiting-close channels' fwdpkgs
for settles or fails to reforward. This could result in a force close
in a rare edge case if a restart occurred at the wrong time. Now,
waiting-close fwdpkgs are checked and the issue is avoided.
NewForce's Pause method doesn't reset the ticker, so a test flake
would occur in TestChannelLinkCancelFullCommitment where
PendingCommitTicker.Pause() was called, but the underlying timer was
still ticking. When PendingCommitTicker.Resume() was called, an
unlucky Ticks() call could end up firing, leading to the link being
shut down.
In this commit, we parse the new max fee field, and pass it through the
switch, all the way to the peer where it's ultimately passed into the
chan closer state machine.
This intent of this change is to prevent privacy leaks when routing
with aliases and also to allow routing when using an alias. The
aliases are our aliases.
Introduces are two maps:
* aliasToReal:
This is an N->1 mapping for a channel. The keys are the set of
aliases and the value is the confirmed, on-chain SCID.
* baseIndex:
This is also an N->1 mapping for a channel. The keys are the set
of aliases and the value is the "base" SCID (whatever is in the
OpenChannel.ShortChannelID field). There is also a base->base
mapping, so not all keys are aliases.
The above maps are populated when a link is added to the switch and
when the channel has confirmed on-chain. The maps are not removed
from if the link is removed, but this is fine since forwarding won't
occur.
* getLinkByMapping
This function is introduced to adhere to the spec requirements that
using the confirmed SCID of a private, scid-alias-feature-bit
channel does not work. Lnd implements a stricter version of the spec
and disallows this behavior if the feature-bit was negotiated, rather
than just the channel type. The old, privacy-leak behavior is
preserved.
The spec also requires that if we must fail back an HTLC, the
ChannelUpdate must use the SCID of whatever was in the onion, to avoid
a privacy leak. This is also done by passing in the relevant SCID to
the mailbox and link. Lnd will also cancel back on the "incoming" side
if the InterceptableSwitch was used or if the link failed to decrypt
the onion. In this case, we are cautious and replace the SCID if an
alias exists.
This introduces a BigSize migration that is used to expand the width
of the ChannelStatus and ChannelType fields. Three channel "types"
are added - ZeroConfBit, ScidAliasChanBit, and ScidAliasFeatureBit.
ScidAliasChanBit denotes that the scid-alias channel type was
negotiated for the channel. ScidAliasFeatureBit denotes that the
scid-alias feature bit was negotiated during the *lifetime* of the
channel. Several helper functions on the OpenChannel struct are
exposed to aid callers from different packages.
The RefreshShortChanID has been renamed to Refresh.
A new function BroadcastHeight is used to guard access to the
mutable FundingBroadcastHeight member. This prevents data races.
With this, extra calls to RemoveLink will wait for the link to
fully stop. This is accomplished by a map that stores a single stop
channel that callers to RemoveLink will listen on. This map is not
consulted when the Switch is shutting down and calls Stop on each
individual link. Though that could be added in the future, it is
not necessary.
Warning messages are intended to add "softer" failure modes for peers,
so to start with we simply log the warnings sent to us. While we "may"
disconnect from the peer according to the spec, we start with the least
extreme option (which is also not a change in behavior because
previously we'd just log that we received an unknown odd message).
This allows Switch-initiated payments to be failed back if they don't
make it into a commitment. Prior to this commit, a Switch-initiated
HTLC could get "lost" meaning the circuit wouldn't get deleted except
if conditions were "right" and the network result store would never
be made aware of the HTLC's fate. Switch-initiated HTLC's are now
passed to the link's mailbox to ensure they can be failed back.
This change also special-cases the ErrDuplicateKeystone error from
OpenCircuits(...) so that callers of updateCommitTx() in the link
don't send an Error to the peer if they encounter the keystone error.
With the first async change, the keystone error should now always
be recoverable.
This commit changes the `NewBreachRetribution` to use the new revocation
log format, while maintaining the compatibilty to use an older
revocation log format. Unit tests have been added to make sure a breach
retribution can be created in both log formats.
This also means the watch tower needs to pass the relevant commit tx at
its backup height when creating the breach retribution during backing
up. This is achieved by recording the current remote commitment state
before advancing the remote commitment chain.
In this commit we move the tracking of the outstanding intercepted htlcs
to InterceptableSwitch. This is a preparation for making the htlc
interceptor required.
Required interception involves tracking outstanding htlcs across
multiple grpc client sessions. The per-session routerrpc
forwardInterceptor object is therefore no longer the best place for
that.
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
The counter-party shouldn't be doing this anyways as they would be
giving away a preimage for free. Them doing this would bork their
own channel due to open circuits not getting trimmed on startup.
Removing this faulty behavior also makes it easier to reason about
the circuit logic.
This commit modifies the channel state machine to be able to derive the
proper commitment and second-level HTLC output scripts required by the
new script-enforced leased channel commitment type.
Pass htlc amount down to the channel so that we don't need to rely
on minHtlc (and pad it when the channel sets a 0 min htlc). Update
test to just check some sane values since we're no longer relying
on minHtlc amount at all.
This commit makes SendHTLC (we are the source) evaluate the dust
threshold of the outgoing channel against the default threshold of
500K satoshis. If the threshold is exceeded by adding this HTLC, we
fail backwards. It also makes handlePacketForward (we are forwarding)
evaluate the dust threshold of the incoming channel and the outgoing
channel and fails backwards if either channel's dust sum exceeds the
default threshold.
This commit extends the Mailbox interface with the SetDustClosure,
SetFeeRate, and DustPackets methods. This enables the mailbox to
report the dust exposure to the Switch when the Switch decides whether
to forward a dust packet. The dust is counted from the time an Add is
introduced via AddPacket until it is removed via AckPacket. This can
lead to some packets being counted twice before they are signed for,
but this is a trade-off between accuracy and simplicity.
With go 1.17 a change to the build flags was implemented:
https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
The formatter now automatically adds the forward-compatible build tag
format and the linter checks for them, so we need to include them in our
code.
When Alice receives a CommitSig near the end of the test, the test
assumed that she wouldn't reply with a RevokeAndAck message before
shutdownAssert was called. This led to a test flake. Instead, only
call shutdownAssert after we've received the RevokeAndAck.
In this commit, we take an initial step towards converting the existing
breach arbiter and utxo nursery logic into contract resolvers by moving
the files as is, into the `contractcourt` pacakge.
This commit is primarily move only, though we had to massage some
interfaces and config names along the way to make things compile and the
tests run properly.
In this commit, a new method `cleanClosedChannels` is added and called
when a circuit map is created. This method will delete the payment
circuits and keystones for closed channels.
This allows a caller to ensure to optimistically shut down the link
if the channel is clean. If the channel is not clean, an error is
returned and the link continues functioning as normal. The caller
should also call RemoveLink to ensure that the link isn't seen as
usable within the switch.
In lnd, log messages about channels are generally logged with a
reference to their channel point rather than the short channel id.
Channel point is reorg-resistant and also easier to look up in for
example a block explorer.
In the link however, all log messages are accompanied by short channel
id. This makes it difficult to grep a log for all channel activity. The
PEER message for example which are often crucial to analyse, are logged
with channel points.
This commit modifies the link logging to also use channel points.
This commit allows the peer to be tested without relying on a raw
htlcswitch.Switch pointer. This is accomplished by using a messageSwitch
interface and adding the CreateAndAddLink method to the
htlcswitch.Switch.
Even though the sphinx router's persistent replay log is not crucial in
the operation of lnd as its state can be re-created by creating a new
brontide connection, we want to make lnd fully stateless and therefore
have the option of not storing any state on disk.
Until now, clients of SubscribeHTLCEvents didn't have access to the settled preimage. The API allows to intercept forward event and to be updated on forward events however the forward+settle event does not include the payment preimage. This pr changes allows it.
This commit adds height-based invoice expiry for hodl invoices
that have active htlcs. This allows us to cancel our intentionally
held htlcs before channels are force closed. We only add this for
hodl invoices because we expect regular invoices to automatically
be resolved.
We still keep hodl invoices in the time-based expiry queue,
because we want to expire open invoices that reach their timeout
before any htlcs are added. Since htlcs are added after the
invoice is created, we add new htlcs as they arrive in the
invoice registry. In this commit, we allow adding of duplicate
entries for an invoice to be added to the expiry queue as each
htlc arrives to keep implementation simple. Our cancellation
logic can already handle the case where an entry is already
canceled, so this is ok.
Having it set to nil caused https://github.com/lightningnetwork/lnd/issues/5115
The problem was several layers removed from the fix. The link decides to
clean up a `fwdPkg` only if it's completed, otherwise it renotifies the
HTLCs. A package is only set to complete if it's `addAck` and
`settleFail` filters are full. For forwarded HTLCs, the `addAck` was
never being set so it would never be considered complete under this
criteria.
`addAck` is set for an HTLC when signing the next commitment TX in the
`LightningChannel`. The path for this is:
* `LightningChannel#SettleHtlc` adds the HTLC to `localUpdates`
* `LightningChannel#SignNextCommitment` builds the `ackAddRef` for all
updates with `SourceRef != nil`.
* `LightningChannel#SignNextCommitment` then passes the list of
`ackAddRef` to `OpenChannel#AppendRemoteCommitChain` to persist the new
acks in the filter
Since `SourceRef` was nil for interceptor packages, `SignNextCommitment`
ignored it and the ack was never persisted.
To distinguish the attempt's unique ID from the overall payment
identifier, we name it attemptID everywhere, and note that the
paymentHash argument won't be the actual payment hash for AMP payments.
In order to be consistent with other sub systems an error is now
returned from the Stop functions.
This also allows writing a generic cleanup mechanism to stop all
sub systems in case of a failure.
Messages:
- UpdateFulfillHTLC
- UpdateFee
- UpdateFailMalformedHTLC
- UpdateFailHTLC
- UpdateAddHTLC
- Shutdown
- RevokeAndAck
- ReplyShortChanIDsEnd
- ReplyChannelRange
- QueryShortChanIDs
- QueryChannelRange
- NodeAnnouncement
- Init
- GossipTimestampRange
- FundingSigned
- FundingLocked
- FundingCreated
- CommitSig
- ClosingSigned
- ChannelUpdate
- ChannelReestablish
- ChannelAnnouncement
- AnnounceSignatures
lnwire: update quickcheck tests, use constant for Error
multi: update unit tests to pass deep equal assertions with messages
In this commit, we update a series of unit tests in the code base to now
pass due to the new wire message encode/decode logic. In many instances,
we'll now manually set the extra bytes to an empty byte slice to avoid
comparisons that fail due to one message having an empty byte slice and
the other having a nil pointer.
In this commit, we modify the way we write wire messages across the
entire database. We'll now ensure that we always write wire messages
with a length prefix. We update the `codec.go` file to always write a 2
byte length prefix, this affects the way we write the `CommitDiff` and
`LogUpdates` struct to disk, and the network results bucket in the
switch as it includes a wire message.
This commit caps the update fee the initiator will send when the anchors
channel type is used. We do not limit anything on the receiver side.
10 sat/vbyte is the current default max fee rate we use. This should be
enough to ensure propagation before anchoring down the commitment
transaction.
* mod: bump btcwallet version to accept db timeout
* btcwallet: add DBTimeOut in config
* kvdb: add database timeout option for bbolt
This commit adds a DBTimeout option in bbolt config. The relevant
functions walletdb.Open/Create are updated to use this config. In
addition, the bolt compacter also applies the new timeout option.
* channeldb: add DBTimeout in db options
This commit adds the DBTimeout option for channeldb. A new unit
test file is created to test the default options. In addition,
the params used in kvdb.Create inside channeldb_test is updated
with a DefaultDBTimeout value.
* contractcourt+routing: use DBTimeout in kvdb
This commit touches multiple test files in contractcourt and routing.
The call of function kvdb.Create and kvdb.Open are now updated with
the new param DBTimeout, using the default value kvdb.DefaultDBTimeout.
* lncfg: add DBTimeout option in db config
The DBTimeout option is added to db config. A new unit test is
added to check the default DB config is created as expected.
* migration: add DBTimeout param in kvdb.Create/kvdb.Open
* keychain: update tests to use DBTimeout param
* htlcswitch+chainreg: add DBTimeout option
* macaroons: support DBTimeout config in creation
This commit adds the DBTimeout during the creation of macaroons.db.
The usage of kvdb.Create and kvdb.Open in its tests are updated with
a timeout value using kvdb.DefaultDBTimeout.
* walletunlocker: add dbTimeout option in UnlockerService
This commit adds a new param, dbTimeout, during the creation of
UnlockerService. This param is then passed to wallet.NewLoader
inside various service calls, specifying a timeout value to be
used when opening the bbolt. In addition, the macaroonService
is also called with this dbTimeout param.
* watchtower/wtdb: add dbTimeout param during creation
This commit adds the dbTimeout param for the creation of both
watchtower.db and wtclient.db.
* multi: add db timeout param for walletdb.Create
This commit adds the db timeout param for the function call
walletdb.Create. It touches only the test files found in chainntnfs,
lnwallet, and routing.
* lnd: pass DBTimeout config to relevant services
This commit enables lnd to pass the DBTimeout config to the following
services/config/functions,
- chainControlConfig
- walletunlocker
- wallet.NewLoader
- macaroons
- watchtower
In addition, the usage of wallet.Create is updated too.
* sample-config: add dbtimeout option
Since it turned out borking channels on every received error could cause
us to bork channels in case of a sync error with C-lightning, we revert
this for now.
Similarly as with kvdb.View this commits adds a reset closure to the
kvdb.Update call in order to be able to reset external state if the
underlying db backend needs to retry the transaction.
This commit adds a reset() closure to the kvdb.View function which will
be called before each retry (including the first) of the view
transaction. The reset() closure can be used to reset external state
(eg slices or maps) where the view closure puts intermediate results.
This commit changes the logic when garbage collecting forwarding
packages such that they are removed once when the function is called,
and then again upon subsequent ticks. This allows us to bump the
peer timer to 1 hour to limit the number of db transactions happening
in lnd. The forwarding packages need to be removed initially as
otherwise a flappy node will never have them garbage collected.
This commit moves all localized instances of mock implementations of
the Signer interface to the lntest/mock package. This allows us to
remove a lot of code and have it housed under a single interface in
many cases.
In this commit we implement a wrapper arround the switch, called
InterceptableSwitch. This kind of wrapper behaves like a proxy which
intercepts forwarded packets and allows an external interceptor to
signal if it is interested to hold this forward and resolve it
manually later or let the switch execute its default behavior.
This infrastructure allows the RPC layer to expose interceptor
registration API to the user and by that enable the implementation
of custom routing behavior.
As part of the preparation to the switch interceptor feature, this
function is changed to return error instead of error channel that
is closed automatically.
Returning an error channel has become complex to maintain and
implement when adding more asynchronous flows to the switch.
The change doesn't affect the current behavior which logs the
errors as before.
Previously it wasn't possible to store a preimage in the invoice
database and signal that a payment should not be settled right away. The
only way to hold a payment was to insert the magic UnknownPreimage value
in the invoice database. This commit introduces a distinct flag to
signal that an invoice is a hold invoice and thereby allows the preimage
to be present in the database already.
Preparation for (key send) hodl invoices for which we already know the
preimage.
The btcsuite/fastsha256 registers itself in the crypto package of
golang as a replacement for sha256. This causes problems in TLS1.3
connections that require the hash implementations to be serializable
and results in the "tls: internal error: failed to clone hash" error.
By removing all uses of the library we fix that error.
This commit extends the link with a new synchronous delivery point for
local UpdateAddHTLC messages. The switch method SendHTLC is updated to
use this delivery point and thereby becomes a synchronous call.
For MPP payments, synchronous hand-off is important. Otherwise the next
pathfinding round could start without the channel balance updated yet.
Modifies some of the older mailbox tests that were left untouched as of
PR #4174 to use the new mailbox test context. This ensures that the
config members of each mailbox's config are properly initialized. In
certain instances where travis is slow, this would cause test panics.
This commit adds a PendingCommitTicker to the link config, which allows
us to control how quickly we fail the link if the commitment dance
stalls. Now that the mailbox has the ability to cancel packets, when the
link fails it will reset the mailbox packets on exit, forcing a
reevaluation of the HTLCs against their mailbox expiries.
Now that packet failure is handled by the mailbox, we can now enforce
a delivery deadline and fail the packet if it the deadilne is exceeded.
This gives senders quicker feedback about tried routes, and allows them
to try alternative paths to the destination in the meantime.
This commit splits the packet courier internally into two distinct
queues, one for adds and one for settles+fails. This allows us to
prioritize HTLCs that will clear the commitment transaction and make
space for adds. Previously this responsibility was handled by the
overflow queue.
This commit delays the advancement of the pktHead until after the
message has been delivered. This is a prepatory step, as in the future
we may fail to deliver the packet due to a deadline expiring.
This commit moves the current logic for sending failures out of the link
and into the mailbox in preparation for our failing delayed htlcs. We do
so because the mailbox may need to fail packets while the link is
offline, and needs to be able to complete the task without member
methods on the link.
This commit removes the overflowQueue from the link. We do so in order
to promote better UX for senders, so that HTLCs are failed faster when
the commitment is full. This gives the sender the opportunity to try
another, more open path, rather than perceive the HTLC as being stuck.
At the same time, we remove the total number of active goroutines in lnd
by a factor of N where N is the number of active channels.
This commit is preparation for the test added in the subsequent commit.
We modify makeHoldPayment to return any failures direectly when trying
to add an HTLC to the switch. This lets us know that the HTLC was indeed
sent without failure when the method returns.
A following commits will move/modify callsites of AckPacket, Start, and
Stop, none of which use the return value and ultimately cause the linter
to complain. However, none of these in-memory operations can fail so we
just remove the returned errors altogether.
The linter complains about not checking the return value from
WipeChannel in certain places. Instead of checking we simply remove the
returned error because the in-memory modifications cannot fail.
