Commit graph

4870 commits

Author SHA1 Message Date
Valentine Wallace
0e88538b78
Disambiguate ignorable and ignorable_option
Would rather not rename ignorable to ignorable_required, so rename both of them
to upgradable_*
2023-02-24 14:21:11 -05:00
Valentine Wallace
52551a9fc8
Support deserializing an Option-al MaybeReadable
Prior to this change, our impl_writeable_tlv_based macros only supported
deserializing a MaybeReadable if it's non-Optional.
2023-02-24 14:21:11 -05:00
Valentine Wallace
2037a241f4
Remove all_paths_failed from PaymentPathFailed
This field was previous useful in manual retries for users to know when all
paths of a payment have failed and it is safe to retry. Now that we support
automatic retries in ChannelManager and no longer support manual retries, the
field is no longer useful.

For backwards compat, we now always write false for this field. If we didn't do
this, previous versions would default this field's value to true, which can be
problematic because some clients have relied on the field to indicate when a
full payment retry is safe.
2023-02-24 14:21:08 -05:00
wpaulino
22d1bab7ac
Merge pull request #2050 from douglaz/import-lightning-io-futures
Only import lightning::io if futures are enabled
2023-02-24 11:03:32 -08:00
Allan Douglas R. de Oliveira
19ef2bb6ac Only import lightning::io if futures are enabled 2023-02-24 02:56:11 +00:00
Elias Rohrer
e878f75bef
Make test_esplora_syncs more robust
The test generally works and the parts in question *should* never fail.
However, they did, so we give the test server instances some leeway.
2023-02-23 20:05:16 -06:00
Jeffrey Czyz
9c2a3d090b
Fix amount overflow in Invoice building
An overflow can occur when multiplying the offer amount by the requested
quantity when no amount is given in the request. Return an error instead
of overflowing.
2023-02-23 18:25:50 -06:00
Jeffrey Czyz
32ed69a2bd
Fix amount overflow in Offer parsing and building
An overflow can occur when multiplying the offer amount by the requested
quantity when checking if the given amount is enough. Return an error
instead of overflowing.
2023-02-23 18:25:50 -06:00
Jeffrey Czyz
3d41df025d
Fuzz test for bech32 decoding
Fuzz testing bech32 decoding along with deserializing the underlying
message can result in overly exhaustive searches. Instead, the message
deserializations are now fuzzed separately. Add fuzzing for bech32
decoding.
2023-02-23 18:25:49 -06:00
Jeffrey Czyz
56a01de61d
Expose Bech32Encode trait for fuzzing
In order to fuzz test Bech32Encode parsing independent of the underlying
message deserialization, the trait needs to be exposed. Conditionally
expose it only for fuzzing.
2023-02-23 18:25:49 -06:00
Jeffrey Czyz
16168d4c07
Fuzz test for parsing Invoice
An invoice is serialized as a TLV stream and encoded as bytes. Add a
fuzz test that parses the TLV stream and deserializes the underlying
Invoice. Then compare the original bytes with those obtained by
re-serializing the Invoice.
2023-02-23 18:25:49 -06:00
Jeffrey Czyz
e049e97993
Fuzz test for parsing InvoiceRequest
An invoice request is serialized as a TLV stream and encoded as bytes.
Add a fuzz test that parses the TLV stream and deserializes the
underlying InvoiceRequest. Then compare the original bytes with those
obtained by re-serializing the InvoiceRequest.
2023-02-23 18:24:41 -06:00
Matt Corallo
d7c818a3ad Rename BestBlock::from_genesis to from_network for clarity 2023-02-24 00:22:58 +00:00
Matt Corallo
2c3e12e309 Remove genesis block hash from public API
Forcing users to pass a genesis block hash has ended up being
error-prone largely due to byte-swapping questions for bindings
users. Further, our API is currently inconsistent - in
`ChannelManager` we take a `Bitcoin::Network` but in `NetworkGraph`
we take the genesis block hash.

Luckily `NetworkGraph` is the only remaining place where we require
users pass the genesis block hash, so swapping it for a `Network`
is a simple change.
2023-02-24 00:22:58 +00:00
Valentine Wallace
f361aa62a4
Add missing import path in ser macro 2023-02-23 17:04:04 -05:00
Matt Corallo
46fd7035b3
Merge pull request #2014 from valentinewallace/2023-02-rework-partial-pmt-fail
Rework auto-retry send errors
2023-02-23 21:54:16 +00:00
Valentine Wallace
a6e9123d3f
Clarify Retry::Timeout vs PaymentParams::expiry_time in docs 2023-02-23 15:50:25 -05:00
Valentine Wallace
12bcc9ae43
Fix outdated PendingOutboundPayment::Abandoned docs 2023-02-23 15:50:25 -05:00
Valentine Wallace
1224dac862
On initial send retries, avoid previously failed scids
Previously, we could have tried the same failed channels over and over until
retries are exhausted.
2023-02-23 15:50:25 -05:00
Valentine Wallace
b826d1735d
In-line retry_with_route method
Since it's only used one place now
2023-02-23 15:50:25 -05:00
Valentine Wallace
d471d9746c
Rework auto retry send errors
Prior to this, we returned PaymentSendFailure from auto retry send payment
methods. This implied that we might return a PartialFailure from them, which
has never been the case. So it makes sense to rework the errors to be a better
fit for the methods.

We're taking error handling in a totally different direction now to make it
more asynchronous, see send_payment_internal for more information.
2023-02-23 15:50:23 -05:00
Valentine Wallace
5e4f0bcff0
Fix InvalidRoute error to be ChannelUnavailable
InvalidRoute is reserved for malformed routes, not routes where a channel or
its peer is unavailable
2023-02-22 23:05:43 -05:00
Matt Corallo
96c8507fbf
Merge pull request #1897 from TheBlueMatt/2022-11-monitor-updates-always-async
Always process `ChannelMonitorUpdate`s asynchronously
2023-02-22 19:12:31 +00:00
Matt Corallo
2adb8eeb49 Don't generate a ChannelMonitorUpdate for closed chans on shutdown
The `Channel::get_shutdown` docs are very clear - if the channel
jumps to `Shutdown` as a result of not being funded when we go to
initiate shutdown we should not generate a `ChannelMonitorUpdate`
as there's no need to bother with the shutdown script - we're
force-closing anyway.

However, this wasn't actually implemented, potentially causing a
spurious monitor update for no reason.
2023-02-22 17:34:46 +00:00
Matt Corallo
685b08d8c1 Use the new monitor persistence flow for funding_created handling
Building on the previous commits, this finishes our transition to
doing all message-sending in the monitor update completion
pipeline, unifying our immediate- and async- `ChannelMonitor`
update and persistence flows.
2023-02-22 17:34:46 +00:00
Matt Corallo
c35253d6a8 Use new monitor persistence flow in funding_signed handling
In the previous commit, we moved all our `ChannelMonitorUpdate`
pipelines to use a new async path via the
`handle_new_monitor_update` macro. This avoids having two message
sending pathways and simply sends messages in the "monitor update
completed" flow, which is shared between sync and async monitor
updates.

Here we reuse the new macro for handling `funding_signed` messages
when doing an initial `ChannelMonitor` persistence. This provides
a similar benefit, simplifying the code a trivial amount, but
importantly allows us to fully remove the original
`handle_monitor_update_res` macro.
2023-02-22 17:34:46 +00:00
Matt Corallo
4e002dcf5c Always process ChannelMonitorUpdates asynchronously
We currently have two codepaths on most channel update functions -
most methods return a set of messages to send a peer iff the
`ChannelMonitorUpdate` succeeds, but if it does not we push the
messages back into the `Channel` and then pull them back out when
the `ChannelMonitorUpdate` completes and send them then. This adds
a substantial amount of complexity in very critical codepaths.

Instead, here we swap all our channel update codepaths to
immediately set the channel-update-required flag and only return a
`ChannelMonitorUpdate` to the `ChannelManager`. Internally in the
`Channel` we store a queue of `ChannelMonitorUpdate`s, which will
become critical in future work to surface pending
`ChannelMonitorUpdate`s to users at startup so they can complete.

This leaves some redundant work in `Channel` to be cleaned up
later. Specifically, we still generate the messages which we will
now ignore and regenerate later.

This commit updates the `ChannelMonitorUpdate` pipeline across all
the places we generate them.
2023-02-22 17:34:46 +00:00
Matt Corallo
46c6fb7f91 Move TODO from handle_monitor_update_res into Channel
The TODO mentioned in `handle_monitor_update_res` about how we
might forget about HTLCs in case of permanent monitor update
failure still applies in spite of all our changes. If a channel is
drop'd in general, monitor-pending updates may be lost if the
monitor update failed to persist.

This was always the case, and is ultimately the general form of the
the specific TODO, so we simply leave comments there
2023-02-22 00:51:13 +00:00
Matt Corallo
9802afa53b Handle MonitorUpdateCompletionActions after monitor update sync
In a previous PR, we added a `MonitorUpdateCompletionAction` enum
which described actions to take after a `ChannelMonitorUpdate`
persistence completes. At the time, it was only used to execute
actions in-line, however in the next commit we'll start (correctly)
leaving the existing actions until after monitor updates complete.
2023-02-22 00:51:13 +00:00
Matt Corallo
435b3b4802
Merge pull request #1988 from TheBlueMatt/2023-01-limited-chans
Limit the number of pending un-funded inbound channel
2023-02-22 00:39:13 +00:00
Matt Corallo
d5fb804a32 Limit the number of pending un-funded inbound channel
Because we store some (not large, but not zero) state per-peer,
it's useful to limit the number of peers we have connected, at
least with some buffer.

Much more importantly, each channel has a relatively large cost,
especially around the `ChannelMonitor`s we have to build for each.

Thus, here, we limit the number of channels per-peer which aren't
(yet) on-chain, as well as limit the number of (inbound) peers
which don't have a (funded-on-chain) channel.

Fixes #1889
2023-02-21 22:01:47 +00:00
Matt Corallo
4155f54716 Add an inbound flag to the peer_connected message handlers
Its useful for the message handlers to know if a peer is inbound
for DoS decision-making reasons.
2023-02-21 22:00:42 +00:00
Matt Corallo
e954ee8256
Merge pull request #2035 from TheBlueMatt/2023-02-fix-no-con-discon
Fix (and DRY) the conditionals before calling peer_disconnected
2023-02-21 21:28:05 +00:00
Jeffrey Czyz
9a65709239
Derive traits for InvoiceRequest and Invoice
Offer and Refund derive Debug, Clone, and PartialEq. For consistency,
derive these traits for InvoiceRequest and Invoice as well.
2023-02-21 14:07:51 -06:00
Matt Corallo
068c856638
Merge pull request #2040 from alecchendev/2023-02-indexed-map-btreeset-to-vec
Replace `BTreeSet` in `IndexedMap` with sorted `Vec`
2023-02-21 19:57:51 +00:00
Matt Corallo
be6f263825 Remove the peer_disconnected no_connection_possible flag
Long ago, we used the `no_connection_possible` to signal that a
peer has some unknown feature set or some other condition prevents
us from ever connecting to the given peer. In that case we'd
automatically force-close all channels with the given peer. This
was somewhat surprising to users so we removed the automatic
force-close, leaving the flag serving no LDK-internal purpose.

Distilling the concept of "can we connect to this peer again in the
future" to a simple flag turns out to be ripe with edge cases, so
users actually using the flag to force-close channels would likely
cause surprising behavior.

Thus, there's really not a lot of reason to keep the flag,
especially given its untested and likely to be broken in subtle
ways anyway.
2023-02-21 19:17:06 +00:00
Matt Corallo
0f07d5c0b0 Add a further debug_assert that disconnecting peers are connected 2023-02-21 18:54:52 +00:00
Matt Corallo
10e06331f3 Correct funding_transaction_generated err msg and fix fuzz check
This fixes new errors in `full_stack_target` pointed out by
Chaincode's generous fuzzing infrastructure. Specifically, there's
no reason to check the error message in the
`funding_transaction_generated` return value - it can only return
a failure if the channel has closed since the funding transaction
was generated (which is fine) or if the signer refuses to sign
(which can't happen in fuzzing).
2023-02-21 18:54:52 +00:00
Matt Corallo
ca1b8bdf60 Correct the "is peer live" checks in PeerManager
In general, we should be checking if a `Peer` has `their_features`
set as the "is this peer connected and have they finished the
handshake" flag as it indicates an `Init` message was received.

While none of these appear to be reachable bugs, there were a
number of places where we checked other flags for this purpose,
which may lead to sending messages before `Init` in the future.

Here we clean these cases up to always use the correct check (via
the new util method).
2023-02-21 18:54:52 +00:00
Matt Corallo
73e2fdf332 Add test of an initial message other than Init
This test fails without the previous commit.
2023-02-21 18:54:52 +00:00
Matt Corallo
3554678e9c Fix (and DRY) the conditionals before calling peer_disconnected
If we have a peer that sends a non-`Init` first message, we'll call
`peer_disconnected` without ever having called `peer_connected`
(which has to wait until we have an `Init` message). This is a
violation of our API guarantees, though should generally not be an
issue.

Because this bug was repeated in a few places, we also take this
opportunity to DRY up the logic which checks the peer state before
calling `peer_disconnected`.

Found by the new `ChannelManager` assertions and the
`full_stack_target` fuzzer.
2023-02-21 17:12:27 +00:00
Alec Chen
62a88f97de Replace BTreeSet in IndexedMap with sorted Vec
The `Vec` is sorted not on `IndexedMap::insert`, but on
`IndexedMap::range` to avoid unnecessary work while reading a network
graph.
2023-02-19 22:22:11 -06:00
Valentine Wallace
9f41bd7f64
Pass pending_events into pay_internal
Useful for generating Payment(Path)Failed events in this method
2023-02-19 18:01:01 -05:00
Valentine Wallace
cb81d27f42
Pass payment hash into pay_internal
Useful for generating Payment(Path)Failed events in this method
2023-02-19 18:01:01 -05:00
Matt Corallo
558b2f2904
Merge pull request #2026 from valentinewallace/2023-02-dedup-pending-forwardable-evs
Deduplicate `PendingHTLCsForwardable` events on generation
2023-02-19 18:25:03 +00:00
Valentine Wallace
a2489b126f
Deduplicate PendingHTLCsForwardable events when queueing 2023-02-17 17:41:21 -05:00
Valentine Wallace
bf03d4ccbe
On retryable update_fail, don't queue redundant PendingHTLCsForwardable 2023-02-17 17:35:09 -05:00
Valentine Wallace
5ea433f71f
Check for abandon-able payments on startup 2023-02-17 17:14:43 -05:00
Matt Corallo
ba07622d05 Add a new monitor update result handling macro
Over the next few commits, this macro will replace the
`handle_monitor_update_res` macro. It takes a different approach -
instead of receiving the message(s) that need to be re-sent after
the monitor update completes and pushing them back into the
channel, we'll not get the messages from the channel at all until
we're ready for them.

This will unify our message sending into only actually fetching +
sending messages in the common monitor-update-completed code,
rather than both there *and* in the functions that call `Channel`
when new messages are originated.
2023-02-17 19:09:28 +00:00
Matt Corallo
34218cc4ee Add storage for ChannelMonitorUpdates in Channels
In order to support fully async `ChannelMonitor` updating, we need
to ensure that we can replay `ChannelMonitorUpdate`s if we shut
down after persisting a `ChannelManager` but without completing a
`ChannelMonitorUpdate` persistence. In order to support that we
(obviously) have to store the `ChannelMonitorUpdate`s in the
`ChannelManager`, which we do here inside the `Channel`.

We do so now because in the coming commits we will start using the
async persistence flow for all updates, and while we won't yet
support fully async monitor updating it's nice to get some of the
foundational structures in place now.
2023-02-17 19:09:28 +00:00