Commit graph

2981 commits

Author SHA1 Message Date
Matt Corallo
801d6e5256
Merge pull request #1068 from TheBlueMatt/2021-09-ser-cleanup
Simplify Message Serialization and Parse TLV Suffix
2021-09-18 01:42:29 +00:00
Matt Corallo
997dc5f5f3 Convert most P2P msg serialization to a new macro with TLV suffixes
The network serialization format for all messages was changed some
time ago to include a TLV suffix for all messages, however we never
bothered to implement it as there isn't a lot of use validating a
TLV stream with nothing to do with it. However, messages are
increasingly utilizing the TLV suffix feature, and there are some
compatibility concerns with messages written as a part of other
structs having their format changed (see previous commit).

Thus, here we go ahead and convert most message serialization to a
new macro which includes a TLV suffix after a series of fields,
simplifying several serialization implementations in the process.
2021-09-18 01:01:41 +00:00
Matt Corallo
f60da31e56 Add forward-compat due serialization variants of HTLCFailureMsg
Going forward, all lightning messages have a TLV stream suffix,
allowing new fields to be added as needed. In the P2P protocol,
messages have an explicit length, so there is no implied length in
the TLV stream itself. HTLCFailureMsg enum variants have messages
in them, but without a size prefix or any explicit end. Thus, if a
HTLCFailureMsg is read as a part of a ChannelManager, with a TLV
stream at the end, there is no way to differentiate between the end
of the message and the next field(s) in the ChannelManager.

Here we add two new variant values for HTLCFailureMsg variants in
the read path, allowing us to switch to the new values if/when we
add new TLV fields in UpdateFailHTLC or UpdateFailMalformedHTLC so
that older versions can still read the new TLV fields.
2021-09-18 01:01:41 +00:00
Matt Corallo
7f9eef553c [fuzz] Swap mode on most messages to account for TLV suffix 2021-09-18 01:01:41 +00:00
Matt Corallo
07b674ecb1 Drop writer size hinting/message vec preallocation
In order to avoid significant malloc traffic, messages previously
explicitly stated their serialized length allowing for Vec
preallocation during the message serialization pipeline. This added
some amount of complexity in the serialization code, but did avoid
some realloc() calls.

Instead, here, we drop all the complexity in favor of a fixed 2KiB
buffer for all message serialization. This should not only be
simpler with a similar reduction in realloc() traffic, but also
may reduce heap fragmentation by allocating identically-sized
buffers more often.
2021-09-18 01:01:41 +00:00
Matt Corallo
831f124721
Merge pull request #1053 from valentinewallace/2021-08-dedup-payment-sent
Deduplicate PaymentSent events for MPP payments
2021-09-17 20:59:29 +00:00
Matt Corallo
8fad498900 Increase our default/minimum dust limit to 354 sat/vbytes
330 sat/vbyte, the current value, is not sufficient to ensure a
future segwit script longer than 32 bytes meets the dust limit if
used for a shutdown script. Thus, we can either check the value
on shutdown or we can simply require segwit outputs and require a
dust value of no less than 354 sat/vbyte.

We swap the minimum dust value to 354 sat/vbyte here, requiring
segwit scripts in a future commit.

See https://github.com/lightningnetwork/lightning-rfc/issues/905
2021-09-17 20:32:59 +00:00
Matt Corallo
57f54bcc11 Drop broken test that is unfixable due to being undocumented
This should be reverted at some point, but the test is deficient
and breaks on later changes that are important to land ASAP.
2021-09-17 20:32:59 +00:00
Valentine Wallace
c828ff42c0
Add all_paths_failed field to PaymentFailed
see field docs for details
2021-09-17 15:36:27 -04:00
Valentine Wallace
8f1763159e
Prevent duplicate PaymentSent events
by removing all pending outbound payments associated with the same
MPP payment after the preimage is received
2021-09-17 15:36:24 -04:00
Valentine Wallace
ad81addd98
Add MPP ID to pending_outbound_htlcs
We'll use this to correlate MPP shards in upcoming commits
2021-09-17 15:36:21 -04:00
Valentine Wallace
641e20763f
Implement Readable/Writeable for HashSet
To be used in upcoming commits for MPP ID storage
2021-09-17 15:23:45 -04:00
Valentine Wallace
c986e52ce8
Add MppId field to HTLCSource as a way to correlate mpp payment paths 2021-09-17 15:23:45 -04:00
Valentine Wallace
f6f950db75
test utils: refactor fail_payment_along_route for mpp 2021-09-17 15:23:42 -04:00
Valentine Wallace
612d1fb120
Update Watch docs to disallow dup channel outpoints
on watch_channel
2021-09-17 14:50:44 -04:00
Valentine Wallace
e06484b0f4
Don't apply monitor updates after watch_channel PermFail
The full stack fuzzer found an unreachable panic where we receive a
FundingSigned with a duplicate channel outpoint.
2021-09-17 14:50:20 -04:00
Matt Corallo
088daf79aa
Merge pull request #1070 from TheBlueMatt/2021-09-fix-bindings-ignore
Move CounterpartyForwardingInfo from channel to channelmanager
2021-09-17 17:26:54 +00:00
Matt Corallo
78d6c3cb01
Merge pull request #1066 from valentinewallace/2021-08-fix-double-temp-failure
Allow multiple calls to `monitor_update_failed`
2021-09-15 20:51:37 +00:00
Valentine Wallace
3d77cc790c
Allow multiple monitor_update_failed calls
without requiring calls to channel_monitor_updated in between.

Found by the fuzzer
2021-09-15 15:37:27 -04:00
Valentine Wallace
3af0c8e146
Update fuzz README with latest instructions 2021-09-15 15:37:27 -04:00
Matt Corallo
24065c89a9
Merge pull request #1074 from p2pderivatives/add-node-id-to-custom-msg-cb
Add node id to custom message callback
2021-09-15 18:54:15 +00:00
Matt Corallo
35573bb3d7
Merge pull request #1034 from TheBlueMatt/2021-07-maturing-claims
Expose in-flight claim balances
2021-09-15 18:44:04 +00:00
Matt Corallo
bb9df69d0d
Merge pull request #1043 from jkczyz/2021-07-network-update-handler
Handle network updates from failed payments in BackgroundProcessor
2021-09-15 18:13:20 +00:00
Matt Corallo
cae2123793 Expand ANTI_REORG_DELAY docs to say its a library-wide assumption 2021-09-15 18:07:34 +00:00
Matt Corallo
a675886e10 Add an accessor to ChainMonitor to get the claimable balances
The common user desire is to get the set of claimable balances for
all non-closed channels. In order to do so, they really want to
just ask their `ChainMonitor` for the set of balances, which they
can do here by passing the `ChannelManager::list_channels` output
to `ChainMonitor::get_claimable_balances`.
2021-09-15 18:07:34 +00:00
Matt Corallo
00906418aa Expose the amount of funds available for claim in ChannelMonitor
In general, we should always allow users to query for how much is
currently in-flight being claimed on-chain at any time.

This does so by examining the confirmed claims on-chain and
breaking down what is left to be claimed into a new
`ClaimableBalance` enum.

Fixes #995.
2021-09-15 18:07:34 +00:00
Matt Corallo
f9febb0351 Fix indentation in ChannelMonitor 2021-09-15 18:07:34 +00:00
Matt Corallo
c02b6a3807 Track how our HTLCs are resolved on-chain persistently
This tracks how any HTLC outputs in broadcast commitment
transactions are resolved on-chain, storing the result of the HTLC
resolution persistently in the ChannelMonitor.

This can be used to determine which outputs may still be available
for claiming on-chain.
2021-09-15 18:07:34 +00:00
Matt Corallo
73ee30d9da Track the tx which spends our funding output in ChannelMonitor
This allows us to easily look up how our channel was closed and
track which balances may be spendable on-chain.
2021-09-15 18:07:34 +00:00
Matt Corallo
cd578b55f3 Rename CounterpartyCommitmentTransaction to Params as it is static 2021-09-15 18:07:34 +00:00
Matt Corallo
44e48679fb Drop unused CounterpartyCommitmentTransaction::per_htlc HashMap 2021-09-15 18:07:34 +00:00
Matt Corallo
aaba0a2ec9 Store to-self value in the current commitment tx in ChannelMonitor 2021-09-15 18:07:34 +00:00
Jeffrey Czyz
992df51001
Update NetworkGraph in BackgroundProcessor
Decorate the user-supplied EventHandler with NetGraphMsgHandler in
the BackgroundProcessor. The resulting handler will intercept
PaymentFailed events in order to update the NetworkGraph in the
background before delegating to the user's event handler.
2021-09-15 11:53:13 -05:00
Jeffrey Czyz
e2f088c371
Expand and format BackgroundProcessor docs 2021-09-15 11:53:12 -05:00
Jeffrey Czyz
ba2c00b3f8
EventHandler for applying NetworkUpdate
PaymentFailed events contain an optional NetworkUpdate describing
changes to the NetworkGraph as conveyed by a node along a failed payment
path according to BOLT 4. An EventHandler should apply the update to the
graph so that future routing decisions can account for it.

Implement EventHandler for NetGraphMsgHandler to update NetworkGraph.
Previously, NetGraphMsgHandler::handle_htlc_fail_channel_update
implemented this behavior.
2021-09-15 11:50:31 -05:00
Jeffrey Czyz
eff9a47075
Refactor PaymentFailureNetworkUpdate event
MessageSendEvent::PaymentFailureNetworkUpdate served as a hack to pass
an HTLCFailChannelUpdate from ChannelManager to NetGraphMsgHandler via
PeerManager. Instead, remove the event entirely and move the contained
data (renamed NetworkUpdate) to Event::PaymentFailed to be processed by
an event handler.
2021-09-15 11:50:27 -05:00
Tibo-lg
2c6a078d2c Add node id to custom message callback 2021-09-15 09:29:29 +09:00
Matt Corallo
3f9efe717b Move CounterpartyForwardingInfo from channel to channelmanager
CounterpartyForwardingInfo is public (previously exposed with a
`pub use`), and used inside of ChannelCounterparty in
channelmanager.rs. However, it is defined in channel.rs, away from
where it is used.

This would be fine, except that the bindings generator is somewhat
confused by this - it doesn't currently support interpreting
`pub use` as a struct to expose, instead ignoring it.

Fixes https://github.com/lightningdevkit/ldk-garbagecollected/issues/44
2021-09-13 17:31:59 +00:00
Matt Corallo
de9fba82f2
Merge pull request #1072 from TheBlueMatt/2021-09-tighter-max_fee-constant
Reduce our stated max closing-transaction fee to be the true value
2021-09-13 16:42:36 +00:00
Matt Corallo
ce3a04ba1a
Merge pull request #1073 from gilescope/less-deps 2021-09-13 04:22:26 +00:00
Giles Cope
b74ab25b61
Not needed now. We refer to std::futures. 2021-09-11 20:11:10 +01:00
Jeffrey Czyz
bd3ee0ab3d
Fail with PERM|8 (permanent_channel_failure)
This affects the htlc_fail_async_shutdown test.
2021-09-09 23:11:12 -05:00
Jeffrey Czyz
a6749d582d
Remove test_invalid_channel_announcement
It doesn't seem to be testing anything useful that isn't covered
elsewhere.
2021-09-09 23:11:12 -05:00
Jeffrey Czyz
798cf6ea83
Add a read-only view of NetworkGraph
Hide the internal locking of NetworkGraph by providing a read-only
view. This way the locking order is handled internally.
2021-09-09 23:11:12 -05:00
Jeffrey Czyz
16ad7f17a1
Remove RwLock from around NetworkGraph
Now that NetworkGraph uses interior mutability, the RwLock used around
it in NetGraphMsgHandler is no longer needed. This allows for shared
ownership without a lock.
2021-09-09 23:11:09 -05:00
Jeffrey Czyz
777661ae52
Individually lock NetworkGraph fields
In preparation for giving NetworkGraph shared ownership, wrap individual
fields in RwLock. This allows removing the outer RwLock used in
NetGraphMsgHandler.
2021-09-09 23:01:28 -05:00
Jeffrey Czyz
a6e650630d
Pass Event by reference to EventHandler
Passing an Event by reference rather and by move gives more flexibility
for composing event handlers without needing to clone events.
2021-09-09 22:57:51 -05:00
Matt Corallo
b348d9cb60 Reduce our stated max closing-transaction fee to be the true value
When communicating the maximum fee we're willing to accept on a
cooperative closing transaction to our peer, we currently tell them
we'll accept `u64::max_value()` if they're the ones who have to pay
it. Spec-wise this is fine - they aren't allowed to try to claim
our balance, and we don't care how much of their own funds they
want to spend on transaction fees.

However, the Eclair folks prefer to check all values on the wire
do not exceed 21 million BTC, which seems like generally good
practice to avoid overflows and such issues. Thus, our close
messages are rejected by Eclair.

Here we simply relax our stated maximum to be the real value - our
counterparty's current balance in satoshis.

Fixes #1071
2021-09-10 00:24:58 +00:00
Matt Corallo
423f1b1803
Merge pull request #1064 from lightning-signer/2021-08-closing-tx-phase2 2021-09-09 19:31:47 +00:00
Devrandom
eebc0a921e Use ClosingTransaction in BaseSign 2021-09-09 20:49:24 +02:00