When a node retires a failed path as part of a larger MPP payment,
the node may wish to use a path which is constrained by an
`htlc_minimum_msat` value. In this case, the node is forced to
overpay, likely overshooting the `total_msat` it set in the earlier
onions for the same MPP payment.
There are two possible solutions to this - either allow the
`total_msat` value to change in later HTLCs or allow the node to
(slightly) overshoot the `total_msat` value.
Allowing `total_msat` to change across HTLCs is nontrivial to
implement - HTLCs may arrive out-of-order, causing the receiving
node to have to track all seen `total_msat` values and accept a
set of HTLCs which meet any of the seen `total_msat` values.
Instead, this commit changes the MPP logic to simply allow a sender
to overshoot the stated `total_msat`.
Sadly the backwards-compatibility story for this is not great.
There doesn't seem to be a good way to resolve this issue in a
backwards-compatible way. Instead we just bite the bullet and make
the incompatible change, hoping the overshooting is rare enough
that it's not a major issue.
My measurements a few weeks ago reveal that only 5 nodes do not
advertize this feature, of over 17000. I have a patch to
remove support from c-lightning, too.
[ 6 months later: t-bast notes that they only see 0.2% of htlcs using
legacy, and my node hasn't seen one for 2 months w/ 12000 htlcs --RR ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is the only one, so I simply removed it. We'd notice if a new field
was introduced which didn't change the output these days, but this has been
here since 2017.
Here's the difference in extract-formats.py's output:
```diff
@@ -177,6 +177,9 @@
msgtype,final_incorrect_htlc_amount,19
msgdata,final_incorrect_htlc_amount,incoming_htlc_amt,u64,
msgtype,channel_disabled,UPDATE|20
+msgdata,channel_disabled,flags,u16,
+msgdata,channel_disabled,len,u16,
+msgdata,channel_disabled,channel_update,byte,len
msgtype,expiry_too_far,21
msgtype,invalid_onion_payload,PERM|22
msgdata,invalid_onion_payload,type,bigsize,
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Some ChaCha20 implementations API's support both 64- and 96-bit nonces, while
others only support a single one.
Functionally, both nonce sizes are equivalent for LN usage, since the
nonce is always zeroed. However, while evaluating spec compliance of
ChaCha20 libraries, the fact that some do not support the 8 byte nonce
variant prompted a closer investigation about the nonce requirement.
Since RFC8439 is the one linked to in the current BOLT0004 spec and that
RFC only specifies the 96-bit nonce variant, that requirement is made
more explicit by this commit.
* Rename all the 'varint' to 'bigsize'.
Having both is confusing; we chose the name bigsize, so use it
explicitly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* BOLT 7: use `byte` instead of `u8`.
`u8` isn't a type; see BOLT #1 "Fundamental Types".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* BOLT 1: promote bigsize to a Fundamental Type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ECDSA signatures in Bitcoin are DER-encoded but public keys are not.
The compressed format for public keys is for example standardized in
Sections 2.3.3 and 2.3.4 of
Standards for Efficient Cryptography, SEC 1: Elliptic Curve
Cryptography, Certicom Research, Version 2, 2009,
https://www.secg.org/sec1-v2.pdf
In this commit, we modify the existing instructions to create the Sphinx
packet to no longer start out with a zero initialize set of 1366 bytes.
Instead, we now instruct the sender to use _random_ bytes derived from a
CSPRG. This fixes a recently discovered privacy leak that allows an
adversarial exit hop to ascertain a lower bound on the true path length.
Note that this doesn't affect packet processing, so this is a backwards
compatible change. Only clients need to update in order to avoid this
privacy leak.
After this change is applied, the test vectors as is don't match the
spec, as they're created using the original all zero starting bytes. We
can either update these with our specified set of random bytes, or leave
them as is, as they're fully deterministic as is.
An alternative path would be to generate more random bytes from the
shared secret as we do elsewhere (the chacha based CSPRNG).
As reading of commit 6729755f shows, `final_expiry_too_soon` was
17, not PERM|17.
Note that because we folded a previously non-permanent failure into
the now-permanent PERM|15 failure code, modifications to payment
algorithms may now be needed to specificalyl detect this case,
otherwise payment algorithms may give up in some edge cases where
blocks are mined while payments are in-transit between sender and
receiver.
This means the BOLT11 invoice must offer it (we already say it must
set the field if it offers it), and that the receiving node must
require it (again, we already say it must check it if it requires it).
Without the payment_secret, MPP payments are especially vulnerable to
probing attacks: unlike normal payments (with amounts) they can be
detected with 1msat payment probes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I recently made a cut & paste bug with the protocol tests, and
paid an HTLC of amount 100M msat, but with only a 1M msat `amt_to_forward`
in the hop_data. To my surprise, it was accepted.
This is because we allow overpaying the routing fee (considered 0
for the final hop). This doesn't make sense for the final hop: anything
but exact equality implies a bug, or that the previous node took the
wrong amount from the payment.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The specification currently doesn't specify the case where the onion per-hop
payload can't be correctly decoded.
This is somewhat fine with the fixed frames because every field of the payload
can always be interpreted as a numeric value from the input bytes, so it leads
to application errors in upper layers when those values are actually
interpreted (and we realize that for instance we have an invalid
short_channel_id` value).
With variable-length tlv streams in the onion payloads, we will encounter
decoding errors (duplicate tlv types, invalid ordering, etc) and the spec
should define the failure code to use in that case.
In commit 914ebab908 the
incorrect_payment_amount error was merged into
incorrect_or_unknown_payment_details to prevent a probing attack
that allowed intermediate nodes to learn the final destination of
a payment.
A similar attack is possible using the htlc expiry value. By trying
payments with the correct amount but low expiry values to candidate
destinations, an incorrect_or_unknown_payment_details error can be
elicited. This would signal that the probe payment was sent to the
final destination.
For the intermediate node to determine the correct amount, an estimate
must be calculated. A logical choice would be the outgoing amount of the
intermediate node plus some allowance for routing fees that would
otherwise be paid to subsequent nodes along the path.
Picking a low enough - but not too low - expiry value is more tricky.
A reasonable guess could be made using external knowledge of the
final destination's implementation defaults or the type of invoice that
is paid to. Especially in the case of an hodl invoice that typically has
a large expiry delta, it is easier to make a correct guess.
This form of attack is arguably harder to realize than the amount probe
that was previously possible. The attacker may accidentally
pay the invoice if the expiry value guess satisfies the invoice
final cltv requirement. In that case, the attacker still has the
incoming htlc to pull which limits the loss.
As discussed during the IRC meeting on 2019-07-22 this would have been a
duplication of signals. It was decided to use one for now, with the option of
coming back should we ever need the last 32 bytes of the onion.
As discussed during the spec meeting this allows us not to use the 32 byte
HMAC to identify the last hop, and use a 2-byte signal instead.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
The clarifications were tacked on after the fact, but they should really be
part of the conventions. I also updated the links to use the reference style,
which results in better text flow and makes it easier to read the source.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Mention that `outgoing_cltv_value` has to be equal to
`min_final_cltv_expiry` and `amt_to_forward` has to be equal to
`amount` if the [BOLT #11](11-payment-encoding.md) invoice is used
It's trivial to make types->lengths, but not so much the other way.
The types I used here are the ones I found useful in implementation, and
I think add some clarity, though we can certainly argue about them.
There's no normative changes to the spec in here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
914ebab908 effectively deprecated this, but
left it for "reject if more than 2x expected amount" case.
Leaving it for gross overpayment still leaves an attack on the current
network in practice (all implementations I know of reject grossly excessive
payments), and removing it causes our code to nicely break when regenerating,
since that error code is now not defined anywhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Because the errors are separate, if an intermediate node sees a
payment hash for relay and has several guesses as to the
destination of the payment, they can check their guesses by sending
HTLCs with the same payment hashes first and seeing the error sent
back.
By adding the htlc_msat that the final node received to
unknown_or_incorrect_payment_details, origin nodes can still
identify bad value-relaying peers.
This commit documents the allowance of non-strict
forwarding, permitting forwarding nodes to select
between any available outgoing channel with the peer
that would otherwise be specified by the
short_channel_id in the onion packet.
It also includes recommendations for fee schedules
when using non-strict forwarding, either by using
a uniform fee schedule with a peer or only
considering like-policied channels, to ensure the
channel is truly equivalent in terms of fee revenue
for the forwarder.
