ECDSA signatures in Bitcoin are DER-encoded but public keys are not.
The compressed format for public keys is for example standardized in
Sections 2.3.3 and 2.3.4 of
Standards for Efficient Cryptography, SEC 1: Elliptic Curve
Cryptography, Certicom Research, Version 2, 2009,
https://www.secg.org/sec1-v2.pdf
In this commit, we modify the existing instructions to create the Sphinx
packet to no longer start out with a zero initialize set of 1366 bytes.
Instead, we now instruct the sender to use _random_ bytes derived from a
CSPRG. This fixes a recently discovered privacy leak that allows an
adversarial exit hop to ascertain a lower bound on the true path length.
Note that this doesn't affect packet processing, so this is a backwards
compatible change. Only clients need to update in order to avoid this
privacy leak.
After this change is applied, the test vectors as is don't match the
spec, as they're created using the original all zero starting bytes. We
can either update these with our specified set of random bytes, or leave
them as is, as they're fully deterministic as is.
An alternative path would be to generate more random bytes from the
shared secret as we do elsewhere (the chacha based CSPRNG).
As reading of commit 6729755f shows, `final_expiry_too_soon` was
17, not PERM|17.
Note that because we folded a previously non-permanent failure into
the now-permanent PERM|15 failure code, modifications to payment
algorithms may now be needed to specificalyl detect this case,
otherwise payment algorithms may give up in some edge cases where
blocks are mined while payments are in-transit between sender and
receiver.
This means the BOLT11 invoice must offer it (we already say it must
set the field if it offers it), and that the receiving node must
require it (again, we already say it must check it if it requires it).
Without the payment_secret, MPP payments are especially vulnerable to
probing attacks: unlike normal payments (with amounts) they can be
detected with 1msat payment probes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I recently made a cut & paste bug with the protocol tests, and
paid an HTLC of amount 100M msat, but with only a 1M msat `amt_to_forward`
in the hop_data. To my surprise, it was accepted.
This is because we allow overpaying the routing fee (considered 0
for the final hop). This doesn't make sense for the final hop: anything
but exact equality implies a bug, or that the previous node took the
wrong amount from the payment.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The specification currently doesn't specify the case where the onion per-hop
payload can't be correctly decoded.
This is somewhat fine with the fixed frames because every field of the payload
can always be interpreted as a numeric value from the input bytes, so it leads
to application errors in upper layers when those values are actually
interpreted (and we realize that for instance we have an invalid
short_channel_id` value).
With variable-length tlv streams in the onion payloads, we will encounter
decoding errors (duplicate tlv types, invalid ordering, etc) and the spec
should define the failure code to use in that case.
In commit 914ebab908 the
incorrect_payment_amount error was merged into
incorrect_or_unknown_payment_details to prevent a probing attack
that allowed intermediate nodes to learn the final destination of
a payment.
A similar attack is possible using the htlc expiry value. By trying
payments with the correct amount but low expiry values to candidate
destinations, an incorrect_or_unknown_payment_details error can be
elicited. This would signal that the probe payment was sent to the
final destination.
For the intermediate node to determine the correct amount, an estimate
must be calculated. A logical choice would be the outgoing amount of the
intermediate node plus some allowance for routing fees that would
otherwise be paid to subsequent nodes along the path.
Picking a low enough - but not too low - expiry value is more tricky.
A reasonable guess could be made using external knowledge of the
final destination's implementation defaults or the type of invoice that
is paid to. Especially in the case of an hodl invoice that typically has
a large expiry delta, it is easier to make a correct guess.
This form of attack is arguably harder to realize than the amount probe
that was previously possible. The attacker may accidentally
pay the invoice if the expiry value guess satisfies the invoice
final cltv requirement. In that case, the attacker still has the
incoming htlc to pull which limits the loss.
As discussed during the IRC meeting on 2019-07-22 this would have been a
duplication of signals. It was decided to use one for now, with the option of
coming back should we ever need the last 32 bytes of the onion.
As discussed during the spec meeting this allows us not to use the 32 byte
HMAC to identify the last hop, and use a 2-byte signal instead.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
The clarifications were tacked on after the fact, but they should really be
part of the conventions. I also updated the links to use the reference style,
which results in better text flow and makes it easier to read the source.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Mention that `outgoing_cltv_value` has to be equal to
`min_final_cltv_expiry` and `amt_to_forward` has to be equal to
`amount` if the [BOLT #11](11-payment-encoding.md) invoice is used
It's trivial to make types->lengths, but not so much the other way.
The types I used here are the ones I found useful in implementation, and
I think add some clarity, though we can certainly argue about them.
There's no normative changes to the spec in here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
914ebab908 effectively deprecated this, but
left it for "reject if more than 2x expected amount" case.
Leaving it for gross overpayment still leaves an attack on the current
network in practice (all implementations I know of reject grossly excessive
payments), and removing it causes our code to nicely break when regenerating,
since that error code is now not defined anywhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Because the errors are separate, if an intermediate node sees a
payment hash for relay and has several guesses as to the
destination of the payment, they can check their guesses by sending
HTLCs with the same payment hashes first and seeing the error sent
back.
By adding the htlc_msat that the final node received to
unknown_or_incorrect_payment_details, origin nodes can still
identify bad value-relaying peers.
This commit documents the allowance of non-strict
forwarding, permitting forwarding nodes to select
between any available outgoing channel with the peer
that would otherwise be specified by the
short_channel_id in the onion packet.
It also includes recommendations for fee schedules
when using non-strict forwarding, either by using
a uniform fee schedule with a peer or only
considering like-policied channels, to ensure the
channel is truly equivalent in terms of fee revenue
for the forwarder.
* BOLT 4: update sphinx packet test vector
In this commit, we update the test vector for the final onion packet. In
commit 068b0bccf9, the per-hop payloads
were updated to use 8 byte amounts everywhere. However, the test vectors
were not updated. In 578573f92f the test
vectors were updated to use the proper version prefix. However, this
assumed that the state of the vectors as is was correct.
To remedy this, we've updated the test vectors to reflect the final
result using the current format for encoding the per-hop payloads. This
final test vector was generated using the original tool that we used to
confirm compatibility between the C and Go versions.
The description now suggests the use of an ephemeral private key, so
the reference code is simplified by using that concept. The reference
code is also updated to make fewer calls to undefined functions.
The new description introduces the concept of an ephemeral private key,
which I find easier to reason about and suggests a linear instead of
quadratic construction algorithm.
