* Rename all the 'varint' to 'bigsize'.
Having both is confusing; we chose the name bigsize, so use it
explicitly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* BOLT 7: use `byte` instead of `u8`.
`u8` isn't a type; see BOLT #1 "Fundamental Types".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* BOLT 1: promote bigsize to a Fundamental Type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ECDSA signatures in Bitcoin are DER-encoded but public keys are not.
The compressed format for public keys is for example standardized in
Sections 2.3.3 and 2.3.4 of
Standards for Efficient Cryptography, SEC 1: Elliptic Curve
Cryptography, Certicom Research, Version 2, 2009,
https://www.secg.org/sec1-v2.pdf
In this commit, we modify the existing instructions to create the Sphinx
packet to no longer start out with a zero initialize set of 1366 bytes.
Instead, we now instruct the sender to use _random_ bytes derived from a
CSPRG. This fixes a recently discovered privacy leak that allows an
adversarial exit hop to ascertain a lower bound on the true path length.
Note that this doesn't affect packet processing, so this is a backwards
compatible change. Only clients need to update in order to avoid this
privacy leak.
After this change is applied, the test vectors as is don't match the
spec, as they're created using the original all zero starting bytes. We
can either update these with our specified set of random bytes, or leave
them as is, as they're fully deterministic as is.
An alternative path would be to generate more random bytes from the
shared secret as we do elsewhere (the chacha based CSPRNG).
As reading of commit 6729755f shows, `final_expiry_too_soon` was
17, not PERM|17.
Note that because we folded a previously non-permanent failure into
the now-permanent PERM|15 failure code, modifications to payment
algorithms may now be needed to specificalyl detect this case,
otherwise payment algorithms may give up in some edge cases where
blocks are mined while payments are in-transit between sender and
receiver.
This means the BOLT11 invoice must offer it (we already say it must
set the field if it offers it), and that the receiving node must
require it (again, we already say it must check it if it requires it).
Without the payment_secret, MPP payments are especially vulnerable to
probing attacks: unlike normal payments (with amounts) they can be
detected with 1msat payment probes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I recently made a cut & paste bug with the protocol tests, and
paid an HTLC of amount 100M msat, but with only a 1M msat `amt_to_forward`
in the hop_data. To my surprise, it was accepted.
This is because we allow overpaying the routing fee (considered 0
for the final hop). This doesn't make sense for the final hop: anything
but exact equality implies a bug, or that the previous node took the
wrong amount from the payment.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The specification currently doesn't specify the case where the onion per-hop
payload can't be correctly decoded.
This is somewhat fine with the fixed frames because every field of the payload
can always be interpreted as a numeric value from the input bytes, so it leads
to application errors in upper layers when those values are actually
interpreted (and we realize that for instance we have an invalid
short_channel_id` value).
With variable-length tlv streams in the onion payloads, we will encounter
decoding errors (duplicate tlv types, invalid ordering, etc) and the spec
should define the failure code to use in that case.
In commit 914ebab908 the
incorrect_payment_amount error was merged into
incorrect_or_unknown_payment_details to prevent a probing attack
that allowed intermediate nodes to learn the final destination of
a payment.
A similar attack is possible using the htlc expiry value. By trying
payments with the correct amount but low expiry values to candidate
destinations, an incorrect_or_unknown_payment_details error can be
elicited. This would signal that the probe payment was sent to the
final destination.
For the intermediate node to determine the correct amount, an estimate
must be calculated. A logical choice would be the outgoing amount of the
intermediate node plus some allowance for routing fees that would
otherwise be paid to subsequent nodes along the path.
Picking a low enough - but not too low - expiry value is more tricky.
A reasonable guess could be made using external knowledge of the
final destination's implementation defaults or the type of invoice that
is paid to. Especially in the case of an hodl invoice that typically has
a large expiry delta, it is easier to make a correct guess.
This form of attack is arguably harder to realize than the amount probe
that was previously possible. The attacker may accidentally
pay the invoice if the expiry value guess satisfies the invoice
final cltv requirement. In that case, the attacker still has the
incoming htlc to pull which limits the loss.
As discussed during the IRC meeting on 2019-07-22 this would have been a
duplication of signals. It was decided to use one for now, with the option of
coming back should we ever need the last 32 bytes of the onion.
As discussed during the spec meeting this allows us not to use the 32 byte
HMAC to identify the last hop, and use a 2-byte signal instead.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
The clarifications were tacked on after the fact, but they should really be
part of the conventions. I also updated the links to use the reference style,
which results in better text flow and makes it easier to read the source.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Mention that `outgoing_cltv_value` has to be equal to
`min_final_cltv_expiry` and `amt_to_forward` has to be equal to
`amount` if the [BOLT #11](11-payment-encoding.md) invoice is used
It's trivial to make types->lengths, but not so much the other way.
The types I used here are the ones I found useful in implementation, and
I think add some clarity, though we can certainly argue about them.
There's no normative changes to the spec in here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
914ebab908 effectively deprecated this, but
left it for "reject if more than 2x expected amount" case.
Leaving it for gross overpayment still leaves an attack on the current
network in practice (all implementations I know of reject grossly excessive
payments), and removing it causes our code to nicely break when regenerating,
since that error code is now not defined anywhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Because the errors are separate, if an intermediate node sees a
payment hash for relay and has several guesses as to the
destination of the payment, they can check their guesses by sending
HTLCs with the same payment hashes first and seeing the error sent
back.
By adding the htlc_msat that the final node received to
unknown_or_incorrect_payment_details, origin nodes can still
identify bad value-relaying peers.
This commit documents the allowance of non-strict
forwarding, permitting forwarding nodes to select
between any available outgoing channel with the peer
that would otherwise be specified by the
short_channel_id in the onion packet.
It also includes recommendations for fee schedules
when using non-strict forwarding, either by using
a uniform fee schedule with a peer or only
considering like-policied channels, to ensure the
channel is truly equivalent in terms of fee revenue
for the forwarder.
* BOLT 4: update sphinx packet test vector
In this commit, we update the test vector for the final onion packet. In
commit 068b0bccf9, the per-hop payloads
were updated to use 8 byte amounts everywhere. However, the test vectors
were not updated. In 578573f92f the test
vectors were updated to use the proper version prefix. However, this
assumed that the state of the vectors as is was correct.
To remedy this, we've updated the test vectors to reflect the final
result using the current format for encoding the per-hop payloads. This
final test vector was generated using the original tool that we used to
confirm compatibility between the C and Go versions.
The description now suggests the use of an ephemeral private key, so
the reference code is simplified by using that concept. The reference
code is also updated to make fewer calls to undefined functions.
The new description introduces the concept of an ephemeral private key,
which I find easier to reason about and suggests a linear instead of
quadratic construction algorithm.
second pass copy edit to line 253, according to stylesheet
update node terminology to remove ambiguity; update conventions section and implement consistent usage of terms: origin node, final node, processing node, hop, sending peer, and receiving peer
Complete rewrite, including a routing example and the new
min_final_cltv expirt. I hope this makes it clear.
(Thanks to everyone who reviewed and gave feedback; you rock!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a partial response to #250. Reordering the HMAC and Encrypt
steps do not give us much, but we might want to hide the route
length. So we suggest that the node should continue unwrapping until
the maximum route length of 20 is reached.
* BOLT 04: increase max size of onion payload messages
This commit increases the max size of the encapsulated onion error
messages. This is a follow up change to the recent change that added a
`chain_hash` field to the `channel_update` message. With the addition of
this field, the largest payload encoded within the onion errors has
expanded to 138 bytes:
* msat_amount || 2_byte_len || channel_update.
As a result, the old fixed limit (including padding) is now
insufficient. We use 256 bytes here in order to give us room for future
message expansions.
We reuse the numeric values that we previously assigned to message
types in the failure_code, but there is no possibility for a mixup
since the latter is not transmitted directly on the transport layer
but wrapped in a return packet. Hence there is no way of confusing the
two. Added a short clarification.
Reported-by: Janus Troelsen @ysangkok
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Consistency with BOLT 7 makes this much clearer.
Closes: #195
Reported-by: https://github.com/nayuta-ueno
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We had 4 byte fields for amounts because people have no ability to assess
risk, and this limited the damage to $70 at a time.
But then that means $1 maximum HTLCs on Litecoin, which isn't enough
for a cup of (decent) coffee.
Rather than have boutique hacks for Litecoin we enlarge the fields now,
and simply have a bitcoin-specific restriction that the upper 4 bytes be 0.
The ctlv_expiry field is moved down in update_add_htlc, to preserve alignment.
Suggested-by: Olaoluwa Osuntokun <laolu32@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Specify the payload for the last node in the route and how it is used to return
errors. The idea is to prevent the next to last node to guess if the next node is
the final one.
This also converts data structures to the same format used elsewhere.
One other minor change, from:
In addition, every _(address, HMAC)_-pair is incrementally obfuscated at each hop.
to:
In addition, `hops_data` is incrementally obfuscated at each hop.
The old wording was left over from the previous format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a multi-part message in MIME format.
This patch should apply to http://github.com/lightningnetwork/lightning-rfc
Nonidealities:
Aspell triggers spelling errors on the hexadecimal strings in
the test vectors. I don't have enough aspell-fu to figure
out how to make Aspell ignore these.
There are 2 possible pluralizations of `HTLC`: `HTLCs` and
`HTLC's`. I'd prefer the latter, but for now I support both.
We should standardize pluralization; we can edit the
`.aspell.en.pws` file to remove the pluralization we won't
choose.
These test vectors should match BOLT04 after the change to merge
per-hop payloads and routing info into a single `hop_data` field. They
were generated by the golang version and crosschecked with the
`lightningd` version.
The per-hop `hop_data` were changed to be initialized by byte-filling
the `short_channel_id` matching their position in the route, and by
setting the `amt_to_forward` and `outgoing_cltv` fields to the same
value, i.e., for hop 3 the values are:
short_channel_id = 0x0303030303030303
amt_to_forward = 0x0000003
outgoing_cltv = 0x0000003
1. Only one per-hop thing, called `per-hop`, or `hops_data` when in aggregate.
2. Move HMAC to the end of stuff it covers, both of the packet itself, and the per-hop.
3. Use `channel-id` instead of RIPEMD(nodepubkey).
4. Use 4 byte amounts.
5. This is all for realm "0", we can have future realms. We also have 16
bytes of unused padding.
6. No longer need the `gamma` key, but document the `_um_` key used for
errors.
7. Use normal 32-byte HMAC, not truncated 20-bytes, which more than eats
up the room we saved.
The result is that the onion is now 1366 not 1254 bytes, but simpler.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We didn't note the actual requirements: we MUST reject replays we have forwarded
or paid to avoid replay attacks. The details are difficult however; we have
to clean them out at some stage, and restrict the size somehow. Suggest some
ways we could do that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is particularly important if people start overpaying: a hop
may try to deduct 1 extra millisatoshi, which would be rejected by the
next unless the next is the final hop, enabling detection.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Not doing this check means an inconsistency in behaviour, which could
theoretically allow a hop to probe: if the next hop is the last, it might
not care, but if it's not it will return an error.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The idea of the "SHOULD fail if amount is too much" was courtesy against
overpaying, but that's a bad idea of you value privacy and some vendor has
well-known prices. Allow a factor of two, at least.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
