mirrors/lightning-bolts
1
0
Fork 0
mirror of https://github.com/lightning/bolts.git synced 2024-11-19 10:00:04 +01:00
Code Issues Projects Releases Wiki Activity

BOLT 4: Simplify onion format.

Browse Source 
1. Only one per-hop thing, called `per-hop`, or `hops_data` when in aggregate.
2. Move HMAC to the end of stuff it covers, both of the packet itself, and the per-hop.
3. Use `channel-id` instead of RIPEMD(nodepubkey).
4. Use 4 byte amounts.
5. This is all for realm "0", we can have future realms.  We also have 16
   bytes of unused padding.
6. No longer need the `gamma` key, but document the `_um_` key used for
   errors.
7. Use normal 32-byte HMAC, not truncated 20-bytes, which more than eats
   up the room we saved.

The result is that the onion is now 1366 not 1254 bytes, but simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This commit is contained in:
Rusty Russell 2017-04-04 14:51:06 +09:30
parent 48df730d42
commit 8b29062f78
2 changed files with 57 additions and 97 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
02-peer-protocol.md
View File

@ -575,7 +575,7 @@ is destined, is described in [BOLT #4](04-onion-routing.md).
* [4:amount-msat]
* [4:cltv-expiry]
* [32:payment-hash]
* [1254:onion-routing-packet]
* [1366:onion-routing-packet]
#### Requirements

152
04-onion-routing.md
View File

@ -37,14 +37,7 @@ node and discard the packet.
There are a number of conventions we will adhere to throughout the document:
- The maximum route length is limited to 20 hops.
- Nodes are addressed using 20 byte identifiers. These are computed
from the node's public key, in accordance to the Bitcoin address
creation, as `RIPEMD160(SHA256(pubkey))` where `pubkey` is the
serialized compressed public key of the node. Refer to
[`OP_HASH160`](https://en.bitcoin.it/wiki/Script#Crypto) for
details.
- HMAC: the integrity verification of the packet is based on Keyed-Hash Message Authentication Code as defined by the [FIPS 198 Standard](http://csrc.nist.gov/publications/fips/fips198-1/FIPS-198-1_final.pdf)/[RFC 2104](https://tools.ietf.org/html/rfc2104), using `SHA256` as hashing algorithm.
The resulting HMAC is then truncated at 20 bytes in order to reduce the overhead.
- Elliptic Curve: for all computations involving elliptic curves, the
Bitcoin curve, [`secp256k1`](http://www.secg.org/sec2-v2.pdf), is used.
- Pseudo-Random Stream: [`ChaCha20`](https://tools.ietf.org/html/rfc7539) is used to generate a pseudo-random byte stream.
@ -57,16 +50,16 @@ There are a number of conventions we will adhere to throughout the document:
A number of encryption and verification keys are derived from the shared secret:
- _rho_: used as key when generating the pseudo-random byte stream
used to obfuscate the routing information.
- _gamma_: used as key when generating the pseudo-random byte stream used to obfuscate the per-hop payloads.
used to obfuscate the per-hop information.
- _mu_: used during the HMAC generation.
- _um_: used during error reporting.
The key generation takes a key-type (_rho_=`0x72686F`, _gamma_=`0x67616d6d61` or _mu_=`0x6d75`) and a 32 byte secret as inputs and returns a 32 byte key.
The key generation takes a key-type (_rho_=`0x72686F`, _mu_=`0x6d75` or _um_=`0x756d`) and a 32 byte secret as inputs and returns a 32 byte key.
Keys are generated by computing an HMAC, with `SHA256` as hashing algorithm, using the key-type, i.e., _rho_, _mu_ or _gamma_, as HMAC-key and the 32 byte shared secret as the message.
Keys are generated by computing an HMAC, with `SHA256` as hashing algorithm, using the key-type, i.e., _rho_, _mu_ or _um_, as HMAC-key and the 32 byte shared secret as the message.
The resulting HMAC is then returned as the key.
Notice that the key-type does not include a C-style `0x00` termination-byte, e.g., the length of the _gamma_ key-type is 5 bytes, not 6.
Notice that the key-type does not include a C-style `0x00` termination-byte, e.g., the length of the _rho_ key-type is 3 bytes, not 4.
## Pseudo Random Byte Stream
@ -76,81 +69,59 @@ The use of a fixed nonce is safe since the keys are never reused.
## Packet Structure
The packet consists of 2 parts:
The packet consists of 4 parts:
- The fixed size _header_ containing meta information about the
packet and the routing information necessary to forward the
message.
- A fixed size _per-hop payload_ containing information for each hop
- A `version` byte
- A 33-byte compressed `secp256k1` `public key`, used during the shared secret generation
- A 1300-byte `hops_data` consisting of twenty fixed size packets containing information for each hop
as they forward the message.
- A 32-byte `HMAC` used to verify the packet's integrity.
The overall structure of the packet is depicted below. The network format of the packet consists of the individual parts being serialized into one continguous byte-stream and then transferred to the recipient of the packet. Due to the fixed size of the packet it does not need to be prefixed by its length when transferred over a connection.
~~~~
+--------+-----------------+
| header | per-hop payload |
+--------+-----------------+
+------------------+-----------------------+------------------------+-----------------+
| version (1 byte) | public key (33 bytes) | hops_data (20x65 bytes) | HMAC (32 bytes) |
+------------------+-----------------------+------------------------+-----------------+
~~~~
The header is a fixed 854 byte array containing the necessary information for each hop to identify the next hop, and verify the integrity of the packet.
It consists of a version byte, a 33 byte compressed `secp256k1` public key, used during the shared secret generation, a 20 byte HMAC used to verify the packet's integrity and an 800 byte routing information field.
For this specification the version byte has a constant value of `0x00`.
~~~~
+------------------+-----------------------+-----------------+------------...-----------+
| Version (1 byte) | Public Key (33 bytes) | HMAC (20 bytes) | Routing Info (800 bytes) |
+------------------+-----------------------+-----------------+------------...-----------+
~~~~
The routing info field is a structure that holds obfuscated versions of the next hop's address and the associated HMAC.
It is 800 bytes long, i.e., 20 byte MAC and 20 byte address times 20 hops, and has the following structure:
The `hops_data` field is a structure that holds obfuscated versions of the next hop's address, transfer information and the associated HMAC. It is 1300 bytes long, and has the following structure:
~~~~
+-------------+----------+-------------+----------+-----+--------+
| n_1 address | n_1 HMAC | n_2 address | n_2 HMAC | ... | filler |
+-------------+----------+-------------+----------+-----+--------+
+-----------+-------------+----------+-----------+-------------+----------+-----------+
| n_1 realm | n_1 per-hop | n_1 HMAC | n_2 realm | n_2 per-hop | n_2 HMAC | ...filler |
+-----------+-------------+----------+----------+--------------+----------+------------+
~~~~
Where the `filler` consists of obfuscated deterministically generated
Where `per-hop` is 32 bytes whose contents depend on `realm`, and `filler` consists of obfuscated deterministically generated
padding. For details about how the `filler` is generated please see
below. In addition, every _(address, HMAC)_-pair is incrementally
obfuscated at each hop.
The per-hop payloads has a similar structure:
The `realm` byte determines the format of the `per-hop`; so far
only `realm` 0 is defined, and for that, the `per-hop` format is:
~~~~
+-------------+-------------+-------------+-----+------------+
| n_1 payload | n_2 payload | n_3 payload | ... | hop filler |
+-------------+-------------+-------------+-----+------------+
+----------------------+--------------------------+-------------------------------+--------------------+
| channel_id (8 bytes) | amt_to_forward (4 bytes) | outgoing_cltv_value (4 bytes) | padding (16 bytes) |
+----------------------+--------------------------+-------------------------------+--------------------+
~~~~
With the `hopfiller` being constructed in the same way as the routing
info `filler` and each payload being incrementally obfuscated at each
hop.
### Per Hop Payload Format
Using the per-hop-payload, the sender is able to precisely specify the path and
structure of the HTLC's forwarded at each hop. As the per-hop-payload is
protected under the packet-wide HMAC, the information within the
per-hop-payload is fully authenticated with each pair-wise relationship between
Using the `per-hop`, the sender is able to precisely specify the path and
structure of the HTLC's forwarded at each hop. As the `per-hop` is
protected under the packet-wide HMAC, the information within
is fully authenticated with each pair-wise relationship between
the HTLC sender, and each intermediate node in the path. Using this end-to-end
authentication, forwarding nodes are able to ensure that the incoming node
didn't forward an ill-crafted HTLC by cross-checking the HTLC parameters with
the values as specified within the per-hop-payload.
The format of the per-hop-payload for a version 0 packet is as follows:
```
+----------------+--------------------------+-------------------------------+--------------------------------------------+
| realm (1 byte) | amt_to_forward (8 bytes) | outgoing_cltv_value (4 bytes) | unused_with_v0_version_on_header (7 bytes) |
+----------------+--------------------------+-------------------------------+--------------------------------------------+
```
the values as specified within the `per-hop`.
Field Description:
* `realm` - The realm (chain) link that the outgoing HTLC should be forwarded
on. Within a version 0 packet, this field currently carries no meaning as
cross-chain forwarding has not yet been fully specified.
* `channel_id` - The channel used to route the message, which implies the
next hop is the other end of the channel.
* `amt_to_forward` - The amount in milli-satoshi to forward to the next
(outgoing) hop specified within the routing information.
@ -183,9 +154,11 @@ Field Description:
`outgoing_cltv_value` whether it is the final hop or not, to avoid
leaking that information.
* `padding` - for future use, and also to ensure that future non-0-realm
`per-hop` won't change the overall `hops_data` size.
Nodes forwarding HTLC's MUST construct the outgoing HTLC as specified within
the per-hop-payload. Otherwise, deviation from the specified HTLC parameters
`per-hop`. Otherwise, deviation from the specified HTLC parameters
may lead to extraneous routing failure.
## Packet Construction
@ -215,41 +188,35 @@ This recursive algorithm is initialized by setting the first hop's (`k=1`) ephem
The sender then iteratively computes the ephemeral public keys, shared secrets and blinding factors for nodes `{n_2, ..., n_r}`.
Once the sender has all the required information it can construct the packet.
Constructing a packet routed over `r` hops requires `r` 32 byte ephemeral public keys, `r` 32 byte shared secrets, `r` 32 byte blinding factors and `r` 20 byte per-hop payloads.
The construction returns one 1254 byte packet and the first hop's address.
Constructing a packet routed over `r` hops requires `r` 32 byte ephemeral public keys, `r` 32 byte shared secrets, `r` 32 byte blinding factors and `r` 65 byte `per-hop` payloads.
The construction returns one 1366 byte packet and the first hop's address.
The packet construction is performed in reverse order of the route, i.e., the last hop's operations are applied first.
The per-hop payload is initialized with 400 `0x00` bytes.
The routing info is initialized with 800 `0x00` bytes.
The next address and the HMAC are initialized to 20 `0x00` bytes each.
The packet is initialized with 1366 `0x00` bytes.
Two fillers are generated with the shared secrets: a routing info filler with 40 byte hopsize and a per-hop payload filler with 20 byte hopsize.
A 65-byte filler is generated with the shared secret:
See below for details on filler generation.
For each hop in the route in reverse order the sender applies the
following operations:
- It generates a _rho_-key, _mu_-key and a _gamma_-key using the hop's shared secret.
- The routing info field is right-shifted by 40 bytes, discarding the last 40 bytes that exceed the 800 bytes.
The address is copied into the first 20 bytes of the routing info and the HMAC is copied into
the following 20 bytes.
The _rho_-key is used to generate 800 bytes of pseudo-random byte stream and applied with `XOR` to the routing info field.
Should this be the last hop, i.e., the first iteration, then the tail of the routing info field is overwritten with the routing info `filler`.
- The per-hop payload field is right-shifted by 20 bytes, and the last 20 bytes discarded, resulting in 400 bytes of per-hop payload.
The current hop's per-hop payload is copied into the first 20 bytes.
The _gamma_-key is used to generate 400 bytes of pseudo-random byte stream which are then applied using `XOR` to the per-hop payloads field.
Should this be the last hop then the tail of the per-hop payloads field is overwritten with the per-hop payload filler.
- The next HMAC is computed over the concatenated routing info, per-hop payload and associated data, with the _mu_-key as HMAC-key.
- The next address is computed from the current node's public key using the Bitcoin address hash derivation.
- It generates a _rho_-key and _mu_-key using the hop's shared secret.
- The `hops_data` field is right-shifted by 65 bytes, discarding the last 65 bytes that exceed the 1300 bytes.
The `version`, `channel_id`, `amt_to_forward`, `outgoing_cltv_value`, `padding` and `HMAC` are copied into
the following 65 bytes.
The _rho_-key is used to generate 1300 bytes of pseudo-random byte stream and applied with `XOR` to the `hops_data` field.
Should this be the last hop, i.e., the first iteration, then the tail of the `hops_data` field is overwritten with the routing info `filler`.
- The next HMAC is computed over the concatenated `hops_data` and associated data, with the _mu_-key as HMAC-key.
The final value for the HMAC is the HMAC as it should be sent to the first hop.
The packet generation returns the serialized packet, consisting of the version byte, the ephemeral pubkey for the first hop, the HMAC for the first hop, the obfuscated routing info and the obfuscated per-hop payload.
The packet generation returns the serialized packet, consisting of the version byte, the ephemeral pubkey for the first hop, the HMAC for the first hop, the obfuscated `hops_data`.
The following code implements the packet construction in Go:
```Go
FIXME
func ConstructPacket(paymentPath []*btcec.PublicKey, sessionKey *btcec.PrivateKey,
rawHopPayloads [][]byte) ([]byte) {
numHops := len(paymentPath)
@ -335,7 +302,7 @@ The node then computes the shared secret as described below, using the private k
The node MUST detect a duplicated routing info which it has already forwarded or redeemed locally; it MAY immediately redeem the HTLC using the preimage (if known), otherwise it MUST abort processing and report a route failure. This prevents a node on the route from retrying a payment multiple times and attempting to track its progress by traffic analysis. Note that this could be done using a log of previous shared secrets or HMACs, which can be forgotten once that HTLC would not be accepted anyway (eg. once `outgoing_cltv_value` has passed). Such a log may use a probabilistic data structure, but MUST rate-limit commitments as necessary to constrain the worst-case storage requirements or false positives of this log.
The shared secret is used to compute a _mu_-key. The node then computes the HMAC of the packet, starting from byte 54, which corresponds to the routing info, per-hop payloads and associated data, using the _mu_-key.
The shared secret is used to compute a _mu_-key. The node then computes the HMAC of the `hops_data` using the _mu_-key.
The resulting HMAC is compared with the HMAC from the packet.
Should the computed HMAC and the HMAC from the packet differ then the node MUST abort processing and report a route failure.
Comparison of the computed HMAC and the HMAC from the packet MUST be time-constant to avoid leaking information.
@ -343,18 +310,14 @@ Comparison of the computed HMAC and the HMAC from the packet MUST be time-consta
At this point the node can generate a _rho_-key and a _gamma_-key.
The routing info is deobfuscated and the information about the next hop is extracted.
In order to do so the node copies the routing info field, appends 40 `0x00` bytes and generates 840 pseudo-random bytes using the _rho_-key and applies it using `XOR` to the copy of the routing information.
The first 20 bytes of the resulting routing info are the address of the next hop, followed by the 20 byte HMAC.
The routing info for the outgoing packet, destined for the next hop, consists of the 800 bytes starting at byte 40.
In order to do so the node copies the `hops_data` field, appends 65 `0x00` bytes and generates 1365 pseudo-random bytes using the _rho_-key and applies it using `XOR` to the copy of the `hops_data`
The first 65 bytes of the resulting routing info are `per-hop` field for the next hop. The next 1300 bytes is the `hops_data` for the outgoing packet.
The per-hop payload is deobfuscated in a similar way.
The node creates a copy of the per-hop payloads field and appends 20 `0x00` bytes of padding.
It generates 420 bytes of pseudo-random bytes using the _gamma_-key and applies it using `XOR` to the padded copy of the per-hop payloads.
The first 20 bytes of the padded copy are the node's per-hop payload, while the remaining 400 bytes are the per-hop payload destined for the next hop.
If the `realm` is unknown, then the node MUST drop the packet and signal a route failure.
A special HMAC value of 20 `0x00` bytes indicates that the currently processing hop is the intended recipient and that the packet should not be forwarded.
A special `per-hop` `HMAC` value of 32 `0x00` bytes indicates that the currently processing hop is the intended recipient and that the packet should not be forwarded.
Should the HMAC not indicate route termination and the next hop be a peer of the current node, then the new packet is assembled by blinding the ephemeral key with the current node's public key and shared secret, and serializing the routing info and per-hop payload.
Should the HMAC not indicate route termination and the next hop be a peer of the current node, then the new packet is assembled by blinding the ephemeral key with the current node's public key and shared secret, and serializing the `hops_data`.
The resulting packet is then forwarded to the addressed peer.
The addressed peer MUST be a direct neighbor of the node processing the packet.
Should the processing node not have a peer with the matching address, then it MUST drop the packet and signal a route failure.
@ -383,15 +346,12 @@ For this reason the field is padded before forwarding.
Since the padding is part of the HMAC the sender will have to generate an identical padding in order to compute the HMACs correctly for each hop.
The filler is also used to pad the field-length in case the selected route is shorter than the maximum allowed route length.
We call the number of bytes extracted from the field _hopsize_.
In case of the route info the hopsize is 40 bytes (20 bytes address and 20 bytes HMAC), while in the case of the per-hop payload it is 20 bytes.
Before deobfuscating the field, the node pads the field with hopsize `0x00` bytes, such that the total length of the field is `(20 + 1) * hopsize`.
It then generates the pseudo-random byte stream of matching length and applies it with `XOR` to the field.
Before deobfuscating the `hops_data`, the node pads it with 65 `0x00` bytes, such that the total length is `(20 + 1) * 65`.
It then generates the pseudo-random byte stream of matching length and applies it with `XOR` to the `hops_data`.
This deobfuscates the information destined for it, and simultaneously obfuscates the added
`0x00`-bytes at the end.
In order to compute the correct HMAC, the sender has to generate the field's state at the hop.
In order to compute the correct HMAC, the sender has to generate the `hops_data` at the hop.
This also includes the incrementally obfuscated padding added by each hop.
The incrementally obfuscated padding is called the _filler_.
@ -448,7 +408,7 @@ In addition each node locally stores the previous hop it received the forward pa
The node returning the message builds a return packet consisting of the following fields:
1. data:
* [20:hmac]
* [32:hmac]
* [2:failure-len]
* [failure-len:failuremsg]
* [2:pad-len]