2016-11-21 06:48:12 +01:00
# BOLT #7: P2P Node and Channel Discovery
2017-12-07 00:49:29 +01:00
This specification describes simple node discovery, channel discovery, and channel update mechanisms that do not rely on a third-party to disseminate the information.
2016-11-21 06:48:12 +01:00
Node and channel discovery serve two different purposes:
2018-01-03 04:21:36 +01:00
- Channel discovery allows the creation and maintenance of a local view of the network's topology, so that a node can discover routes to desired destinations.
- Node discovery allows nodes to broadcast their ID, host, and port, so that other nodes can open connections and establish payment channels with them.
2017-12-07 00:49:29 +01:00
To support channel discovery, peers in the network exchange
2018-01-03 04:21:36 +01:00
`channel_announcement` messages containing information regarding new
channels between the two nodes. They can also exchange `channel_update`
2017-12-07 00:49:29 +01:00
messages, which update information about a channel. There can only be
one valid `channel_announcement` for any channel, but at least two
`channel_update` messages are expected.
2016-11-21 06:48:12 +01:00
2017-12-07 00:49:29 +01:00
To support node discovery, peers exchange `node_announcement`
2018-01-03 04:21:36 +01:00
messages, which supply additional information about the nodes. There may be
multiple `node_announcement` messages, in order to update the node information.
2017-12-07 00:49:29 +01:00
2017-12-08 19:15:48 +01:00
# Table of Contents
* [The `announcement_signatures` Message ](#the-announcement_signatures-message )
* [The `channel_announcement` Message ](#the-channel_announcement-message )
* [The `node_announcement` Message ](#the-node_announcement-message )
* [The `channel_update` Message ](#the-channel_update-message )
* [Initial Sync ](#initial-sync )
* [Rebroadcasting ](#rebroadcasting )
* [HTLC Fees ](#htlc-fees )
* [Pruning the Network View ](#pruning-the-network-view )
* [Recommendations for Routing ](#recommendations-for-routing )
* [References ](#references )
2017-12-07 00:49:29 +01:00
## The `announcement_signatures` Message
2016-11-21 06:48:12 +01:00
2018-01-03 04:21:36 +01:00
This is a direct message between the two endpoints of a channel and serves as an opt-in mechanism to allow the announcement of the channel to the rest of the network.
It contains the necessary signatures, by the sender, to construct the `channel_announcement` message.
2017-02-07 01:53:39 +01:00
1. type: 259 (`announcement_signatures`)
2. data:
2017-05-11 03:46:05 +02:00
* [`32`:`channel_id`]
* [`8`:`short_channel_id`]
* [`64`:`node_signature`]
* [`64`:`bitcoin_signature`]
2017-02-07 01:53:39 +01:00
2017-06-27 04:44:29 +02:00
The willingness of the initiating node to announce the channel is signaled during channel opening by setting the `announce_channel` bit in `channel_flags` (see [BOLT #2 ](02-peer-protocol.md#the-open_channel-message )).
BOLT 0,1,2,7: use txout not channel-id for demuxing. (#119)
At cost of a few extra bytes between peers, this avoids the whole "oops, we were on a chain fork" problem, and simplifies generation of temporary channel-ids (just pick a random one).
Now we move the announcement_signature exchange to at least 6 confirms, which makes re-xmit tricky; I resolved that by insisting on reconnect that we send if we haven't received, and reply to the first one.
The term "channel shortid" wasn't used anywhere, so I removed it; it's now a gossip-only thing anyway.
One subtle change: pkt_error on unknown channels is now "MUST ignore"; this section was reworked anyway, and we'll want this if the #120 goes through, where one side might have forgotten unformed channels).
Closes: #114
Suggested-by: Olaoluwa Osuntokun <laolu32@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* FIXUP! Two bytes for funding-output-index.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* FIXUP! Channel-id rework, temp ids, 32 bits only.
Re-add the idea of temporary channel ids: far simpler since they're now
big enough we can just fill with noise.
Remove the alignment issues by combining txid and outnum using XOR; we
could reduce to 128 bit if we really wanted to, but we don't.
Error handling is now simple again, but while editing I changed the
behaviour for unknown channels to MUST ignore (this is important for
Change the 8-byte gossip channel id to `short-channel-id`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* FIXUP! Minor text tweaks from Pierre-Marie and Christian
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-02 05:20:13 +01:00
### Requirements
2018-01-03 04:21:36 +01:00
The `announcement_signatures` message is created by constructing a `channel_announcement` message, corresponding to the newly established channel, and signing it with the secrets matching an endpoint's `node_id` and `bitcoin_key` . The message is then sent using an `announcement_signatures` message.
BOLT 0,1,2,7: use txout not channel-id for demuxing. (#119)
At cost of a few extra bytes between peers, this avoids the whole "oops, we were on a chain fork" problem, and simplifies generation of temporary channel-ids (just pick a random one).
Now we move the announcement_signature exchange to at least 6 confirms, which makes re-xmit tricky; I resolved that by insisting on reconnect that we send if we haven't received, and reply to the first one.
The term "channel shortid" wasn't used anywhere, so I removed it; it's now a gossip-only thing anyway.
One subtle change: pkt_error on unknown channels is now "MUST ignore"; this section was reworked anyway, and we'll want this if the #120 goes through, where one side might have forgotten unformed channels).
Closes: #114
Suggested-by: Olaoluwa Osuntokun <laolu32@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* FIXUP! Two bytes for funding-output-index.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* FIXUP! Channel-id rework, temp ids, 32 bits only.
Re-add the idea of temporary channel ids: far simpler since they're now
big enough we can just fill with noise.
Remove the alignment issues by combining txid and outnum using XOR; we
could reduce to 128 bit if we really wanted to, but we don't.
Error handling is now simple again, but while editing I changed the
behaviour for unknown channels to MUST ignore (this is important for
Change the 8-byte gossip channel id to `short-channel-id`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* FIXUP! Minor text tweaks from Pierre-Marie and Christian
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-02 05:20:13 +01:00
2017-05-11 03:46:05 +02:00
The `short_channel_id` is the unique description of the funding transaction.
2018-01-03 04:21:36 +01:00
It is constructed as follows:
1. the most significant 3 bytes: indicating the block height
2. the next 3 bytes: indicating the transaction index within the block
3. the least significant 2 bytes: indicating the output index that pays to the channel.
A node:
- if the `open_channel` message has the `announce_channel` bit set:
- MUST send the `announcement_signatures` message.
- MUST NOT send `announcement_signatures` messages until `funding_locked`
has been sent AND the funding transaction has at least six confirmations.
- otherwise:
- MUST NOT send the `announcement_signatures` message.
- upon reconnection:
- MUST respond to the first `announcement_signatures` message with its own
`announcement_signatures` message.
- if it has NOT received an `announcement_signatures` message:
- SHOULD retransmit the `announcement_signatures` message.
A recipient node:
- if the `node_signature` OR the `bitcoin_signature` is NOT correct:
- MAY fail the channel.
- if it has sent AND received a valid `announcement_signatures` message:
- SHOULD queue the `channel_announcement` message for its peers.
2017-02-07 01:53:39 +01:00
2017-12-07 00:49:29 +01:00
## The `channel_announcement` Message
2016-11-21 06:48:12 +01:00
2018-01-03 04:21:36 +01:00
This message contains ownership information regarding a channel. It ties
each on-chain Bitcoin key to the associated Lightning node key, and vice-versa.
The channel is not practically usable until at least one side has announced
its fee levels and expiry, using `channel_update` .
2016-11-21 06:48:12 +01:00
2018-01-03 04:21:36 +01:00
Proving the existence of a channel between `node_1` and `node_2` requires:
2016-12-12 02:27:53 +01:00
2017-12-08 19:15:48 +01:00
1. proving that the funding transaction pays to `bitcoin_key_1` and
`bitcoin_key_2`
2. proving that `node_1` owns `bitcoin_key_1`
3. proving that `node_2` owns `bitcoin_key_2`
2016-12-12 02:27:53 +01:00
2018-01-03 04:21:36 +01:00
Assuming that all nodes know the unspent transaction outputs, the first proof is
accomplished by a node finding the output given by the `short_channel_id` and
verifying that it is indeed a P2WSH funding transaction output for those keys
specified in [BOLT #3 ](03-transactions.md#funding-transaction-output ).
2016-12-12 02:27:53 +01:00
2018-01-03 04:21:36 +01:00
The last two proofs are accomplished through explicit signatures:
`bitcoin_signature_1` and `bitcoin_signature_2` are generated for each
`bitcoin_key` and each of the corresponding `node_id` s are signed.
2016-12-12 02:27:53 +01:00
2018-01-03 04:21:36 +01:00
It's also necessary to prove that `node_1` and `node_2` both agree on the
announcement message: this is accomplished by having a signature from each
`node_id` (`node_signature_1` and `node_signature_2` ) signing the message.
2016-12-12 02:27:53 +01:00
2016-11-23 02:03:25 +01:00
1. type: 256 (`channel_announcement`)
2016-11-21 06:48:12 +01:00
2. data:
2017-05-11 03:46:05 +02:00
* [`64`:`node_signature_1`]
* [`64`:`node_signature_2`]
* [`64`:`bitcoin_signature_1`]
* [`64`:`bitcoin_signature_2`]
BOLT7: Reorder feature bitmaps in order to allow future changes
Appending new fields to the end of the messages allows us to add new
fields to an existing message, however it does not allow removing
existing fields, e.g., dropping the pubkeys like #187 proposes. Moving
the features bitmap at the beginning of the signed payload allows
this type of change in the future. Nodes verify the integrity of the
message and then check whether there are any even bits they don't
implement. These even bits being required features would then result
in the message being discarded.
In addition to what we discussed during the call I also went ahead and
did the same reordering on `node_announcement`, which I think has the
same issue.
There is a subtle change in semantics, i.e., previously we would
add channels with unknown bits to our local view, but then ignore them
when computing a route. Now we no longer add them to our view, and may
discard the announcement altogether, stopping the broadcast. This is
safe I think since otherwise we'd be forwarding things we can only
verify the signatures of, but nothing else.
2017-06-27 13:50:43 +02:00
* [`2`:`len`]
* [`len`:`features`]
2017-07-10 21:46:39 +02:00
* [`32`:`chain_hash`]
2017-05-11 03:46:05 +02:00
* [`8`:`short_channel_id`]
* [`33`:`node_id_1`]
* [`33`:`node_id_2`]
* [`33`:`bitcoin_key_1`]
* [`33`:`bitcoin_key_2`]
2016-11-21 06:48:12 +01:00
### Requirements
2018-01-04 01:23:30 +01:00
The creating node:
- MUST set `chain_hash` to the 32-byte hash that uniquely identifies the chain
that the channel was opened within:
- MUST set the _Bitcoin blockchain_ `chain_hash` value (encoded in hex) to: `000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f` .
- MUST set `short_channel_id` to refer to the confirmed funding transaction,
as specified in [BOLT #2 ](02-peer-protocol.md#the-funding_locked-message ).
- Note: the corresponding output MUST be a P2WSH, as described in [BOLT #3 ](03-transactions.md#funding-transaction-output ).
- MUST set `node_id_1` and `node_id_2` to the public keys of the two nodes
operating the channel, such that `node_id_1` is the numerically-lesser of the
two DER-encoded keys sorted in ascending numerical order.
- MUST set `bitcoin_key_1` and `bitcoin_key_2` to `node_id_1` and `node_id_2` 's
respective `funding_pubkey` s.
- MUST compute the double-SHA256 hash `h` of the message, beginning at offset
256, up to the end of the message.
- Note: the hash skips the 4 signatures but hashes the rest of the message,
including any future fields appended to the end.
- MUST verify that `node_signature_1` and `node_signature_2` are valid
signatures of the hash `h` , using `node_id_1` and `node_id_2` 's respective
secrets.
- MUST verify that `bitcoin_signature_1` and `bitcoin_signature_2` are valid
signatures of the hash `h` , using `bitcoin_key_1` and `bitcoin_key_2` 's
respective secrets.
- SHOULD set `len` to the minimum length required to hold the `features` bits
it sets.
The receiving node:
- MUST verify the integrity and authenticity of the message by verifying the
signatures.
- if there is an unknown even bit in the `features` field:
- MUST NOT parse the remainder of the message.
- MUST NOT add the channel to its local network view.
- SHOULD NOT forward the announcement.
- if the `short_channel_id` 's output does NOT correspond to a P2WSH (using
`bitcoin_key_1` and `bitcoin_key_2` , as specified in
[BOLT #3 ](03-transactions.md#funding-transaction-output )) OR the output is
spent:
- MUST ignore the message.
- if the specified `chain_hash` is unknown to the receiver:
- MUST ignore the message.
- otherwise:
- if `bitcoin_signature_1` , `bitcoin_signature_2` , `node_signature_1` OR
`node_signature_2` are invalid OR NOT correct:
- SHOULD fail the connection.
- otherwise:
- if `node_id_1` or `node_id_2` are blacklisted:
- SHOULD ignore the message.
- otherwise:
- if the transaction referred to was NOT previously announced as a
channel:
- SHOULD queue the message for rebroadcasting.
- MAY choose NOT to for messages longer than the minimum expected
length.
- if it has previously received a valid `channel_announcement` , for the
same transaction, in the same block, but for a different `node_id_1` or
`node_id_2` :
- SHOULD blacklist the previous message's `node_id_1` and `node_id_2` ,
as well as this `node_id_1` and `node_id_2` AND forget any channels
connected to them.
- otherwise:
- SHOULD store this `channel_announcement` .
- once its funding output has been spent or reorganized out:
- SHOULD forget a channel.
2016-11-21 06:48:12 +01:00
2017-01-31 14:40:59 +01:00
### Rationale
2016-11-21 06:48:12 +01:00
2018-01-04 01:23:30 +01:00
Both nodes are required to sign to indicate they are willing to route other
payments via this channel (i.e. be part of the public network); requiring their
Bitcoin signatures proves that they control the channel.
2016-11-21 06:48:12 +01:00
2018-01-04 01:23:30 +01:00
The blacklisting of conflicting nodes disallows multiple different
announcements. Such conflicting announcements should never be broadcast by any
node, as this implies that keys have leaked.
2016-11-21 06:48:12 +01:00
2018-01-04 01:23:30 +01:00
While channels should not be advertised before they are sufficiently deep, the
requirement against rebroadcasting only applies if the transaction has not moved
to a different block.
2016-11-21 06:48:12 +01:00
2018-01-04 01:23:30 +01:00
In order to avoid storing excessively-large messages, yet still allow for
reasonable future expansion, nodes are permitted to restrict rebroadcasting
(perhaps statistically).
2016-12-12 02:11:39 +01:00
2018-01-04 01:23:30 +01:00
New channel features are possible in the future: backwards compatible (or
optional) features will have _odd_ feature bits, while incompatible features
will have _even_ feature bits
(["It's OK to be odd!"](00-introduction.md#glossary-and-terminology-guide)).
Incompatible features will result in the announcement not being forwarded by
nodes that do not understand them.
2016-12-12 02:11:25 +01:00
2017-12-08 19:15:48 +01:00
## The `node_announcement` Message
2016-11-21 06:48:12 +01:00
This allows a node to indicate extra data associated with it, in
2017-12-07 00:49:29 +01:00
addition to its public key. To avoid trivial denial of service attacks,
2016-11-21 06:48:12 +01:00
nodes for which a channel is not already known are ignored.
2016-11-23 02:03:25 +01:00
1. type: 257 (`node_announcement`)
2016-11-21 06:48:12 +01:00
2. data:
2017-05-11 03:46:05 +02:00
* [`64`:`signature`]
BOLT7: Reorder feature bitmaps in order to allow future changes
Appending new fields to the end of the messages allows us to add new
fields to an existing message, however it does not allow removing
existing fields, e.g., dropping the pubkeys like #187 proposes. Moving
the features bitmap at the beginning of the signed payload allows
this type of change in the future. Nodes verify the integrity of the
message and then check whether there are any even bits they don't
implement. These even bits being required features would then result
in the message being discarded.
In addition to what we discussed during the call I also went ahead and
did the same reordering on `node_announcement`, which I think has the
same issue.
There is a subtle change in semantics, i.e., previously we would
add channels with unknown bits to our local view, but then ignore them
when computing a route. Now we no longer add them to our view, and may
discard the announcement altogether, stopping the broadcast. This is
safe I think since otherwise we'd be forwarding things we can only
verify the signatures of, but nothing else.
2017-06-27 13:50:43 +02:00
* [`2`:`flen`]
* [`flen`:`features`]
2017-05-11 03:46:05 +02:00
* [`4`:`timestamp`]
* [`33`:`node_id`]
* [`3`:`rgb_color`]
* [`32`:`alias`]
* [`2`:`addrlen`]
* [`addrlen`:`addresses`]
2016-11-21 06:48:12 +01:00
The `timestamp` allows ordering in the case of multiple announcements;
2017-05-11 03:46:05 +02:00
the `rgb_color` and `alias` allow
2016-11-21 06:48:12 +01:00
intelligence services to give their nodes cool monikers like IRATEMONK
and WISTFULTOLL and use the color black.
2016-12-20 04:53:11 +01:00
`addresses` allows the node to announce its willingness to accept
incoming network connections: it contains series of `address
2017-12-07 00:49:29 +01:00
descriptor`s for connecting to the node. The first byte describes the
2016-12-20 04:53:11 +01:00
address type, followed by the appropriate number of bytes for that type.
The following `address descriptor` types are defined:
2017-12-07 00:49:29 +01:00
* `0` : padding. data = none (length 0).
2017-05-15 21:06:58 +02:00
* `1` : ipv4. data = `[4:ipv4_addr][2:port]` (length 6)
* `2` : ipv6. data = `[16:ipv6_addr][2:port]` (length 18)
2017-05-18 02:22:15 +02:00
* `3` : tor v2 onion service. data = `[10:onion_addr][2:port]` (length 12)
2017-05-15 21:16:29 +02:00
* Version 2 onion service addresses. Encodes an 80-bit truncated `SHA-1` hash
2017-12-14 03:06:28 +01:00
of a 1024-bit `RSA` public key for the onion service (a.k.a. Tor hidden service).
2017-05-18 02:22:15 +02:00
* `4` : tor v3 onion service. data `[35:onion_addr][2:port]` (length 37)
2017-05-15 21:16:29 +02:00
* Version 3 ([prop224](https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt))
2018-01-03 04:21:36 +01:00
onion service addresses. Encodes: `[32:32_byte_ed25519_pubkey] || [2:checksum] || [1:version]` .
2017-05-15 21:16:29 +02:00
where `checksum = sha3(".onion checksum" | pubkey || version)[:2]`
2016-12-20 04:53:11 +01:00
2016-11-21 06:48:12 +01:00
### Requirements
2017-12-07 00:49:29 +01:00
The creating node MUST set `timestamp` to be greater than that for any previous
`node_announcement` it has created. It MAY base it on a UNIX
timestamp. It MUST set `signature` to the signature of
the double-SHA256 of the entire remaining packet after `signature` , using the
key given by `node_id` . It MAY set `alias` and `rgb_color` to customize the node's appearance in maps and graphs, where the first byte of `rgb` is the red value, the second byte is the green value and the last byte is the blue value. It MUST set `alias` to a valid UTF-8 string, with any `alias` bytes following equal to zero.
2016-11-21 06:48:12 +01:00
2016-12-20 04:53:11 +01:00
The creating node SHOULD fill `addresses` with an address descriptor
2017-12-07 00:49:29 +01:00
for each public network address that expects incoming connections,
2016-12-20 04:53:11 +01:00
and MUST set `addrlen` to the number of bytes in `addresses` .
Non-zero typed address descriptors MUST be placed in ascending order;
any number of zero-typed address descriptors MAY be placed anywhere,
but SHOULD only be used for aligning fields following `addresses` .
The creating node MUST NOT create a type 1 or type 2 address
2017-05-11 03:46:05 +02:00
descriptor with `port` equal to zero, and SHOULD ensure `ipv4_addr`
2017-12-07 00:49:29 +01:00
and `ipv6_addr` are routable addresses. The creating node MUST NOT include
2016-12-20 04:53:11 +01:00
more than one `address descriptor` of the same type.
The creating node SHOULD set `flen` to the minimum length required to
2016-12-12 02:11:25 +01:00
hold the `features` bits it sets.
2017-05-11 03:46:05 +02:00
The receiving node SHOULD fail the connection if `node_id` is not a valid
2016-12-12 01:37:19 +01:00
compressed public key, and MUST NOT further process the message.
The receiving node SHOULD fail the connection if `signature` is not a
2017-05-11 03:46:05 +02:00
valid signature using `node_id` of the double-SHA256 of the entire
2016-12-12 01:37:19 +01:00
message following the `signature` field (including unknown fields
following `alias` ), and MUST NOT further process the message.
BOLT7: Reorder feature bitmaps in order to allow future changes
Appending new fields to the end of the messages allows us to add new
fields to an existing message, however it does not allow removing
existing fields, e.g., dropping the pubkeys like #187 proposes. Moving
the features bitmap at the beginning of the signed payload allows
this type of change in the future. Nodes verify the integrity of the
message and then check whether there are any even bits they don't
implement. These even bits being required features would then result
in the message being discarded.
In addition to what we discussed during the call I also went ahead and
did the same reordering on `node_announcement`, which I think has the
same issue.
There is a subtle change in semantics, i.e., previously we would
add channels with unknown bits to our local view, but then ignore them
when computing a route. Now we no longer add them to our view, and may
discard the announcement altogether, stopping the broadcast. This is
safe I think since otherwise we'd be forwarding things we can only
verify the signatures of, but nothing else.
2017-06-27 13:50:43 +02:00
If the `features` field contains unknown even bits the receiving node MUST NOT parse the remainder of the message and MAY discard the message altogether.
The node MAY forward `node_announcement` s that contain unknown `features` bit set, even though it hasn't parsed the announcement.
2017-12-07 00:49:29 +01:00
The receiving node SHOULD ignore the first `address descriptor` that
does not match the types defined above. The receiving node SHOULD
2016-12-20 04:53:11 +01:00
fail the connection if `addrlen` is insufficient to hold the address
descriptors of the known types.
2016-11-21 06:48:12 +01:00
2017-05-11 08:12:22 +02:00
The receiving node SHOULD ignore `ipv6_addr` or `ipv4_addr`
2017-05-07 02:49:42 +02:00
if `port` is zero.
2017-05-11 03:46:05 +02:00
The receiving node SHOULD ignore the message if `node_id` is not
2016-11-21 06:48:12 +01:00
previously known from a `channel_announcement` message, or if
2016-11-22 02:54:10 +01:00
`timestamp` is not greater than the last-received
2017-12-07 00:49:29 +01:00
`node_announcement` from this `node_id` . Otherwise, if the
2016-11-21 06:48:12 +01:00
`timestamp` is greater than the last-received `node_announcement` from
2017-05-11 03:46:05 +02:00
this `node_id` the receiving node SHOULD queue the message for
2016-12-12 02:11:39 +01:00
rebroadcasting, but MAY choose not to for messages longer than
the minimum expected length.
2016-11-21 06:48:12 +01:00
2016-12-12 02:11:25 +01:00
The receiving node SHOULD NOT connect to a node which has an unknown
2017-12-07 00:49:29 +01:00
`features` bit set in the `node_announcement` that is even.
2016-12-12 02:11:25 +01:00
2017-05-11 03:46:05 +02:00
The receiving node MAY use `rgb_color` and `alias` to reference nodes in interfaces, but SHOULD insinuate their self-signed origin.
2016-11-21 06:48:12 +01:00
### Rationale
2017-12-07 00:49:29 +01:00
New node features are possible in the future; backwards compatible (or
2016-12-12 02:11:25 +01:00
optional) ones will have odd feature bits, incompatible ones will have
BOLT7: Reorder feature bitmaps in order to allow future changes
Appending new fields to the end of the messages allows us to add new
fields to an existing message, however it does not allow removing
existing fields, e.g., dropping the pubkeys like #187 proposes. Moving
the features bitmap at the beginning of the signed payload allows
this type of change in the future. Nodes verify the integrity of the
message and then check whether there are any even bits they don't
implement. These even bits being required features would then result
in the message being discarded.
In addition to what we discussed during the call I also went ahead and
did the same reordering on `node_announcement`, which I think has the
same issue.
There is a subtle change in semantics, i.e., previously we would
add channels with unknown bits to our local view, but then ignore them
when computing a route. Now we no longer add them to our view, and may
discard the announcement altogether, stopping the broadcast. This is
safe I think since otherwise we'd be forwarding things we can only
verify the signatures of, but nothing else.
2017-06-27 13:50:43 +02:00
even feature bits. These may be propagated by nodes even if they
2016-12-12 02:11:25 +01:00
can't use the announcements themselves.
2017-12-07 00:49:29 +01:00
New address types can be added in the future; as address descriptors have
to be ordered in ascending order, unknown ones can be safely ignored.
2016-12-20 04:53:11 +01:00
Future fields beyond `addresses` can still be added, optionally with
padding within `addresses` if they require certain alignment.
2016-11-21 06:48:12 +01:00
2017-12-08 19:15:48 +01:00
## The `channel_update` Message
2016-11-21 06:48:12 +01:00
After a channel has been initially announced, each side independently
2017-10-18 15:31:31 +02:00
announces the fees and minimum expiry delta it requires to relay HTLCs
2017-12-08 19:15:48 +01:00
through this channel. Each uses the 8-byte
channel shortid that matches the `channel_announcement` and 1 bit
2017-05-07 02:54:29 +02:00
in the `flags` field
2017-12-07 00:49:29 +01:00
to indicate which end this is. A node can do this multiple times, if
2016-11-21 06:48:12 +01:00
it wants to change fees.
2018-01-03 04:21:36 +01:00
Note that the `channel_update` message is only useful in the context
2017-10-18 15:31:31 +02:00
of *relaying* payments, not *sending* payments. When making a payment
2018-01-03 04:21:36 +01:00
`A` -> `B` -> `C` -> `D` , only the `channel_update` s related to channels
`B` -> `C` (announced by `B` ) and `C` -> `D` (announced by `C` ) will
2017-12-07 00:49:29 +01:00
come into play. When building the route, amounts and expiries for HTLCs need
to be calculated backward from the destination to the source. The initial
2017-12-18 20:38:04 +01:00
exact value for `amount_msat` and minimal value for `cltv_expiry` , which are
2017-10-18 15:31:31 +02:00
to be used for the last HTLC in the route, are provided in the payment request
(see [BOLT #11 ](11-payment-encoding.md#tagged-fields )).
2017-06-27 15:47:04 +02:00
A node MAY still create a `channel_update` to communicate the channel parameters to the other endpoint, even though the channel has not been announced, e.g., because the `announce_channel` bit was not set.
2017-12-07 00:49:29 +01:00
2017-06-27 15:15:52 +02:00
For further privacy such a `channel_update` MUST NOT be forwarded to other peers.
Note that such a `channel_update` that is not preceded by a `channel_announcement` is invalid to any other peer and would be discarded.
2016-11-23 02:03:25 +01:00
1. type: 258 (`channel_update`)
2016-11-21 06:48:12 +01:00
2. data:
2017-05-11 03:46:05 +02:00
* [`64`:`signature`]
2017-07-10 21:46:39 +02:00
* [`32`:`chain_hash`]
2017-05-11 03:46:05 +02:00
* [`8`:`short_channel_id`]
* [`4`:`timestamp`]
* [`2`:`flags`]
* [`2`:`cltv_expiry_delta`]
2017-05-23 05:06:34 +02:00
* [`8`:`htlc_minimum_msat`]
2017-05-11 03:46:05 +02:00
* [`4`:`fee_base_msat`]
* [`4`:`fee_proportional_millionths`]
2017-12-07 00:49:29 +01:00
The `flags` bitfield is used to indicate the direction of the channel this update concerns: it identifies the node that this update originated from and signals various options concerning the channel.
2017-04-08 00:13:10 +02:00
The following table specifies the meaning of the individual bits:
| Bit Position | Name | Meaning |
| ------------- | ----------- | -------------------------------- |
| 0 | `direction` | Direction this update refers to. |
| 1 | `disable` | Disable the channel. |
2016-11-21 06:48:12 +01:00
### Requirements
The creating node MUST set `signature` to the signature of the
2017-12-07 00:49:29 +01:00
double-SHA256 of the entire remaining packet after `signature` , using its own `node_id` .
2016-11-21 06:48:12 +01:00
2017-07-10 21:46:39 +02:00
The creating node MUST set `chain_hash` and `short_channel_id` to match the
32-byte hash and 8-byte channel ID that uniquely identifies the channel within
the `channel_announcement` message.
2017-12-07 00:49:29 +01:00
The creating node MUST set the `direction` bit of `flags` to 0 if the creating node is `node_id_1` in that message, otherwise 1.
2017-04-08 00:13:10 +02:00
Bits which are not assigned a meaning must be set to 0.
A node MAY create and send a `channel_update` with the `disable` bit set to signal the temporary unavailability of a channel, e.g., due to loss of connectivity, or the permanent unavailability, e.g., ahead of an on-chain settlement.
A subsequent `channel_update` with the `disable` bit unset MAY re-enable the channel.
2016-11-21 06:48:12 +01:00
2018-01-08 04:31:12 +01:00
The creating node MUST set `timestamp` to greater than zero, and MUST set it to greater than any previously-sent `channel_update` for this `short_channel_id` , and SHOULD base it on a UNIX timestamp.
2017-11-28 13:13:19 +01:00
It MUST set `cltv_expiry_delta` to the number of blocks it will subtract from an incoming HTLCs `cltv_expiry` . It MUST set `htlc_minimum_msat` to the minimum HTLC value the other end of the channel will accept, in millisatoshi. It MUST set `fee_base_msat` to the base fee it will charge for any HTLC, in millisatoshi, and `fee_proportional_millionths` to the amount it will charge per transferred satoshi in millionths of a satoshi.
2016-11-21 06:48:12 +01:00
2017-12-07 00:49:29 +01:00
The receiving nodes MUST ignore the `channel_update` if it does not correspond to one of its own channels, if the `short_channel_id` does not match a previous `channel_announcement` , or if the channel has been closed in the meantime.
2017-06-27 15:15:52 +02:00
It SHOULD accept `channel_update` s for its own channels in order to learn the other end's forwarding parameters, even for non-public channels.
2017-12-07 00:49:29 +01:00
The `node_id` for the signature verification is taken from the corresponding `channel_announcement` : `node_id_1` if the least-significant bit of flags is 0 or `node_id_2` otherwise.
2016-12-12 01:37:19 +01:00
The receiving node SHOULD fail the connection if `signature` is not a
2017-05-11 03:46:05 +02:00
valid signature using `node_id` of the double-SHA256 of the entire
2016-12-12 01:37:19 +01:00
message following the `signature` field (including unknown fields
2017-05-11 03:46:05 +02:00
following `fee_proportional_millionths` ), and MUST NOT further process the message.
2016-11-22 20:30:37 +01:00
2017-07-10 21:46:39 +02:00
The receiving node MUST ignore the channel update if the specified
`chain_hash` value is unknown, meaning it isn't active on the specified
chain.
2016-11-22 20:30:37 +01:00
The receiving node SHOULD ignore the message if `timestamp`
2017-12-07 00:49:29 +01:00
is not greater than that of the last-received `channel_announcement` for
this `short_channel_id` and `node_id` . Otherwise, if the `timestamp` is equal to
2016-11-21 06:48:12 +01:00
the last-received `channel_announcement` and the fields other than
2017-05-11 03:46:05 +02:00
`signature` differ, the node MAY blacklist this `node_id` and forget all
2018-01-08 04:31:12 +01:00
channels associated with it. The receiving node MAY discard the `channel_announcement` if the `timestamp` is far in the future.
Otherwise the receiving node SHOULD
2016-12-12 02:11:39 +01:00
queue the message for rebroadcasting, but MAY choose not to for
messages longer than the minimum expected length.
2016-11-21 06:48:12 +01:00
2018-01-08 04:31:12 +01:00
### Rationale
The `timestamp` field is used by nodes for pruning (either if it's too
far in the future, or if it's been two weeks with no update), so it
makes sense to have it be a UNIX timestamp (ie. seconds since UTC
1970-01-01). It can't be a hard requirement, however, given the possible case
of two `channel_update` s within a second.
2017-02-07 17:14:35 +01:00
## Initial Sync
Upon establishing a connection, the two endpoints negotiate whether to perform an initial sync by setting the `initial_routing_sync` flags in the `init` message.
The endpoint SHOULD set the `initial_routing_sync` flag if it requires a full copy of the other endpoint's routing state.
2018-01-03 04:21:36 +01:00
Upon receiving an `init` message with the `initial_routing_sync` flag set, the node sends `channel_announcement` s, `channel_update` s and `node_announcement` s for all known channels and nodes as if they were just received.
2017-02-07 17:14:35 +01:00
2017-12-07 00:49:29 +01:00
If the `initial_routing_sync` flag is not set, or initial sync was completed, then the node resumes normal operation: see the _Rebroadcasting_ section for details.
2017-02-07 17:14:35 +01:00
2016-11-21 06:48:12 +01:00
## Rebroadcasting
2016-11-22 02:54:10 +01:00
Nodes receiving a new `channel_announcement` or a `channel_update` or
2017-12-07 00:49:29 +01:00
`node_announcement` with an updated timestamp SHOULD update their local view of the network's topology accordingly.
2016-11-21 06:48:12 +01:00
If, after applying the changes from the announcement, there are no channels associated with the announcing node, then the receiving node MAY purge the announcing node from the set of known nodes.
Otherwise the receiving node updates the metadata and stores the signature associated with the announcement.
This will later allow the receiving node to rebuild the announcement for its peers.
2017-12-07 00:49:29 +01:00
Once the announcement has been processed, it is added to a list of outgoing announcements for the processing node's peers, perhaps replacing older updates. This list will be flushed at regular intervals.
This store-and-delayed-forward broadcast is called a _staggered broadcast_
2016-11-21 06:48:12 +01:00
### Requirements
Each node SHOULD flush outgoing announcements once every 60 seconds, independently of the arrival times of announcements, resulting in a staggered announcement and deduplication of announcements.
Nodes MAY re-announce their channels regularly, however this is discouraged in order to keep the resource requirements low.
Nodes SHOULD send all `channel_announcement` messages followed by the
latest `node_announcement` and `channel_update` messages upon
connection establishment.
### Rationale
2017-12-07 00:49:29 +01:00
Batching announcements forms a natural rate limit with low overhead.
2016-11-21 06:48:12 +01:00
The sending of all announcements on reconnection is naive, but simple,
2017-12-07 00:49:29 +01:00
and allows bootstrapping for new nodes as well as updating for nodes that
2016-11-21 06:48:12 +01:00
have been offline for some time.
2016-11-22 02:54:10 +01:00
## HTLC Fees
2017-12-07 00:49:29 +01:00
The node creating `channel_update` SHOULD accept HTLCs that pay a fee equal or greater than:
2016-11-22 02:54:10 +01:00
2017-12-07 00:49:29 +01:00
fee_base_msat + ( amount_msat * fee_proportional_millionths / 1000000 )
2016-11-22 02:54:10 +01:00
2017-12-07 00:49:29 +01:00
The node creating `channel_update` SHOULD accept HTLCs that pay an
older fee for some time after sending `channel_update` , to allow for
2016-11-22 02:54:10 +01:00
propagation delay.
2017-07-11 02:43:09 +02:00
## Pruning the Network View
Nodes SHOULD monitor the funding transactions in the blockchain to identify channels that are being closed.
If the funding output of a channel is being spent, then the channel is to be considered closed and SHOULD be removed from the local network view.
2017-12-07 00:49:29 +01:00
Nodes MAY prune nodes added through `node_announcement` messages from their local view if the announced node no longer has any associated open channels.
2017-07-11 02:43:09 +02:00
This is a direct result from the dependency of a `node_announcement` being preceded by a `channel_announcement` .
2017-12-07 00:49:29 +01:00
### Recommendation on Pruning Stale Entries
2017-07-11 02:43:09 +02:00
Several scenarios may result in channels becoming unusable and the endpoints unable to send updates for these channels.
2017-12-07 00:49:29 +01:00
For example, this happens in the case where both endpoints lose access to their private keys and cannot sign a `channel_update` nor close the channel on-chain.
These channels are unlikely to be part of a computed route since they would be partitioned off from the rest of the network, however they would remain in the local network view and information on them would be forwarded to other nodes forever.
For this reason, nodes MAY prune channels should the timestamp of the latest `channel_update` be older than 2 weeks (1209600 seconds).
2017-07-11 02:43:09 +02:00
In addition nodes MAY ignore channels with a timestamp older than 2 weeks.
Notice that this is a node policy and MUST NOT be enforced by peers, e.g., by closing channels when receiving outdated gossip messages.
2016-11-22 02:54:10 +01:00
## Recommendations for Routing
2017-05-11 03:46:05 +02:00
When calculating a route for an HTLC, the `cltv_expiry_delta` and the fee both
need to be considered: the `cltv_expiry_delta` contributes to the time that funds
2017-12-07 00:49:29 +01:00
will be unavailable on worst-case failure. The tradeoff between these
2016-11-22 02:54:10 +01:00
two is unclear, as it depends on the reliability of nodes.
2017-07-11 12:26:26 +02:00
If a route is computed by simply routing to the intended recipient, summing up the `cltv_expiry_delta` s, then nodes along the route may guess their position in the route.
Knowing the CLTV of the HTLC and the surrounding topology with the `cltv_expiry_delta` s gives an attacker a way to guess the intended recipient.
Therefore it is highly suggested to add a random offset to the CLTV that the intended recipient will receive, bumping all CLTVs along the route.
In order to create a plausible offset the sender MAY start a limited random walk on the graph, starting from the intended recipient, sum the `cltv_expiry_delta` s, and then use the sum as the offset.
This effectively creates a _shadow route extension_ to the actual route, providing better protection against this kind of attack than simply picking a random offset.
2016-11-22 02:54:10 +01:00
Other more advanced considerations involve diversity of routes to
2017-12-07 00:49:29 +01:00
avoid single points of failure and detection and channel balance
2016-11-22 02:54:10 +01:00
of local channels.
2017-10-02 11:49:26 +02:00
### Routing Example
Consider four nodes:
```
B
/ \
/ \
2018-01-03 04:21:36 +01:00
A C
2017-10-02 11:49:26 +02:00
\ /
\ /
D
```
Each advertises the following `cltv_expiry_delta` on its end of every
channel:
1. A: 10 blocks
2. B: 20 blocks
3. C: 30 blocks
4. D: 40 blocks
C also uses a`min_final_cltv_expiry` of 9 (the default) when requesting
payments.
2017-12-07 00:49:29 +01:00
Also, each node has a set fee scheme that it uses for each of its
2017-10-02 11:49:26 +02:00
channels:
1. A: 100 base + 1000 millionths
2017-12-07 00:49:29 +01:00
2. B: 200 base + 2000 millionths
3. C: 300 base + 3000 millionths
4. D: 400 base + 4000 millionths
2017-10-02 11:49:26 +02:00
The network will see eight `channel_update` messages:
1. A->B: `cltv_expiry_delta` = 10, `fee_base_msat` = 100, `fee_proportional_millionths` = 1000
1. A->D: `cltv_expiry_delta` = 10, `fee_base_msat` = 100, `fee_proportional_millionths` = 1000
1. B->A: `cltv_expiry_delta` = 20, `fee_base_msat` = 200, `fee_proportional_millionths` = 2000
1. D->A: `cltv_expiry_delta` = 40, `fee_base_msat` = 400, `fee_proportional_millionths` = 4000
1. B->C: `cltv_expiry_delta` = 20, `fee_base_msat` = 200, `fee_proportional_millionths` = 2000
1. D->C: `cltv_expiry_delta` = 40, `fee_base_msat` = 400, `fee_proportional_millionths` = 4000
1. C->B: `cltv_expiry_delta` = 30, `fee_base_msat` = 300, `fee_proportional_millionths` = 3000
1. C->D: `cltv_expiry_delta` = 30, `fee_base_msat` = 300, `fee_proportional_millionths` = 3000
2017-12-07 00:49:29 +01:00
**B->C.** If B were to send 4,999,999 millisatoshi directly to C, it wouldn't
2017-10-02 11:49:26 +02:00
charge itself a fee nor add its own `cltv_expiry_delta` , so it would
2017-12-07 00:49:29 +01:00
use C's requested `min_final_cltv_expiry` of 9. Assume it also adds a
"shadow route" to give an extra CLTV of 42. It could additionally add extra
CLTV deltas at other hops, as these values are a minimum, but it doesn't
2017-10-02 11:49:26 +02:00
here for simplicity:
* `amount_msat` : 4999999
* `cltv_expiry` : current-block-height + 9 + 42
* `onion_routing_packet` :
* `amt_to_forward` = 4999999
* `outgoing_cltv_value` = current-block-height + 9 + 42
2017-12-07 00:49:29 +01:00
**A->B->C.** If A were to send an 4,999,999 millisatoshi to C via B, it needs to
2017-10-02 11:49:26 +02:00
pay B the fee it specified in the B->C `channel_update` , calculated as
per [HTLC Fees ](#htlc_fees ):
2017-12-07 00:49:29 +01:00
fee_base_msat + ( amount_msat * fee_proportional_millionths / 1000000 )
200 + ( 4999999 * 2000 / 1000000 ) = 10199
2017-10-02 11:49:26 +02:00
Similarly, it would need to add the `cltv_expiry` from B->C's
`channel_update` (20), plus C's requested `min_final_cltv_expiry` (9), plus 42 for the
"shadow route". Thus the `update_add_htlc` message from A to B would
be:
* `amount_msat` : 5010198
* `cltv_expiry` : current-block-height + 20 + 9 + 42
* `onion_routing_packet` :
* `amt_to_forward` = 4999999
* `outgoing_cltv_value` = current-block-height + 9 + 42
The `update_add_htlc` from B to C would be the same as the B->C direct
payment above.
2017-12-07 00:49:29 +01:00
**A->D->C.** Finally, if for some reason A chose the more expensive route via D, it
2017-10-02 11:49:26 +02:00
would send the following `update_add_htlc` to D:
* `amount_msat` : 5020398
* `cltv_expiry` : current-block-height + 40 + 9 + 42
* `onion_routing_packet` :
* `amt_to_forward` = 4999999
* `outgoing_cltv_value` = current-block-height + 9 + 42
And the `update_add_htlc` from D to C would be the same as the B->C
direct payment again.
2016-11-21 06:48:12 +01:00
## References
2016-11-22 20:52:59 +01:00
![Creative Commons License ](https://i.creativecommons.org/l/by/4.0/88x31.png "License CC-BY" )
< br >
This work is licensed under a [Creative Commons Attribution 4.0 International License ](http://creativecommons.org/licenses/by/4.0/ ).