This commit adds the interactive transaction construction protcol, as
well as the first practical example of using it, v2 of channel
establishment.
Note that for v2 we also update the channel_id, which now uses the hash
of the revocation_basepoints. We move away from using the funding
transaction id, as the introduction of RBF* makes it such that a single
channel may have many funding transaction id's over the course of
its lifetime.
*Later, also splicing
6fee63fc34 made `htlc_maximum_msat`
compulsory, which was the last "optional" field. All future options
will be TLVs, so we never need to parse the "] (optionxxx)" format
for fields again.
We allow options for messages, but only for documentation (we have
some messages which are only for `gossip_queries`), so we don't
put it in the output.
Here's the before/after diff of the output for all the spec files:
```diff
--- /tmp/before 2023-04-27 14:21:07.417170213 +0930
+++ /tmp/after 2023-04-27 14:19:39.184573086 +0930
@@ -273,7 +273,7 @@
msgdata,channel_update,fee_base_msat,u32,
msgdata,channel_update,fee_proportional_millionths,u32,
msgdata,channel_update,htlc_maximum_msat,u64,
-msgtype,query_short_channel_ids,261,gossip_queries
+msgtype,query_short_channel_ids,261
msgdata,query_short_channel_ids,chain_hash,chain_hash,
msgdata,query_short_channel_ids,len,u16,
msgdata,query_short_channel_ids,encoded_short_ids,byte,len
@@ -281,17 +281,17 @@
tlvtype,query_short_channel_ids_tlvs,query_flags,1
tlvdata,query_short_channel_ids_tlvs,query_flags,encoding_type,byte,
tlvdata,query_short_channel_ids_tlvs,query_flags,encoded_query_flags,byte,...
-msgtype,reply_short_channel_ids_end,262,gossip_queries
+msgtype,reply_short_channel_ids_end,262
msgdata,reply_short_channel_ids_end,chain_hash,chain_hash,
msgdata,reply_short_channel_ids_end,full_information,byte,
-msgtype,query_channel_range,263,gossip_queries
+msgtype,query_channel_range,263
msgdata,query_channel_range,chain_hash,chain_hash,
msgdata,query_channel_range,first_blocknum,u32,
msgdata,query_channel_range,number_of_blocks,u32,
msgdata,query_channel_range,tlvs,query_channel_range_tlvs,
tlvtype,query_channel_range_tlvs,query_option,1
tlvdata,query_channel_range_tlvs,query_option,query_option_flags,bigsize,
-msgtype,reply_channel_range,264,gossip_queries
+msgtype,reply_channel_range,264
msgdata,reply_channel_range,chain_hash,chain_hash,
msgdata,reply_channel_range,first_blocknum,u32,
msgdata,reply_channel_range,number_of_blocks,u32,
@@ -310,7 +310,7 @@
subtype,channel_update_checksums
subtypedata,channel_update_checksums,checksum_node_id_1,u32,
subtypedata,channel_update_checksums,checksum_node_id_2,u32,
-msgtype,gossip_timestamp_filter,265,gossip_queries
+msgtype,gossip_timestamp_filter,265
msgdata,gossip_timestamp_filter,chain_hash,chain_hash,
msgdata,gossip_timestamp_filter,first_timestamp,u32,
msgdata,gossip_timestamp_filter,timestamp_range,u32,
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This script was previously used in Electrum but no more.
So since this script doesn't even parse the output of
extract-formats.py any more, just remove it.
In practice, using '...*type' is the clearest and simplest way to specify
the common case of "the rest of the TLV is an array of 'type'", rather
than some arbitrary expression with a made-up length field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The format for TLV types looked pretty, but @ZmnSCPxj points out that
successive ordered lists in markdown get merged into one megalist.
If we allow ordered or unordered lists, we're a bit more futureproof
against formatting changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were swallowing the unused line after `data`, but it's
normal to do:
```
1. tlvs: `n1`
2. types:
1. type: 1 (`tlv1`)
2. data:
* [`tu64`:`amount_msat`]
1. type: 2 (`tlv2`)
2. data:
* [`short_channel_id`:`scid`]
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Editing the previous mess was horrific. I gave up and rewrote using a
generator.
Changes to output:
1. subtypes and tlvs now handled.
2. The output format now has explicit prefixes, so readers don't have
to rely on number of fields to interpret data.
3. Each field is split into type and count; count is empty if there's
no '*x'.
4. TLV stream typenames are repeated; TLV record type names are not
necessarily unique.
5. The unused offset field is removed.
6. No arguments taken: everything is always printed, and you can grep if you
only want some.
[ Fixup by <niftynei@gmail.com> ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're about to introduce new messages which are gated by options (not
just single fields). Ignore them for the purposes of parsing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We've always specified that fields can be appended, and we're about to
do our first. Make it clear that you can't just send one if we add several
fields as part of the same option.
Change extract-formats to have an optional new field appended: the
input format is to simply add ' (optionname)' in the data
specification.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This changes extract-formats.py so that other scripts can use it, but retains normal functionality.
The new script (structured.py) parses the CSV variant and shows a representation of an OrderedMap.
This could be used to write parsers.
This is a multi-part message in MIME format.
This patch should apply to http://github.com/lightningnetwork/lightning-rfc
Nonidealities:
Aspell triggers spelling errors on the hexadecimal strings in
the test vectors. I don't have enough aspell-fu to figure
out how to make Aspell ignore these.
There are 2 possible pluralizations of `HTLC`: `HTLCs` and
`HTLC's`. I'd prefer the latter, but for now I support both.
We should standardize pluralization; we can edit the
`.aspell.en.pws` file to remove the pluralization we won't
choose.
They're really a blob of bytes, and we weren't aligning them correctly in
two cases anyway. This gets rid of gratuitous padding, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>