6fee63fc34 made `htlc_maximum_msat`
compulsory, which was the last "optional" field. All future options
will be TLVs, so we never need to parse the "] (optionxxx)" format
for fields again.
We allow options for messages, but only for documentation (we have
some messages which are only for `gossip_queries`), so we don't
put it in the output.
Here's the before/after diff of the output for all the spec files:
```diff
--- /tmp/before 2023-04-27 14:21:07.417170213 +0930
+++ /tmp/after 2023-04-27 14:19:39.184573086 +0930
@@ -273,7 +273,7 @@
msgdata,channel_update,fee_base_msat,u32,
msgdata,channel_update,fee_proportional_millionths,u32,
msgdata,channel_update,htlc_maximum_msat,u64,
-msgtype,query_short_channel_ids,261,gossip_queries
+msgtype,query_short_channel_ids,261
msgdata,query_short_channel_ids,chain_hash,chain_hash,
msgdata,query_short_channel_ids,len,u16,
msgdata,query_short_channel_ids,encoded_short_ids,byte,len
@@ -281,17 +281,17 @@
tlvtype,query_short_channel_ids_tlvs,query_flags,1
tlvdata,query_short_channel_ids_tlvs,query_flags,encoding_type,byte,
tlvdata,query_short_channel_ids_tlvs,query_flags,encoded_query_flags,byte,...
-msgtype,reply_short_channel_ids_end,262,gossip_queries
+msgtype,reply_short_channel_ids_end,262
msgdata,reply_short_channel_ids_end,chain_hash,chain_hash,
msgdata,reply_short_channel_ids_end,full_information,byte,
-msgtype,query_channel_range,263,gossip_queries
+msgtype,query_channel_range,263
msgdata,query_channel_range,chain_hash,chain_hash,
msgdata,query_channel_range,first_blocknum,u32,
msgdata,query_channel_range,number_of_blocks,u32,
msgdata,query_channel_range,tlvs,query_channel_range_tlvs,
tlvtype,query_channel_range_tlvs,query_option,1
tlvdata,query_channel_range_tlvs,query_option,query_option_flags,bigsize,
-msgtype,reply_channel_range,264,gossip_queries
+msgtype,reply_channel_range,264
msgdata,reply_channel_range,chain_hash,chain_hash,
msgdata,reply_channel_range,first_blocknum,u32,
msgdata,reply_channel_range,number_of_blocks,u32,
@@ -310,7 +310,7 @@
subtype,channel_update_checksums
subtypedata,channel_update_checksums,checksum_node_id_1,u32,
subtypedata,channel_update_checksums,checksum_node_id_2,u32,
-msgtype,gossip_timestamp_filter,265,gossip_queries
+msgtype,gossip_timestamp_filter,265
msgdata,gossip_timestamp_filter,chain_hash,chain_hash,
msgdata,gossip_timestamp_filter,first_timestamp,u32,
msgdata,gossip_timestamp_filter,timestamp_range,u32,
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In practice, using '...*type' is the clearest and simplest way to specify
the common case of "the rest of the TLV is an array of 'type'", rather
than some arbitrary expression with a made-up length field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The format for TLV types looked pretty, but @ZmnSCPxj points out that
successive ordered lists in markdown get merged into one megalist.
If we allow ordered or unordered lists, we're a bit more futureproof
against formatting changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were swallowing the unused line after `data`, but it's
normal to do:
```
1. tlvs: `n1`
2. types:
1. type: 1 (`tlv1`)
2. data:
* [`tu64`:`amount_msat`]
1. type: 2 (`tlv2`)
2. data:
* [`short_channel_id`:`scid`]
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Editing the previous mess was horrific. I gave up and rewrote using a
generator.
Changes to output:
1. subtypes and tlvs now handled.
2. The output format now has explicit prefixes, so readers don't have
to rely on number of fields to interpret data.
3. Each field is split into type and count; count is empty if there's
no '*x'.
4. TLV stream typenames are repeated; TLV record type names are not
necessarily unique.
5. The unused offset field is removed.
6. No arguments taken: everything is always printed, and you can grep if you
only want some.
[ Fixup by <niftynei@gmail.com> ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're about to introduce new messages which are gated by options (not
just single fields). Ignore them for the purposes of parsing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We've always specified that fields can be appended, and we're about to
do our first. Make it clear that you can't just send one if we add several
fields as part of the same option.
Change extract-formats to have an optional new field appended: the
input format is to simply add ' (optionname)' in the data
specification.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This changes extract-formats.py so that other scripts can use it, but retains normal functionality.
The new script (structured.py) parses the CSV variant and shows a representation of an OrderedMap.
This could be used to write parsers.
They're really a blob of bytes, and we weren't aligning them correctly in
two cases anyway. This gets rid of gratuitous padding, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
