In fact, only 951 of 17419 (5%) of node announcements are missing an address
(and gossipd doesn't know if we can connect to Tor addresses anyway) so
just check it *has* a node_announcement.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Let lightningd feed us hints to try first, but we can extract the
addresses from node_announcement messages ourselves.
(Lightningd used to ask gossipd on our behalf: this is far simpler!)
One side effect of this is that we don't hand back address hints given to us
by lightningd: it would use these again for reconnecting. This is breaks
test_sendpay_grouping, so we disable it temporarily.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't want to to refresh the gossmap internally: this could invalidate the
gossmap held by the current callers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This does not validate a node announcement and address, but it
does select a node at random from the gossmap and asks lightningd
to attempt a connection to it.
Gossipd uses this to ask lightningd -> connectd to initiate
a connection to a new gossip peer. This can be used when
there are insufficient peers already connected to gossip with.
Changelog-Changed: Gossipd can now request connections to additional nodes for improved gossip sync
This can help us backfill any missing gossip if our current
peers haven't been the most reliable.
Changelog-Changed: Gossipd requests a full sync from a random peer every hour.
Based on gossip sync data from random network peers, listening to only 5
peers will not reliably catch all gossip traffic. For now, add extra
redundancy.
This may help the cases we see where gossipd doesn't realize channels
are closed (because of shutdown before it processed the closing).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: `gossipd` will no longer miss some channel closes on restart.
No code changes, just catching up with the BOLT changes which rework our
blinded path terminology (for the better!).
Another patch will sweep the rest of our internal names, this tries only to
make things compile and fix up the BOLT quotes.
1. Inside payload: current_blinding_point -> current_path_key
2. Inside update_add_htlc TLV: blinding_point -> blinded_path
3. Inside blinded_path: blinding -> first_path_key
4. Inside onion_message: blinding -> path_key.
5. Inside encrypted_data_tlv: next_blinding_override -> next_path_key_override
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The "init_cupdate" message is for gossipd to tell lightningd about our *own* latest
channel_update messages, not the remote ones! The "remote_channel_update" message
is for messages from the peer.
This appeared as an occasional BROKEN message in CI:
```
**BROKEN** 035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-chan#4: gossipd gave us channel_update for channel in gossip_state CGOSSIP_NEED_PEER_SIGS
```
Where we had sent (and not received) announcement_signatures, and restarted: the peer had meanwhile
sent us their channel_announcement and channel_update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: we could get confused on restart and not re-transmit our own channel_updates.
Since we don't compact the gossmap on the fly (FIXME!) we can
easily surpass 4GB in the gossmap, and 32 bit offsets are not
sufficient.
I'm a bit surprised we don't crash immediately, but we've definitely
seen issues.
Changelog-Fixed: gossipd: crash errors with large gossip_store (>4MB) growth on longer-running nodes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a hack, but we've had nodes missing gossip. LDK does this,
so until we get a better workaround, use this one.
Changelog-Changed: Protocol: we now always ask the first peer for all its gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This basically means moving the code from gossipd to connectd to handle
these queries.
This will get connectd have finer control over ratelimiting them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was removed from the spec on Apr 25, 2022. We stopped ever sending them
in 0.12.0 (2022-08-23). Now we remove receive support.
Changelog-Protocol: Removed support for zlib-compressed short-channel-ids in query responses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Everyone understands gossip_queries now, but peers leave it unset to indicate
they have nothing useful to say.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We only write these in two places: one where we get a message from lightningd about
our own channel, and one where we get a reply from lightningd about a txout check.
The former case we explicitly check that we don't already have it in gossmap, so
add checks to the latter case, and give verbose detail if it's found.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't actually support it yet, but this threads through the type change,
puts it in "decode" etc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We now *never* consider asking them anything, even if they are
capable (e.g. enabling full gossip stream). This aligns with
the latest spec proposal, where lack of `gossip_queries` means
"we don't have anything useful to say".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's a u64, we should pass by copy. This is a big sweeping change,
but mainly mechanical (change one, compile, fix breakage, repeat).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This has the benefit of being shorter, as well as more reliable (you
will get a link error if we can't print it, not a runtime one!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We have various functions to convert to a string, rename them all so we can
count on fmt_X being the formatter for struct X, and make them all return
`char *`.
Sometimes they existed but were private, sometimes they had a
different name. Most take a pointer, but simple types pass by copy:
short_channel_id, amount_msat and amount_sat.
The following public functions changed:
1. psbt_to_b64 -> fmt_wally_psbt.
2. pubkey_to_hexstr -> fmt_pubkey.
3. short_channel_id_to_str -> fmt_short_channel_id (scid by copy now!)
4. fmt_signature -> fmt_secp256k1_ecdsa_signature
5. fmt_amount_sat/fmt_amount_msat pass copy not pointer, return non-const char *.
6. node_id_to_hexstr -> fmt_node_id
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we delete it the first time a channel before it is closed, we will
crash when we try to move it again:
```
$ lightning_gossipd: gossip_store: get delete entry offset 47411992/51608943 (version v23.11-378-gac2a386-modded)
0x1002544b send_backtrace
common/daemon.c:33
0x1003415f status_failed
common/status.c:221
0x10016ef3 gossip_store_get_with_hdr
gossipd/gossip_store.c:466
0x10016faf check_msg_type
gossipd/gossip_store.c:491
0x1001722b gossip_store_set_flag
gossipd/gossip_store.c:509
0x1001752b gossip_store_del
gossipd/gossip_store.c:561
0x10017f5b remove_channel
gossipd/gossmap_manage.c:299
0x100181cf kill_spent_channel
gossipd/gossmap_manage.c:1144
0x1001a7df gossmap_manage_new_block
gossipd/gossmap_manage.c:1183
0x10014673 new_blockheight
gossipd/gossipd.c:483
0x10014d73 recv_req
gossipd/gossipd.c:594
```
Reported-by: @microsatosi on Discord
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Normally, channels are marked dying, the 12 blocks later, removed.
But for local channels, we can access any spliced channel already, so
we remove them immediately from our local gossip. This left a hole in
our logic, if that channel was the last one keeping a
node_announcement alive.
Solution is to unify with the "moved node_announcement" path.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We make sure a node_announcement is preceeded by at least one channel_announcement,
but dying ones don't count (as they are not broadcast!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We accept node_announcements on dying channels, but make sure we
set the dying flag it channels are alll dying.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We can update dying channels, though it seems weird! We accept gossip about them,
we just don't propagate it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids us gossiping about nodes which don't have live channels.
Interstingly, we previously tested that we *did* gossip such node
announcements, and now we fix that test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
