We want it to keep the latest, so it can make its own error msgs without
asking us. This installs (but does not use!) the message handler.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
An "err" is only returned if the channel_update is malformed: more common
is that it's fine, but we don't know the scid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's weird to have connectd ask gossipd, when lightningd can just do it
and hand all the addresses together.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We now let gossipd do it.
This also means there's nothing left in 'struct per_peer_state' to
send across the wire (the fds are sent separately), so that gets
removed from wire messages too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
local_chan was mainly around so we could "soft" disable channels (and
really disable them once we used the channel_update in an error
message).
Instead we introduce the idea of a "deferred_update": it's either
deferred indefinitely (a peer goes offline, if we need to send it in
an error we'll apply it immediatly), or simply delayed to avoid
spamming everyone.
The resulting rewrite is much clearer, IMHO.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Increasingly we want to know is it local, and get the direction: it's
more efficient to do both at once.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).
config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's very similar to the previous, but there are a few changes:
1. The enctlv fields are numbered differently.
2. The message itself is a different number.
The onionmsg_path type is the same, however, so we keep that constant
at least.
The result is a lot of cut & paste, but we will delete the old one
next release.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is from 6e99c5feaf60cb797507d181fe583224309318e9
We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513. We don't handle it until the next patch.
Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.
We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Temporarily disable sendpay_blinding test which uses obsolete onionmsg;
there's still some debate on the PR about how blinded HTLCs will work.
Changelog-EXPERIMENTAL: onionmessage: removed support for v0.10.1 onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We can have an update pending because it's too fast, but
refresh_local_channel is supposed to make sure we're up-to-date, so
force immediate application in that case.
Otherwise, we call update_local_channel at the bottom which frees the
pending update. This can mean that we miss a change in fees, for example.
Changelog-Fixed: errors: Errors returning a `channel_update` no longer return an outdated one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
To minimize the diffs, we #if 0 the code. We'll reenable it once
channeld is ready.
We also temporarily disable the ping tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This adds a new hook: onion_message_ourpath for when we know a message
came in via a blinded path we created. The onion_message_blinded hook
is now called for all other messages, since all messages are now
blinded.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
By popular merge-hell demand.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
One change from the obsolete version handling, gossipd will no longer send
forwarding onion msgs to lightningd, but will forward it directly.
That was the effect before, anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We never tested that we can correctly unwrap on the next step after
unblinding: it failed because we mangled the onion in place! Fix that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Before:
Ten builds, laptop -j5, no ccache:
```
real 0m36.686000-38.956000(38.608+/-0.65)s
user 2m32.864000-42.253000(40.7545+/-2.7)s
sys 0m16.618000-18.316000(17.8531+/-0.48)s
```
Ten builds, laptop -j5, ccache (warm):
```
real 0m8.212000-8.577000(8.39989+/-0.13)s
user 0m12.731000-13.212000(12.9751+/-0.17)s
sys 0m3.697000-3.902000(3.83722+/-0.064)s
```
After:
Ten builds, laptop -j5, no ccache: 8% faster
```
real 0m33.802000-35.773000(35.468+/-0.54)s
user 2m19.073000-27.754000(26.2542+/-2.3)s
sys 0m15.784000-17.173000(16.7165+/-0.37)s
```
Ten builds, laptop -j5, ccache (warm): 1% faster
```
real 0m8.200000-8.485000(8.30138+/-0.097)s
user 0m12.485000-13.100000(12.7344+/-0.19)s
sys 0m3.702000-3.889000(3.78787+/-0.056)s
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly). Make it a requirement,
either of form "foo.h" or <dir/foo.h>.
The noise is the change to our print templates.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds). This scheme
follows the latest draft, in which features are flagged compulsory.
We also add several helper functions.
Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We want to use this to handle the simple description for channel_type.
It also needs to handle variable-size types (just like subtypes).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If there's a rate-card for liquidity, we don't know about it until
after startup (the plugin *should* call us at init to tell us what their
current rates are)
This mainly helps our CI under valgrind, which starts a fresh instance
and immediately calls the invoice command. This can cause the topology
plugin to try to access the gossmap file before it's created.
We can also move the gossmap reading in topology to init time.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This involves removing some fields from the now-misnamed routing.h
datastructures, and various internal messages.
One non-obvious change is to our "keepalive" logic which refreshes
channels every 13 days: instead of using the 'enabled' flag on the
last channel broadcast to decide whether to refresh it, we use the
local connected status directly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is better than using the previous "keep statting the file" approach,
since we can also tell you how long the replacement is, to avoid a
gratitous load.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They're not defined to be, though we've not seen this on Linux (testing
showed that it is page-level atomic, which means it can still happen across
page boundaries though!). This was pointed out by whitslack in
https://github.com/ElementsProject/lightning/issues/4288
In practice, this just means not complaining when it happens, and also
not trying to get tricky to use it on MacOS (we can safely seek & write,
since we're single-threaded).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Removed bogus UNUSUAL log about gossip_store 'short test'.
If gossip_store is an incorrect version, we will recreate it: with an incorrect
version! This means we never get persistent gossip, *and* the pay plugin will
fail to map the gossip_store. Everyone will be sad.
Debugged-by: Matt Whitlock
Fixes: #4376Fixes: #4288
Typing-by: Rusty Russell <rusty@rustcorp.com.au>
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag
Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
We move over to the new "warning" paradigm, instead of using
an "rbf_fail" message.
Every failure is either a warning or an error; on warnings we
hang up and reconnect later, effectively resetting the state.
We assume if they set this to 0 (which nobody did previously), they're
using it as a modern flag and use it to indicate when they're
finished. Otherwise, we count how many blocks they've sent and use
that to determine whether they've finished.
See: https://github.com/lightningnetwork/lightning-rfc/pull/826
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we use `sync_complete` for gossip range query replies, with detection for older spec nodes.
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
This takes from the draft spec at https://github.com/lightningnetwork/lightning-rfc/pull/834
Note that if this draft does not get included, the peer will simply
ignore the warning message (we always close the connection afterwards
anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now report the new (draft) warning message.
We should actually be including this (as it may define _GNU_SOURCE
etc) before any system headers. But where we include <assert.h> we
often didn't, because check-includes would complain that the headers
included it too.
Weaken that check, and include config.h in C files before assert.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We fix up the test by using pay, instead of sendpay (and making pay log
the expected message).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: sendpay no longer extracts updates from errors, the caller should do it from the `raw_message`.
This overcomes the internal spam filter on updates, which can be useful
if we're actually trying to send through such a node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: always accept channel_updates from errors, even they'd otherwise be rejected as spam.
Fixes: #4300
Don't include exp directly, use an ifdef in common/bolt12
(like we do for peer and onion wiregen files).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that this also changes so the feature is not represented in channels,
reflecting the recent drafts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-onion-messages` enables send, receive and relay of onion messages.
The previous onion_message code required a confirmed, not-shutting-down
channel, not just a connection. That's overkill; plus before widespread
adoption we will want to connect directly as a last resort.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!
Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Avoids much cut & paste. Some tests don't need any of it, but most
want at least some of this infrastructure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We tried to send an absurdly-long message, because our limit
was really large, as we were subtracting the tlv length, not
the tlv overhead.
In addition, we were treating the array as a tal object, which
it isn't if the offset is non-zero.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We used to create the entire reply, the if it was too big, split in
half and retry.
Now that the main network is larger, this always happens with a full
request, which is inefficient.
Instead, produce a reply assuming no compression, then compress as a
bonus. This is simpler and more efficient, at cost of sending more
packets.
I also renamed an internal dev var to make it clearer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's not (yet?) compulsory to have the timestamps, but handing them around
together makes sense (a missing timestamp has the same effect as a zero
timestamp).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The spec (since d4bafcb67dcf1e4de4d16224ea4de6b543ae73bf in March
2020) requires that reply_channel_range be in order (and all
implementations did this anyway).
But when I tried this, I found that LND doesn't (always) obey this,
since don't divide on block boundaries. So we have to loosen the
constraints here a little.
We got rid of the old LND compat handling though, since everyone should
now be upgraded (there are CVEs out for older LNDs).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Support for receiving full gossip from ancient LND nodes.
Instead of a boutique message, use a "real" channel_announcement for
private channels (with fake sigs and pubkeys). This makes it far
easier for gossmap to handle local channels.
Backwards compatible update, since we update old stores.
We also fix devtools/dump-gossipstore to know about the tombstone markers.
Since we increment our channel_announce count for local channels now,
the stats in the tests changed too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.
This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
