Commit Graph

932 Commits

Author SHA1 Message Date
Rusty Russell
1c230eee86 test: hack in test for (now-obsolete!) test vector Thomas emailed me.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
5361107c9c gossipd: handle receipt of modern onion message.
And wire it through to the hook; update the plugins to recognize
modern vs obs2 onions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
e7b263304e lightningd: Send updated onion spec messages.
It's very similar to the previous, but there are a few changes:

1. The enctlv fields are numbered differently.
2. The message itself is a different number.

The onionmsg_path type is the same, however, so we keep that constant
at least.

The result is a lot of cut & paste, but we will delete the old one
next release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
b3af5f5a2c spec: import latest onionmessage spec, based on routeblinding.
This is from 6e99c5feaf60cb797507d181fe583224309318e9

We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513.  We don't handle it until the next patch.

Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.

We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
1ec6346f3d common: rename current onion message structures to obs2_.
Yes, we changed the spec again.  Hopefully for the last time!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
b74848f6f6 common: remove support for pre v0.10.2 onionmessages.
Temporarily disable sendpay_blinding test which uses obsolete onionmsg;
there's still some debate on the PR about how blinded HTLCs will work.

Changelog-EXPERIMENTAL: onionmessage: removed support for v0.10.1 onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
6c9b752751 memleak: make notleak use the tal name instead of a boutique struct.
This lets us mark it directly.

Get rid of long-unused "notleaks" member of struct lightningd too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 10:38:13 +10:30
Rusty Russell
57328fe59e tests: use common_setup/common_shutdown to avoid leaks.
It also does more checks (like taken() checks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
9c254c8387 gossipd: when we refresh channel, don't override pending updates.
We can have an update pending because it's too fast, but
refresh_local_channel is supposed to make sure we're up-to-date, so
force immediate application in that case.

Otherwise, we call update_local_channel at the bottom which frees the
pending update.  This can mean that we miss a change in fees, for example.

Changelog-Fixed: errors: Errors returning a `channel_update` no longer return an outdated one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-21 20:05:54 +02:00
Rusty Russell
1c85b27b4c gossipd: remove ping/pong handling
To minimize the diffs, we #if 0 the code.  We'll reenable it once
channeld is ready.

We also temporarily disable the ping tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell
6c8c6a7c7d gossipd: handle modern onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell
f9a21d9fc9 lightningd: handle modern onion termination.
This adds a new hook: onion_message_ourpath for when we know a message
came in via a blinded path we created.  The onion_message_blinded hook
is now called for all other messages, since all messages are now
blinded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell
79e09b92ef Makefile: remove generated files.
By popular merge-hell demand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
2021-09-22 15:25:14 +02:00
Rusty Russell
f2a4bd6ad8 wire: import new onion message spec.
One change from the obsolete version handling, gossipd will no longer send
forwarding onion msgs to lightningd, but will forward it directly.
That was the effect before, anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell
191798bb56 gossipd: rename onionmsg messages to lightningd to obs_onionmsg.
The new ones are slightly different.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
e3ae7883bb channeld: rename onion_message to obs_onion_message.
This splits the existing old-spec pathways to prepare for the
new ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
8b694907e6 gossipd: fix blinded onion forwarding.
We never tested that we can correctly unwrap on the next step after
unblinding: it failed because we mangled the onion in place!  Fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
00a0d09340 tools/check-includes.sh: test that c files include their .h files.
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly).  Make it a requirement,
either of form "foo.h" or <dir/foo.h>.

The noise is the change to our print templates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
1b8551923d tools: handle generating print templates for CSV without tlvs, messages
We want to use this to handle the simple description for channel_type.

It also needs to handle variable-size types (just like subtypes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
a5fee67b91 common/memleak: take over dump_memleak(), allow print pointer.
This will let plugins use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
ceb40dea38 lightningd: don't turn zero-length tlv fields into NULL.
Fixes: #4667
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
Rusty Russell
87239dc691 gossipd: de-duplicate code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-20 13:28:38 -04:00
niftynei
8819278a88 lease rates: helpers for parsing, serializing + equating lease_rates
We're gonna need them
2021-07-20 13:28:38 -04:00
niftynei
265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
niftynei
9e839b177c gossipd: turn off lease offers if the rates are empty
If received lease rates are empty (all zeroes), turn them off
2021-07-20 13:28:38 -04:00
niftynei
876ff6821e setleaserates: new RPC to pass in lease rates
Changelog-Experimental: JSON-RPC: new RPC `setleaserates`, for passing in the rates to advertise for a channel lease (option_will_fund)
2021-07-20 13:28:38 -04:00
niftynei
48d244c304 gossipd: signal whether node_ann is updated, are we starting up?
If there's a rate-card for liquidity, we don't know about it until
after startup (the plugin *should* call us at init to tell us what their
current rates are)
2021-07-20 13:28:38 -04:00
niftynei
55f5a5caba gossip/liquidity-ad: node_ann comparison, optional TLV check
Correctly mark whether or not the TLV's are the same/different, given two
node announcements
2021-07-20 13:28:38 -04:00
niftynei
49bae2a873 lease_rates: parse them out of node_announcements
When we get a node announcement out, we can now pull out its offer
characteristic
2021-07-20 13:28:38 -04:00
niftynei
5b3c02f401 liquidity-ads: import from spec
Import the wires from spec. Here we go!
2021-07-20 13:28:38 -04:00
Christian Decker
b4ead97517 tlv: Allow passing some extra types to accept when parsing the stream 2021-06-26 10:55:13 +09:30
Rusty Russell
75720ad0e1 lightningd: wait for gossipd to finish initalizing before starting plugins.
This mainly helps our CI under valgrind, which starts a fresh instance
and immediately calls the invoice command.  This can cause the topology
plugin to try to access the gossmap file before it's created.

We can also move the gossmap reading in topology to init time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell
7e7ab4cb3b gossipd: remove routing, listchannels and listnodes infrastructure.
This involves removing some fields from the now-misnamed routing.h
datastructures, and various internal messages.

One non-obvious change is to our "keepalive" logic which refreshes
channels every 13 days: instead of using the 'enabled' flag on the
last channel broadcast to decide whether to refresh it, we use the
local connected status directly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell
8810ce7241 gossipd/test: remove routing tests.
We're about to remove routing from gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell
e734097125 common/wireaddr: fromwire_wireaddr_array helper.
This was previously internal to gossipd and was called "read_addresses".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-16 10:29:17 +09:30
Rusty Russell
2fea448498 gen/impl_template: fix generation of singleton varsize elements.
And as Lisa requested, add testcases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell
e2f225e4cd lightningd: use connectd to send the final error message, not openingd/dualopend
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-03 16:16:34 +09:30
Rusty Russell
f7adbd5d58 EXPERIMENTAL: import spec for quiescence.
Imported from commit b96218b06b68cf349457b282f05d48ebd89c7273

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-01 12:08:51 -05:00
Rusty Russell
2bb365a931 common/route: route_from_dijkstra returns route_hop array.
This is what (most) callers actually want, so unify it into one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
e531a38963 gossipd / plugin: clean up names in struct route_hop.
We're going to unify them, but the names are not the normal ones.

Fix that first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
9dadcc858b common/gossip_store: avoid fd pass for new store, use end marker.
This is also simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
0d4f014021 gossip_store: create end marker for EOF.
This is better than using the previous "keep statting the file" approach,
since we can also tell you how long the replacement is, to avoid a
gratitous load.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
7b56b2b8ce gossipd: don't start listening to connectd until we get init message.
This was always a theoretical race, but with the next change we actually
hit it under valgrind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-24 13:56:58 +09:30
niftynei
51c398b4c3 gossip: allow us to only look at 1 block
Breaks a lnprototest otherwise!
2021-04-08 10:34:14 +09:30