Commit Graph

1379 Commits

Author SHA1 Message Date
Rusty Russell
7745513c51 bolt12: change our payer_key calculation.
It was very tied to x-only keys; we could support it in a backwards
compatibility mode for a while, but getting refunds or proving old
pre-finalization invoices is not worth spending time on.

Changelog-EXPERIMENTAL: offers: old `payer_key` proofs won't work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
4e39b3ff3d hsmd: don't use point32 for bolt12, but use pubkeys (though still always 02)
This is the one place where we hand point32 over the wire internally, so
remove it.

This is also our first hsm version change!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
bed905a394 lightningd: use 33 byte pubkeys internally.
We still use 32 bytes on the wire, but internally don't use x-only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
987adb9718 Makefile: check that hsm_version.h changes if wire/hsmd_wire.csv contents does
Otherwise I know we'll miss it.  Simply check for a mention: we could well
change things multiple times within a single release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
fee9a7ce04 hsmd: introduce a simple API versioning scheme.
With the rise of external HSMs like VLS, this is no longer an
internal-only API.  Fortunately, it doesn't change very fast so
maintenance should not be a huge burden.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
2749b1f61e common/onion: remove old blinded payment handling.
It's been obsoleted and needs replacing; less confusing if we remove
it first.

Also, these fields are now present even without an expermintal build
(we'll control at runtime).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell
41ef85318d onionmessages: remove obsolete onion message parsing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell
342e330b56 doc: update references to old BOLTs repo.
This reveals that common/test/run-bolt12_merkle-json.c was broken!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-29 16:10:57 +09:30
Rusty Russell
f00cc23f67 sphinx: rename confusing functions, ensure valid payloads.
"sphinx_add_hop" takes a literal hop to include,
"sphinx_add_modern_hop" prepends the length.  Now we always prepend a
length, make it clear that the literal version is a shortcut:

* sphinx_add_hop -> sphinx_add_hop_has_length
* sphinx_add_modern_hop -> sphinx_add_hop

In addition, we check that length is actually correct!  This means
`createonion` can no longer create legacy or otherwise-invalid onions:
fix tests and update man page to remove legacy usage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `createonion` no longer allows non-TLV-style payloads.
2022-09-28 13:40:57 +02:00
Rusty Russell
8771c86379 common/onion: expunge all trace of different onion styles.
In particular, remove special routines to pull length: it's there,
take it and check it yourself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-28 13:40:57 +02:00
Rusty Russell
c8ad9e18a9 common/onion: remove all trace of legacy parsing.
We still have an "enum forward_style" for the database, where old-style
forwards can still exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer forward HTLCs with legacy onions.
2022-09-28 13:40:57 +02:00
Rusty Russell
1de4e46276 tests: add onion-test-vector from "BOLT 4: Remove legacy format, make var_onion_optin compulsory."
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-28 13:40:57 +02:00
Rusty Russell
f53155d93b BOLT: update to clarify HTLC tx amount calculation.
Simple quote update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell
bb49e1bea5 common: assume htlc_maximum_msat, don't check bit any more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell
253b25522b BOLT: update to version which requires option_channel_htlc_max.
We will now simply reject old-style ones as invalid.  Turns out the
only trace we could find is a channel between two nodes unconnected to
the rest of the network.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now require all channel_update messages include htlc_maximum_msat (as per latest BOLTs)
2022-09-24 15:22:27 +09:30
Rusty Russell
daa5269ea2 gossipd: bump gossip_store to indicate all channel_update have htlc_max.
And in the next patch, gossipd will no longer put new ones in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell
6338758018 gossmap: make API more robust against future changes.
Many changes to gossmap (including the pending ones!) don't actually
concern readers, as long as they obey certain rules:

1. Ignore unknown messages.
2. Treat all 16 upper bits of length as flags, ignore unknown ones.

So now we split the version byte into MAJOR and MINOR, and you can
ignore MINOR changes.

We don't expose the internal version (for creating the map)
programmatically: you should really hardcode what major version you
understand!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell
45cdfd2ff7 BOLT: update to fix gossip pruning quote.
Which I disagreed with, and has been fixed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Rusty Russell
3079afb024 lightningd: add delforward command.
Changelog-Added: JSON-RPC: `delforward` command to delete listforwards entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Christian Decker
75fe1c1a66 common: Add multiplication primitives for amount_msat and amount_sat
The scale variants weren't usable since they use `double` as a scale
factor, which in turn causes imprecisions.
2022-09-21 11:25:47 +02:00
Rusty Russell
5b58eda748 libplugin: mark the cmd notleak() whenever command_still_pending() called.
This is what we do in lightningd, which makes memleak much more forgiving:
you can hang temporaries off cmd without getting reports of leaks (also
when send_outreq called).

We remove all the notleak() calls in plugins which worked around this!
And avoid multiple notleak labels, since both send_outreq() and
command_still_pending() can be called multiple times.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell
701dd3dcef memleak: remove exclusions from memleak_start()
Add memleak_ignore_children() so callers can do exclusions themselves.

Having two exclusions was always such a hack!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell
3380f559f9 memleak: simplify API.
Mainly renaming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell
fd71dfc7f7 gossmap: optimize asserts().
They are surprisingly expensive!

Running `time ./plugins/renepay/test/run-not_mcf-gossmap gossip_store-sgl.rustcorp.com.au-2022-04-19 024b9a1fa8e006f1e3937f65f66c408e6da8e1ca728ea43222a7381df1cc449605 02ebb3b8a2316b3e876ea3f3d8124a3ab97f30b128f619608eb06b5251235dc2d9 10000000000 0.1`:

Before (-Og):
	real	0m1.495s
Before (no opt):
	real	0m2.552s

After (-Og):
	real	0m0.579s
After (no opt):
	real	0m1.061s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 10:18:55 +09:30
Rusty Russell
4cdb4167d2 gossmap: make local_addchan create private channel_announcement in correct order.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 10:18:55 +09:30
Rusty Russell
016e332304 gossmap: add functions to map index back to node/chan.
Useful if you want to store per-chan or per-node side-data in an array.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 10:18:55 +09:30
Rusty Russell
6a48ed9e82 gossmap: fail to get capacity of locally-added chans (don't crash!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 10:18:55 +09:30
Rusty Russell
2da5244e83 jsonrpc: make error codes an enum.
This allows GDB to print values, but also allows us to use them in
'case' statements.  This wasn't allowed before because they're not
constant terms.

This also made it clear there's a clash between two error codes,
so move one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: Error code from bcli plugin changed from 400 to 500.
2022-09-19 10:18:55 +09:30
Rusty Russell
7fa1364645 build: allow DEVELOPER builds with -Og and gcc 9.4.0
Removes some warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 10:18:55 +09:30
Rusty Russell
88354b79bd common: helper to get id field as a string.
We'll be doing this quite a bit, so provide common helper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
112115022c gossmap: don't crash if we see a duplicate channel_announce.
Apparently we had two private channel announcements (the !private assert
failed).  While this shouldn't happen, don't crash because of it.

Fixes: #5578
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: topology plugin could crash when it sees duplicate private channel announcements.
2022-09-14 13:44:59 -05:00
Alex Myers
4167fe8dd9 gossip_store: fix offset error
The gossip_store version byte was unaccounted for in the initial traversal
of gossip_store_end. This lead to an offset and a bogus message length
field. As a result, an early portion of the gossip_store could have been
skipped, potentially leading to gossip propagation issues downstream.

Fixes #5572 #5565

Changelog-fixed: proper gossip_store operation may resolve some previous gossip propagation issues
2022-09-14 13:14:32 +02:00
Rusty Russell
b208c0d8dd doc: upgrade to BOLTs 2ecc091f3484f7a3450e7f5543ae851edd1e0761
I disagree with this change, so I've commented and added a FIXME.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
5b7f14a7cb channeld/dualopend/lightningd: use channel_ready everywhere.
This alters the billboard, but that's a human-readable thing so not
noted in CHANGELOG.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listpeers` `status` now refers to "channel ready" rather than "funding locked" (BOLT language change for zeroconf channels)
Changelog-Added: JSON-RPC: `channel_opened` notification `channel_ready` flag.
Changelog-Deprecated: JSON-RPC: `channel_opened` notification `funding_locked` flag (use `channel_ready`: BOLTs namechange).
2022-09-12 09:34:52 +09:30
Rusty Russell
1b30ea4b82 doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready.  I change that everywhere, and try to fix up the
comments.

Also the `alias` field is called `short_channel_id`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
2022-09-12 09:34:52 +09:30
Rusty Russell
6c33f7db65 common: remove unused parameter "allow_deprecated" from parse_wireaddr_internal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
dc56b2a9ac connectd: better diagnostics on invalid gossip_store entries.
Should help diagnose https://github.com/ElementsProject/lightning/issues/5572
which hit the invalid csum on a >64MB entry, if it happens again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:04:55 +03:00
Rusty Russell
6fe570820e Remove general shadowed variables.
We shadow local variables in several places: generally, these changes
make the code clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-31 12:18:28 +03:00
Rusty Russell
7c9985fd9c channeld: correct reversed shutdown message in billboard.
It was backwards, which made #5496 confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: `listpeers` `status` "They've sent shutdown" and "We've sent shutdown" were backwards.
2022-08-19 11:04:37 -04:00
Rusty Russell
d5d4e7b019 common/features: add channel_type definition, neaten header.
Put all the normal spec-defined ones together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-08 11:49:56 -05:00
niftynei
30aa1d79fb bkpr: for zerconfs, we still wanna know you're opening a channel
We need a record of the channel account before you start sending
payments through it. Normally we don't start allowing payments to be
sent until after the channel has locked in but zeroconf does away with
this assumption.

Instead we push out a "channel_proposed" event, which should only show
up for zeroconfs.
2022-07-28 12:08:18 +09:30
niftynei
0617690981 coin_mvt/bkpr: add "stealable" tag to stealable outputs
If we expect further events for an onchain output (because we can steal
it away from the 'external'/rightful owner), we mark them.

This prevents us from marking a channel as 'onchain-resolved' before
all events that we're interested in have actually hit the chain.

Case that this matters:
Peer publishes a (cheating) unilateral close and a timeout htlc (which
we can steal).
We then steal the timeout htlc.

W/o the stealable flag, we'd have marked the channel as resolved when
the peer published the timeout htlc, which is incorrect as we're still
waiting for the resolution of that timeout htlc (b/c we *can* steal it).
2022-07-28 12:08:18 +09:30
niftynei
a45da63280 bkpr: pass the node_id over for channel_opens, add to account
it's nice to know what node your channel was opened with. in theory we
could use listpeers to merge the data after the fact, except that
channels disappear after they've been closed for a bit. it's better to
just save the info.

we print it out in `listbalances`, as that's a great place account level
information
2022-07-28 12:08:18 +09:30
niftynei
a3d82d5a01 bkpr: exclude non-wallet events in the balance snapshot
Anchor outputs are ignored by the clightning wallet, but we keep track
of them in the bookkeeper. This causes problems when we do the balance
checks on restart w/ the balance_snapshot -- it results in us printing
out a journal_entry to 'get rid of' the anchors that the clightning node
doesnt know about.

Instead, we mark some outputs as 'ignored' and exclude these from our
account balance sums when we're comparing to the clightning snapshot.
2022-07-28 12:08:18 +09:30
niftynei
282ab72e2d tests: valgrind barfing on uninitialized value
------------------------------- Valgrind errors --------------------------------
Valgrind error file: valgrind-errors.493330
==493330== Conditional jump or move depends on uninitialised value(s)
==493330==    at 0x154051: opt_add_addr_withtype (options.c:275)
==493330==    by 0x154406: opt_add_announce_addr (options.c:302)
==493330==    by 0x2696E6: parse_one (parse.c:121)
==493330==    by 0x25CFB5: opt_parse (opt.c:228)
==493330==    by 0x155DB6: handle_opts (options.c:1413)
==493330==    by 0x127317: main (lightningd.c:994)
==493330==
{
   <insert_a_suppression_name_here>
   Memcheck:Cond
   fun:opt_add_addr_withtype
   fun:opt_add_announce_addr
   fun:parse_one
   fun:opt_parse
   fun:handle_opts
   fun:main
}
--------------------------------------------------------------------------------
Leaving base_dir /tmp/ltests-iyf2dw3n intact, it still has test sub-directories with failure details: ['test_announce_dns_without_port_1']
====================================== short test summary info ======================================
ERROR tests/test_gossip.py::test_announce_dns_without_port - ValueError:
2022-07-26 15:11:30 -07:00
Rusty Russell
5979a7778f lightningd: expand exit codes for various failures.
Most unexpected ones are still 1, but there are a few recognizable error codes
worth documenting.

Rename the HSM ones to put ERRCODE_ at the front, since we have non-HSM ones
too now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-20 19:28:33 +09:30
Vincenzo Palazzo
1d671a2380 rpc: checkmessage return an error if pubkey is not found
Returning an warning message when the pub key is not specified and there is no node in the graph.

We try to help people that use core lightning as a signer and nothings else.

Changelog-Deprecated: rpc: checkmessage return an error when the pubkey is not specified and it is unknown in the network graph.
2022-07-19 17:55:31 +02:00
Rusty Russell
9cff125590 common/gossip_store: fix leak on partial read.
Very unusual, but it can happen, and we don't free:

```
lightningd-1 2022-07-12T04:21:22.591Z DEBUG   gossipd: REPLY WIRE_GOSSIPD_DEV_MEMLEAK_REPLY with 0 fds
lightningd-1 2022-07-12T04:21:22.645Z **BROKEN** connectd: MEMLEAK: 0x55e73123d008
lightningd-1 2022-07-12T04:21:22.645Z **BROKEN** connectd:   label=common/gossip_store.c:92:u8[]
lightningd-1 2022-07-12T04:21:22.645Z **BROKEN** connectd:   backtrace:
lightningd-1 2022-07-12T04:21:22.645Z **BROKEN** connectd:     ccan/ccan/tal/tal.c:442 (tal_alloc_)
lightningd-1 2022-07-12T04:21:22.645Z **BROKEN** connectd:     ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)
lightningd-1 2022-07-12T04:21:22.645Z **BROKEN** connectd:     common/gossip_store.c:92 (gossip_store_next)
lightningd-1 2022-07-12T04:21:22.645Z **BROKEN** connectd:     connectd/multiplex.c:433 (maybe_from_gossip_store)
lightningd-1 2022-07-12T04:21:22.645Z **BROKEN** connectd:     connectd/multiplex.c:856 (write_to_peer)
lightningd-1 2022-07-12T04:21:22.646Z **BROKEN** connectd:     ccan/ccan/io/io.c:59 (next_plan)
lightningd-1 2022-07-12T04:21:22.646Z **BROKEN** connectd:     ccan/ccan/io/io.c:407 (do_plan)
lightningd-1 2022-07-12T04:21:22.646Z **BROKEN** connectd:     ccan/ccan/io/io.c:423 (io_ready)
lightningd-1 2022-07-12T04:21:22.646Z **BROKEN** connectd:     ccan/ccan/io/poll.c:453 (io_loop)
lightningd-1 2022-07-12T04:21:22.646Z **BROKEN** connectd:     connectd/connectd.c:2083 (main)
lightningd-1 2022-07-12T04:21:22.646Z **BROKEN** connectd:     ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main)
lightningd-1 2022-07-12T04:21:22.646Z **BROKEN** connectd:     ../csu/libc-start.c:392 (__libc_start_main_impl)
lightningd-1 2022-07-12T04:21:22.646Z **BROKEN** connectd:   parents:
```
2022-07-18 20:50:04 -05:00
Rusty Russell
430d6521a0 common/daemon_conn: add function to read an fd.
We never needed this before.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
Rusty Russell
eff53495db lightningd: make "is peer connected" a tristate.
First, connectd tells us the peer has connected, and we call the connected hook,
and if it says it's fine, we are actually connected and we fire off notifications.

Of course, we could be disconnected while in the connected hook, and that would
mean we tell people about a connection which is no longer current.

Make this clear with a tristate: if we're not marked disconnected by
the time the hooks finish, we're good.  It also gives us a cleaner
"connect" command return when we connected but disconnected before
processing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00