Commit Graph

3598 Commits

Author SHA1 Message Date
Rusty Russell
36a2491a89 json: fix up msat amounts in non-_msat fields.
We had json_add_amount_msat_only(), which was designed to be used to
print out msat fields, if we had sats.

However, we misused it, so split it into the three different cases:
1. json_add_amount_sat_msat: We are using it correctly, with a field called
   xxx_msat.
2. json_add_amount_sats_deprecated: We were using it wrong, so deprecate
   the old field and create a new one which does end in _msat.
3. json_add_sats: we were using it to hand sats as a JSON parameter to an
   interface, where "XXXsat".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `rbf_channel` and `openchannel2` hooks `their_funding` (use `their_funding_msat`)
Changelog-Deprecated: Plugins: `openchannel2` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `funding_satoshis` (use `funding_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `channel_reserve_satoshis` (use `channel_reserve_msat`)
Changelog-Deprecated: Plugins: `channel_opened` notification `amount` (use `funding_msat`)
Changelog-Deprecated: JSON-RPC: `listtransactions` `msat` (use `amount_msat`)
Changelog-Deprecated: Plugins: `htlc_accepted` `forward_amount` (use `forward_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell
c5b032598e lightningd: fix outgoing IO logging for JSONRPC.
Changelog-Fixed: lightningd: `log-level` `io` shows JSONRPC output, as well as input.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
pn
83c31f548f log: Add termination to log level 2022-06-19 16:42:11 +09:30
pn
47a7b4a55b log: Add termination to prefix log 2022-06-19 16:42:11 +09:30
fiatjaf
1eaec223b7 expose short_channel_id and htlc id to htlc_accepted.
Changelog-Added: Plugins: `htlc_accepted` now exposes the `short_channel_id` for the channel from which that HTLC is coming from and the low-level per-channel HTLC `id`, which are necessary for bridging two different Lightning Networks when MPP is involved.
2022-06-18 14:53:32 +09:30
Michael Schmoock
475e4c9bd9 jsonrpc: adds optional remote_addr to listpeers
Changelog-Added: jsonrpc: adds optional `remote_addr` to listpeers
2022-06-17 20:30:16 +09:30
Michael Schmoock
32c4540fc0 jsonrpc: adds dynamicaly detected IP addresses to getinfo
Changelog-Fixed: JSON-RPC: Adds dynamically detected public IP addresses to `getinfo`
2022-06-17 20:30:16 +09:30
Michael Schmoock
de9bc172de connect: adds nodeid to remote_addr log message 2022-06-17 20:30:16 +09:30
Brian Barto
24b02c33cc
lightning-cli plugin start - Assume default relative path
When starting a plugin, if the plugin path cannot be found in
absolute context, assume it is a relative path to the default
plugins dir. As a result, the following now works when my_plugin.py
is installed in the default plugins dir:

lightning-cli plugin start my_plugin.py

Also update the plugin documentation to reflect that the use of a
relative path is now available.

Changelog-Added: plugin start RPC subcommand now assumes relative path to default plugins dir if the path is not found in absolute context. i.e. lightning-cli plugin start my_plugin.py

[ Squashed two commits into one -- RR ]
2022-06-17 14:18:48 +09:30
Rusty Russell
7c8dc62035 channeld: take over gossip_rcvd_filter.c and is_msg_gossip_broadcast.
channeld is the only user of these functions, since it now streams
all gossip itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-17 14:14:02 +09:30
Jon Griffiths
572942c783 psbt: use DER encoded + sighash byte for PSBT_IN_PARTIAL_SIG items
Per BIP-0171, the signature map is of pubkey to "The signature as would
be pushed to the stack from a scriptSig or witness".

Fixes 5298

Changelog-Fixed: PSBT: Fix signature encoding to comply with BIP-0171.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2022-06-09 18:28:35 +02:00
Rusty Russell
f078e54e98 lightningd: remove various deprecated JSON fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: removed `listtransactions` `outputs` `satoshis` field (deprecated v0.10.1)
Changelog-Removed: JSON-RPC: removed `listpeers` `channels` deprecated fields (deprecated v0.10.1)
Changelog-Removed: JSON-RPC: removed `listpeers` `channels` `closer` now omitted, rather than `null` (deprecated v0.10.1)
2022-05-18 10:15:36 +09:30
Rusty Russell
8b62e2584f connectd: remove enable-autotor-v2-mode option
Changelog-Removed: lightningd: removed `enable-autotor-v2-mode` option (deprecated v0.10.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-05-18 10:15:36 +09:30
Rusty Russell
98f64fb623 lightningd: don't crash listpeers if we're opening DF channel.
We call out to connectd to activate the peer, and while we do that,
channel->owner is NULL.  A better pattern would be to set up the unsaved
channel once connectd has given us the peer, but this works for now.

Fixes: #5204
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-21 20:45:32 +09:30
Rusty Russell
37e8d2fb0f connectd: disable advertizement of WEBSOCKET addresses.
This seems to prevent broad propagation, due to LND not allowing it.  See
	https://github.com/lightningnetwork/lnd/issues/6432

We still announce it if you disable deprecated-apis, so tests still work,
and hopefully we can enable it in future.

Fixes: #5196
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Protocol: disabled websocket announcement due to LND propagation issues
2022-04-21 06:13:55 +09:30
Rusty Russell
f0dc028fa9 lightningd: fix overzealous check in htlc_out_check:
```
+11.668971802 lightningdBROKEN: backtrace: lightningd/log.c:821 (fatal_vfmt) 0x558c893c997f
+11.668978165 lightningdBROKEN: backtrace: lightningd/log.c:829 (fatal) 0x558c893c9a30
+11.668984935 lightningdBROKEN: backtrace: lightningd/htlc_end.c:87 (corrupt) 0x558c893b9b7d
+11.668991262 lightningdBROKEN: backtrace: lightningd/htlc_end.c:205 (htlc_out_check) 0x558c893ba352
+11.669016705 lightningdBROKEN: backtrace: lightningd/peer_htlcs.c:1471 (check_already_failed) 0x558c893ea9c9
+11.669023345 lightningdBROKEN: backtrace: lightningd/peer_htlcs.c:1575 (onchain_failed_our_htlc) 0x558c893eb098
+11.669043122 lightningdBROKEN: backtrace: lightningd/onchain_control.c:438 (handle_onchain_htlc_timeout) 0x558c893cd6ec
+11.669049818 lightningdBROKEN: backtrace: lightningd/onchain_control.c:548 (onchain_msg) 0x558c893cdbdc
+11.669056372 lightningdBROKEN: backtrace: lightningd/subd.c:556 (sd_msg_read) 0x558c893fa532
+11.669063030 lightningdBROKEN: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x558c8948e3cd
+11.669069093 lightningdBROKEN: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x558c8948ef9e
+11.669075470 lightningdBROKEN: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x558c8948efdc
+11.669081900 lightningdBROKEN: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x558c894912a8
+11.669087916 lightningdBROKEN: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x558c893c0966
+11.669094531 lightningdBROKEN: backtrace: lightningd/lightningd.c:1181 (main) 0x558c893c6bf9
```

Fixes: #5191
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-20 16:30:05 +09:30
Rusty Russell
151d009435 lightningd: remove over-zealous assert.
This was hit on my node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-13 10:32:29 +09:30
Michael Dance
f067e8c909 Changed external/libwally-core to test_build_fix
Combined with the following commit which is required to
update against changed libsecp256k1 APIs:

Updated deprecated function calls
2022-04-12 15:17:52 +09:30
Rusty Russell
836c1b805b doc: update c-lightning to Core Lightning almost everywhere.
Mostly comments and docs: some places are actually paths, which
I have avoided changing.  We may migrate them slowly, particularly
when they're user-visible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-07 06:53:26 +09:30
Michael Schmoock
21fd3b22d4 config: adds htlc_minimum_msat htlc_maximum_msat announce_discovered_ip
This adds config and commandline options for htlc_min_msat, htlc_max_msat and
announce_discovered_ip. The default is 0msat for htlc_min_msat, unlimited for
htlc_max_msat and enabled for announce_discovered_ip.

The announce_discovered_ip gets the disable commandline switch --disable-ip-discovery

Changelog-added: Config options for htlc_min_msat, htlc_max_msat and announce_discovered_ip.
2022-04-05 05:45:04 +09:30
Rusty Russell
e47786da04 pay/sendpay: also store description in case bolt11 uses description_hash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-04 20:26:04 +09:30
Rusty Russell
9784fa816f lightningd: deprecate invoice expiry suffixes.
Makes types harder, and I've never personally used them.

Changelog-Deprecated: JSON-RPC: `invoice` `expiry` no longer allowed to be a string with suffix, use an integer number of seconds.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:46:01 +10:30
Rusty Russell
7f89763f9e sendpay: add payment_metadata argument.
And document the missing arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
4718ee076c lightningd: metadata received support (log and decline).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
e01abf0b34 bolt11: support payment_metadata.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
2526e804f7 doc: big BOLT update to incorporate warnings language.
We do this (send warnings) in almost all cases anyway, so mainly this
is a textual update, but there are some changes:

1. Send ERROR not WARNING if they send a malformed commitment secret.
2. Send WARNING not ERROR if they get the shutdown_scriptpubkey wrong (vs upfront)
3. Send WARNING not ERROR if they send a bad shutdown_scriptpubkey (e.g. p2pkh in future)
4. Rename some vars 'err' to 'warn' to make it clear we send a warning.

This means test_option_upfront_shutdown_script can be made reliable, too,
and it now warns and doesn't automatically close channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
9f06a59e3c shutdown: don't allow shutdown to p2pkh or p2sh addresses for anchor outputs.
This doesn't have an effect now (except in experimental mode), but it
will when we support anchors.  So we deprecate the use of those in the
close command too.

For experimental mode we have to avoid using p2pkh; adapt that test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `shutdown` no longer allows p2pkh or p2sh addresses.
2022-04-02 09:40:18 +10:30
Rusty Russell
3b536400f9 specs: update CSV files.
Regenerate from current BOLTS via `make extract-bolt-csv`

1. The remote_addr field was added manually into peer_wire.csv: this
   needs to be a patch otherwise it vanishes on regen.
2. We never brought into the channel_disabled fields, because it was
   too much hassle (we never actually generate this!).  Do it now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
9bddfc2048 connectd: take dev-suppress-gossip from gossipd.
Gossipd didn't actually suppress all gossip, resulting in a flake!
Doing it in connectd now makes much more sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 19:38:05 +10:30
Rusty Russell
2f7f7ec255 lightningd: add "style" to listforwards.
Suggested-by: @t-bast
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listforwards` has new entry `style`, currently "legacy" or "tlv".
2022-03-31 19:38:05 +10:30
Rusty Russell
116a77f1be lightningd: Restore forwarding of legacy onions.
Partial revert of 43a833e405
"lightningd: remove support for legacy onion format."; we restore the
ability to decode legacy onions for forwarding, but not to generate them.
(We don't accept them properly since making payment_secret compulsory
anyway, so no real change there!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: ... but we still forward legacy HTLC onions for now.
2022-03-31 19:38:05 +10:30
Rusty Russell
0b7f789291 lightningd: extra sanity checks and rescue attempts for missing HTLCs.
These trip when anything weird happens; turns out that we tell
onchaind about old htlcs (e.g. for penalties), so in that case we can
actually have it tell us about missing HTLCs which we no longer have
in memory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell
e616b4fff7 lightningd: add extra debugging for weird onchain htlc interactions.
This doesn't actually fix anything, but may shed more clues if it
happens again.

The broken() logs are overzealous, see next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell
ea7120a313 lightningd: add --dev-no-ping-timer to avoid ping response timeouts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell
5c949e3116 subd: make channel/peer own the subd.
We get some memleak reports because ld owns the subd, but once
the peer/channel is freed, there's no reference for the brief time
until the subd exits.

This happens for both opening and closingd.  For openingd, the
peer owns it, for others (including dualopend) the channel owns it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-30 06:27:52 +10:30
Rusty Russell
20392ae526 connectd: restore obs2 onion support.
I removed these prematurely: we *haven't* had a release since
introducing them!

This consists of reverting d15d629b8b
"plugins/fetchinvoice: remove obsolete string-based API." and
plugins/fetchinvoice: remove obsolete string-based
API. "onion_messages: remove obs2 support."

Some minor changes due to updated fromwire_tlv API since they
were removed, but not much.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: REVERT: Removed backwards compat with onion messages from v0.10.1.
2022-03-29 10:55:12 +10:30
Rusty Russell
aad4495f56 delinvoice: allow desconly arg to only remove the description.
Means that field is now optional in JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `delinvoice` has a new parameter `desconly` to remove description.
2022-03-29 10:04:16 +10:30
Rusty Russell
ccaf04d268 invoice: add deschashonly parameter.
LNURL wants this so they can include images etc in descriptions.

Replaces: #4892
Changelog-Added: JSON-RPC: `invoice` has a new parameter `deschashonly` to put hash of description in bolt11.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-29 10:04:16 +10:30
Rusty Russell
7abc491f4c pay: fix crash on invalid onion in reply in some cases.
fail can be NULL here.

Reported-by: https://github.com/shafemtol
Fixes: #5053
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:59:46 +10:30
Rusty Russell
a770f51d0e tools/generate_wire.py: make functions allocate the TLV.
Requiring the caller to allocate them is ugly, and differs from
other types.

This means we need a context arg if we don't have one already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
fa0c29f959 tools/generate_wire.py: tlvs should start with tlv_
No more "towire_offer", but "towire_tlv_offer".

This means we double-up on the unfortunately-named `tlv_payload` inside
the onion, but we should rename that in the spec when we remove
old payloads.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
7829f2eb06 onion_messages: remove obs2 support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Removed backwards compat with onion messages from v0.10.1.
2022-03-25 13:55:44 +10:30
Rusty Russell
726b6878d1 offers: import latest variant from draft.
In particular, this changes the name of a field in invoice_request:
`payer_signature` becomes simply `signature`.  So we allow both for
now, and send the old one unless deprecated_apis is disabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Sjors Provoost
560b090bd8 Add more warnings to fundchannel_start
[ Slight modifications to fix schema -- RR ]
2022-03-24 14:11:40 +10:30
Rusty Russell
f7aba31448 lightningd: allow 'style' 'legacy' for clboss compat (with deprecated_apis).
We keep the parameter around for future use (e.g. PTLC support?), but clboss
master still sets 'style' to "legacy".  Accept, but ignore it.

Reported-by: grubman on #c-lightning (IRC)
Changelog-Deprecated: JSON-RPC: `sendpay` `route` argument `style` "legacy" (don't use it at all, we ignore it now and always use "tlv" anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-24 13:19:10 +10:30
Vincenzo Palazzo
53806d1abd cli: make the command line more user friendly.
Also has to fix up tests.

Changelog-Fixed: cli doesn't required anymore to confirm the password if the `hsm_secret` is already encrypted.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30
Vincenzo Palazzo
58ce4cf71c cli: check if the pass and the confirmation pass match
Changelog-Fixed: check if the pass and the confirmation pass match from the command line

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30
Rusty Russell
9c978f2c63 lightningd: free peer->uc when openingd fails.
This avoids reuse which can cause confusion; the long-term fix is to
rewrite this to use real channels like dualfunding does.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
bcf3cef96c pytest: fix flake in test_connection.py::test_opener_feerate_reconnect
Make sure it sees disconnect before reconnect, otherwise the next command
fails since we're now disconnected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
07c4d39b75 memleak: fix double-free if we timeout.
... and then dualopend returns, and we access the fread leak_detect struct.

```
lightningd: FATAL SIGNAL 6 (version 065ca1e)
0x55ecd4be8145 send_backtrace
	common/daemon.c:33
0x55ecd4be81f1 crashdump
	common/daemon.c:46
0x7f200acab51f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7f200acff828 __pthread_kill_implementation
	./nptl/pthread_kill.c:44
0x7f200acff828 __pthread_kill_internal
	./nptl/pthread_kill.c:80
0x7f200acff828 __GI___pthread_kill
	./nptl/pthread_kill.c:91
0x7f200acab475 __GI_raise
	../sysdeps/posix/raise.c:26
0x7f200ac917b6 __GI_abort
	./stdlib/abort.c:79
0x55ecd4c6827f call_error
	ccan/ccan/tal/tal.c:93
0x55ecd4c68470 check_bounds
	ccan/ccan/tal/tal.c:165
0x55ecd4c684c2 to_tal_hdr
	ccan/ccan/tal/tal.c:175
0x55ecd4c68eb8 tal_free
	ccan/ccan/tal/tal.c:479
0x55ecd4b8bdd0 finish_report
	lightningd/memdump.c:138
0x55ecd4b8c115 leak_detect_req_done
	lightningd/memdump.c:201
0x55ecd4c68664 notify
	ccan/ccan/tal/tal.c:237
0x55ecd4c68b9e del_tree
	ccan/ccan/tal/tal.c:402
0x55ecd4c68bf3 del_tree
	ccan/ccan/tal/tal.c:412
0x55ecd4c68bf3 del_tree
	ccan/ccan/tal/tal.c:412
0x55ecd4c68f43 tal_free
	ccan/ccan/tal/tal.c:486
0x55ecd4c5751f io_close
	ccan/ccan/io/io.c:450
0x55ecd4bbce68 subd_shutdown_remaining
	lightningd/subd.c:911
0x55ecd4b8724a shutdown_subdaemons
	lightningd/lightningd.c:541
0x55ecd4b883cc main
	lightningd/lightningd.c:1207
0x7f200ac92fcf __libc_start_call_main
	../sysdeps/nptl/libc_start_call_main.h:58
0x7f200ac9307c __libc_start_main_impl
	../csu/libc-start.c:409
0x55ecd4b5cc54 ???
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
293cf3c2b2 connect: delay return until all subds ready.
We had some flakes because we returned from `connect`, but we hadn't started
subds yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
1e4149f18a lightningd: clean up peer connection handling a little.
Update the address and direction as soon as it connects not just when
we're about to make it active: we want this even if we don't have
an active channel, or if the connect hook rejects it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
4e8239fcfe lightningd: don't tell connectd to discard peer unless no subds left.
Otherwise it waits for subds to exit, but they don't.  Plus, the others
may still be talking!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
75596b3e0f lightningd: use a better channel if available to next hop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
debc1b90d3 lightningd: remove checks which prevent us from opening multiple channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now support opening multiple channels with the same peer.
2022-03-23 13:20:12 +10:30
Rusty Russell
21e1d68e3b lightningd: remove (most) functions to search channels by status.
This is generally verboten now, since there can be multiple.  There are a
few exceptions:

1. We sometimes want to know if there are *any* active channels.
2. Some dev commands still take peer id when they mean channel_id.
3. We still allow peer id when it's fully determined.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `close` by peer id will fail if there is more than one live channel (use `channel_id` or `short_channel_id` as id arg).
2022-03-23 13:20:12 +10:30
Rusty Russell
f85425d106 lightningd: don't assume a single channel per peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
ba1242af3e lightningd: add find_channel_by_scid
More efficient to search a known peer than the whole set.

Also, move find_channel_by_id() from channel_control.c into channel.c
where we'd expect it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
cb5dc48cab lightningd: make setchannelfee handle multiple channels per peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
b3438e9bba lightningd: associate connect commands with peer, not channel.
Sure, we want to connect (usually) because of an active channel, but
it's not specific to the channel itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
33abf93ec1 lightningd: rename activate_peers() to setup_peers().
Activate means a specific thing now (connectd said something), so avoid
confusing it with this function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
90be2cc104 lightningd: remove some "single active channel" assumptions.
Generally this means converting a lazy "peer_active_channel(peer)" call
into an explicit iteration.

1. notify_feerate_change: call all channels (ignores non-active ones anyway).
2. peer_get_owning_subd remove unused function.
3. peer_connected hook: don't save channel, do lookup and iterate channels.
4. In json_setchannelfee "all" remove useless call to peer_active_channel
   since we check state anyway, and iterate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
7de7b7be61 lightningd: use channel_id when a peer is activated.
Rather than intuiting whether this is a new channel / active channel,
use the channel_id.  This simplifies things and makes them explicit,
and prepares for multiple live channels per peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
fe9f391a93 connectd: tell lightningd the channel_id when we give it the active peer.
Now we always have it (either extracted from an unsolicited message,
or told to us by lightningd when it tells us it wants to talk), we can
always send it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
2bc58e2327 lightningd: always tell connectd the channel id.
This means lightningd needs to create the temporary one and tell it to
openingd/dualopend, rather than the other way around.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
57263a3eb2 lightningd: handle reestablish directly from connectd.
We don't need to hand it to channeld: it will read it!  We simply
need to tell it to expect it.

Similarly, openingd/dualopend will never see it, so remove that logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
2424b7dea8 connectd: hold peer until we're interested.
Either because lightningd tells us it wants to talk, or because the peer
says something about a channel.

We also introduce a behavior change: we disconnect after a failed open.
We might want to modify this later, but we it's a side-effect of openingd
not holding onto idle connections.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
77b1087cdf lightningd: move notification of disconnect into when we hear from connectd.
Simpler, and closes a potential race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
deecedb033 connectd: tell lightningd when disconnect is complete.
This avoids races in our tests where we assume it's sync (and is kind
of nicer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
6cc9f37cab connectd: handle connect vs closing race better.
We would return success from connect even though the peer was closing;
this is technically correct but fairly undesirable.  Better is to pass
every connect attempt to connectd, and have it block if the peer is
exiting (and retry), otherwise tell us it's already connected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
10e36e073c openingd: disconnect from peer when an error occurs.
openingd currently holds the connection to idle peers, but we're about
to change that: it will only look after peers which are actively
opening a connection.  We can start this process by disconnecting
whenever we have a negotiation failure.

We could stay connected if we wanted to, but that would be up to
connectd to decide.  Right now it's easier if we disconnect from any
idle peer once it's been active.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
16e9ba0361 connectd: fix confusing names.
The message from lightningd simply acknowleges that we are allowed to
discard the peer (because no subdaemons are talking to it anymore).
This difference becomes more stark once connectd holds on to idle
peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
eb203bf71e lightningd: clean up connect code.
1. The notification should be called every time.
2. channel can never be NULL, since it's tested above.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: `connect` notification now called even if we already have a live channel.
2022-03-23 13:20:12 +10:30
Rusty Russell
b99c04e605 lightningd: add explicit "connected" flag.
We currently intuit this by whether there's a subdaemon owning it.
But we're about to change the rules and allow connectd to hold idle
connections, so we need an explicit flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
5704653d4c setchannel: don't let them advertize htlc_maximum_msat larger than capacity.
And check for the obvious setting min > max.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
999c734bb5 setchannel: add minhtlc
Suggested by @m-schmook, I realized that if we append it later I'll
never get it right: I expect parameters min and max, not max and min!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: you can now alter the `htlc_minimum_msat` and `htlc_maximum_msat` your node advertizes.
2022-03-22 18:45:41 +10:30
Rusty Russell
f29890ed66 lightningd: check htlc_maximum_msat of channels for routehints.
We still use the channel hint here (as it's the only option), we just
warn about lack of capacity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
1751b1becc pytest: add checks that pay and getroute respect htlc_maximum_msat.
We need to add some, since our internal representations of
htlc_maximum_msat round up, and we need to disable mpp which succeeds
in getting a payment through by splitting.

We also allow dev_routes to suppress invoice routehints altogether.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
42f91ff2fa lightningd: deprecate setchannelfee, use setchannel in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `setchannelfee` (use `setchannel`).
2022-03-22 18:45:41 +10:30
Rusty Russell
66e264d6b3 lightningd: new setchannel command.
Based on setchannelfee, but expanded to allow setting max htlc amount (and others
in future?).

The main differences:
1. It doesn't change values which are not specified (that would be hard to
   add fields to!)
2. It says exactly what all values are in any potentially changed channels.

Changelog-Added: JSON-RPC: new `setchannel` command generalizes `setchannelfee`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
4fdcee9a11 channeld: generalize specific_feerates mesage into config_channel.
1. Add the htlc_max param.
2. Allow parameters to be unset, meaning "don't change".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
33bd251296 listpeers: show maximum_htlc_out_msat.
This is htlc_maximum_msat in BOLT 7 speak, but this name matches our existing
fields and is clearer in this context.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
f078a916e7 lightningd: enforce htlc_maximum_msat.
When we let them set it, this matters!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
3217dbe17f channeld: get htlc_maximum_msat from lightningd.
We used to calculate it ourselves.  Unfortunately this needs to
be done in several places, since new_channel() isn't used to fully
create a channel in the case of dual funding :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
6fdcc86f9d lightningd: store htlc_maximum_msat for channel in the db.
We currently don't allow setting it, but it's been requested.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
65be18d355 memleak: handle libwally allocations better.
Things allocated by libwally all get the tal_name "wally_tal",
which cost me a few hours trying to find a leak.

In the case where we're making one of the allocations the parent
of the others (e.g. a wally_psbt), we can do better: supply a name
for the tal_wally_end().

So I add a new tal_wally_end_onto() which does the standard
tal_steal() trick, and also changes the (typechecked!) name.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-21 10:19:50 +10:30
Rusty Russell
43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Rusty Russell
0db05f6e9c lightningd: opt_var_onion is now a compulsory feature.
We're about to drop support for legacy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Ken Sedgwick
36466af3eb hsmd: Add fields to hsmd_sign_{,remote_}commitment_tx for validating signers 2022-03-16 12:06:21 +10:30
Michael Schmoock
db95893aec lightningd: do not use remote_addr for always_use_proxy 2022-03-11 16:42:45 +10:30
Michael Schmoock
67fdc6f8ad gossipd: send updated node_annoucement remote_addr
This is the cheapest algo I came up with that simply checks that the
same `remote_addr` has been report by two different peers. Can be
improved in many ways:

 - Check by connecting to a radonm peers in the network
 - Check for more than two confirmations or a certain fraction
 - ...

Changelog-Added: Send updated node_annoucement when two peers report the same remote_addr.
2022-03-11 16:42:45 +10:30
Michael Schmoock
28b4e57974 lightningd: store recently reported remote_addr 2022-03-11 16:42:45 +10:30
Michael Schmoock
e92176248e chore: fix typo announcable -> announceable
"announcable" is a common misspelling of "announceable", see:

https://en.wiktionary.org/wiki/announcable
2022-03-11 16:42:45 +10:30
Ken Sedgwick
8f56f96801 hsmd: Add wallet index metadata to existing messages 2022-03-10 10:22:29 +10:30
Ken Sedgwick
3abe22213c hsmd: Augment call to hsmd_sign_withdrawal_tx with wallet index metadata 2022-03-10 10:22:29 +10:30
Rusty Russell
d7ffb712e5 dualopend: restore memleak calls.
And implement a timeout (20 seconds) just in case it's not listening.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-10 09:40:09 +10:30
Rusty Russell
00bb6f07d7 lightningd: simplify memleak code.
Instead of doing this weird chaining, just call them all at once and
use a reference counter.

To make it simpler, we return the subd_req so we can hang a destructor
off it which decrements after the request is complete.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-10 09:40:09 +10:30
Rusty Russell
b5a1715c2b connectd: also fail without a scary backtrace when listen fails.
For example, if you do:

```
./lightningd/lightningd --network=regtest --experimental-websocket-port=19846
```

Then you're trying to reuse the normal port as the websocket port, but this
only fails at *listen* time, when we activate connectd.  Catch this too.

Fixes incorrect fatal() message, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-05 15:48:03 +10:30
Rusty Russell
f1ed373c97 connectd: be more graceful when we an address is in use.
Aditya had this issue due to a config line, and the result was hard
to diagnose even for me.

It's now:

```
$ ./lightningd/lightningd --network=regtest --addr=:18444
2022-02-26T05:01:28.705Z **BROKEN** connectd: Failed to bind socket for 0.0.0.0:18444: Address already in use
```

Whereas before it doesn't even give the address it's trying to bind:

```
rusty@rusty-XPS-13-9370:~/devel/cvs/lightning (master)$ ./lightningd/lightningd --network=regtest --addr=:18444
lightning_connectd: Failed to bind on 2 socket: Address already in use (version v0.10.2-331-g86b83e4)
0x558a8b8d9a12 send_backtrace
	common/daemon.c:33
0x558a8b8e91e1 status_failed
	common/status.c:221
0x558a8b8c8e4f make_listen_fd
	connectd/connectd.c:1090
0x558a8b8c8f55 handle_wireaddr_listen
	connectd/connectd.c:1129
0x558a8b8c993d setup_listeners
	connectd/connectd.c:1312
0x558a8b8ca344 connect_init
	connectd/connectd.c:1517
0x558a8b8cbb57 recv_req
	connectd/connectd.c:1896
0x558a8b8d9f9f handle_read
	common/daemon_conn.c:31
0x558a8b9247c1 next_plan
	ccan/ccan/io/io.c:59
0x558a8b9253c9 do_plan
	ccan/ccan/io/io.c:407
0x558a8b92540b io_ready
	ccan/ccan/io/io.c:417
0x558a8b9276fe io_loop
	ccan/ccan/io/poll.c:453
0x558a8b8cbf36 main
	connectd/connectd.c:2033
0x7fe4d02940b2 ???
	???:0
0x558a8b8c285d ???
	???:0
0xffffffffffffffff ???
	???:0
2022-02-26T05:02:27.547Z **BROKEN** connectd: Failed to bind on 2 socket: Address already in use (version v0.10.2-331-g86b83e4)
2022-02-26T05:02:27.547Z **BROKEN** connectd: backtrace: common/daemon.c:38 (send_backtrace) 0x558a8b8d9a68
2022-02-26T05:02:27.547Z **BROKEN** connectd: backtrace: common/status.c:221 (status_failed) 0x558a8b8e91e1
2022-02-26T05:02:27.547Z **BROKEN** connectd: backtrace: connectd/connectd.c:1090 (make_listen_fd) 0x558a8b8c8e4f
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:1129 (handle_wireaddr_listen) 0x558a8b8c8f55
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:1312 (setup_listeners) 0x558a8b8c993d
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:1517 (connect_init) 0x558a8b8ca344
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:1896 (recv_req) 0x558a8b8cbb57
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: common/daemon_conn.c:31 (handle_read) 0x558a8b8d9f9f
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x558a8b9247c1
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x558a8b9253c9
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x558a8b92540b
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x558a8b9276fe
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: connectd/connectd.c:2033 (main) 0x558a8b8cbf36
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: (null):0 ((null)) 0x7fe4d02940b2
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: (null):0 ((null)) 0x558a8b8c285d
2022-02-26T05:02:27.548Z **BROKEN** connectd: backtrace: (null):0 ((null)) 0xffffffffffffffff
2022-02-26T05:02:27.548Z **BROKEN** connectd: STATUS_FAIL_INTERNAL_ERROR: Failed to bind on 2 socket: Address already in use
lightningd: connectd failed (exit status 242), exiting.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-05 15:48:03 +10:30
niftynei
9075c74e3d balance_snapshot: don't count unconfirmed utxos
This was causing journal_entries to show up in the accountant plugin,
since we don't emit events for unconfirmed events until they're actually
confirmed onchain.
2022-03-05 15:12:12 +10:30
niftynei
b33868f7c2 coin_mvt: only chain moves have a blockheight
channel moves don't have blockheights
2022-03-05 15:12:12 +10:30
niftynei
590f12145b listpeers: include the pushed_msat amount
Useful for accounting for missed/historical channel opens, to figure out
what the actual sat contribution from each peer is at a utxo level

Changelog-Added: JSONRPC: `listpeers` now includes a `pushed_msat` value. For leased channels, is the total lease_fee.
2022-03-05 15:12:12 +10:30
niftynei
7a00277b43 coin_mvt: mark coins destined for wallet w/ originating acct
Only shows up on delayed to us outputs, but nice to have anyway.

It's missing for channel index destined deposits, maybe nice to add at
some point in the future; right now you can figure out which close a
wallet deposit comes from via the channel close txid
2022-03-05 15:12:12 +10:30
niftynei
24a1c91045 coin_mvt: report the number of outputs on a channel close tx
The bookkeeper needs to know how many outputs to expect before we can
consider a channel resolved onchain.
2022-03-05 15:12:12 +10:30
niftynei
12cbf614cd coin_mvt: mark every event that already has a destination
Every event that's coming out of here that specifies a different place
to be deposited should be marked as originating from this channel's
 account.
2022-03-05 15:12:12 +10:30
niftynei
b30328a91f coin_mvt: rm unncessary if statement
We don't need to switch on this; if it's zero the below ops will be
effectiely no-ops.
2022-03-05 15:12:12 +10:30
niftynei
ce12d2b8a9 database: pull out database code into a new module
We're going to reuse the database controllers for the accounting plugin
2022-03-05 15:03:34 +10:30
niftynei
03c950bae8 db: decouple fatal reliance, have as impl defined function
`fatal` is defined in lightningd and has logfile dependencies etc.

Make it more generic by allowing declaration in the use file (wallet.c)
2022-03-05 15:03:34 +10:30
niftynei
b0829fc52a lightningd/Make: cleanup lightningd+wallet headers
There is no "wallet_lib_headers" variable in wallet/Makefile

Likewise, there were two "lightningd_headers", a couple of unused
variables and some other nonsene in lightningd/Makefile
2022-03-05 15:03:34 +10:30
Simon Vrouwe
9bbb52097e lightningd: fix parsing of option dev-force-features
It did not pick-up the last (bolt11) bits set.
2022-03-01 09:38:40 +10:30
Simon Vrouwe
905a85dc99 JSON-RPC: getinfo, add field our_features
All build flags and (experimental) options make it hard to find
out what features are supported or enabled.

And the undocumented `--list-features-only`, does not account for all
our featurebits, for example bit 55 (keysend).

Changelog-Added: JSON-RPC: `getinfo` result now includes `our_features` (bits) for various Bolt #9 contexts
2022-03-01 09:38:40 +10:30
Rusty Russell
b67329bec2 lightningd: don't leak address for printing.
Not really a leak, but it would last as long as the peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
William Casarin
ff84b3f773 json_add_invoice: fix crash if missing invstring
If this field is missing for whatever reason (weird db state?)
clightning will crash when listing invoices.

Signed-off-by: William Casarin <jb55@jb55.com>
2022-02-22 09:57:09 +10:30
Michael Schmoock
38e2abf68a peer_exchange: set, read and log remote_addr
Changelog-Added: Protocol: set remote_addr on init tlvs
2022-02-22 05:45:47 +10:30
Rusty Russell
d9b1e69243 dual_funding: don't steal inflight field in update_channel_from_inflight.
If we call update_channel_from_inflight *twice* with the same inflight, we
will get bad results.  Using tal_steal() here was a premature optimization:

```
Valgrind error file: valgrind-errors.496395
==496395== Invalid read of size 8
==496395==    at 0x22A9D3: to_tal_hdr (tal.c:174)
==496395==    by 0x22B4B5: tal_steal_ (tal.c:498)
==496395==    by 0x16A13D: update_channel_from_inflight (peer_control.c:1225)
==496395==    by 0x16A4C7: funding_depth_cb (peer_control.c:1299)
==496395==    by 0x182807: txw_fire (watch.c:232)
==496395==    by 0x182AA9: watch_topology_changed (watch.c:300)
==496395==    by 0x1290ED: updates_complete (chaintopology.c:624)
==496395==    by 0x129BF4: get_new_block (chaintopology.c:835)
==496395==    by 0x125EEF: getrawblockbyheight_callback (bitcoind.c:362)
==496395==    by 0x176ECC: plugin_response_handle (plugin.c:584)
==496395==    by 0x1770F5: plugin_read_json_one (plugin.c:690)
==496395==    by 0x1772D9: plugin_read_json (plugin.c:735)
==496395==  Address 0x89fbb08 is 24 bytes inside a block of size 104 free'd
==496395==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==496395==    by 0x22B193: del_tree (tal.c:421)
==496395==    by 0x22B461: tal_free (tal.c:486)
==496395==    by 0x16A123: update_channel_from_inflight (peer_control.c:1223)
==496395==    by 0x16A4C7: funding_depth_cb (peer_control.c:1299)
==496395==    by 0x182807: txw_fire (watch.c:232)
==496395==    by 0x182AA9: watch_topology_changed (watch.c:300)
==496395==    by 0x1290ED: updates_complete (chaintopology.c:624)
==496395==    by 0x129BF4: get_new_block (chaintopology.c:835)
==496395==    by 0x125EEF: getrawblockbyheight_callback (bitcoind.c:362)
==496395==    by 0x176ECC: plugin_response_handle (plugin.c:584)
==496395==    by 0x1770F5: plugin_read_json_one (plugin.c:690)
==496395==  Block was alloc'd at
==496395==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==496395==    by 0x22AC1C: allocate (tal.c:250)
==496395==    by 0x22B1DD: tal_alloc_ (tal.c:428)
==496395==    by 0x22B3A6: tal_alloc_arr_ (tal.c:471)
==496395==    by 0x22C094: tal_dup_ (tal.c:805)
==496395==    by 0x12B274: new_inflight (channel.c:187)
==496395==    by 0x136D4C: wallet_commit_channel (dual_open_control.c:1260)
==496395==    by 0x13B084: handle_commit_received (dual_open_control.c:2839)
==496395==    by 0x13B6AF: dual_opend_msg (dual_open_control.c:2976)
==496395==    by 0x1809FF: sd_msg_read (subd.c:553)
==496395==    by 0x218F5D: next_plan (io.c:59)
==496395==    by 0x219B65: do_plan (io.c:407)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
0c24334738 lightningd: clean up subds before freeing HTLCs.
Otherwise we get weird effects, as htlcs are being freed:

```
2022-01-26T05:07:37.8774610Z lightningd-1: 2022-01-26T04:47:48.770Z DEBUG   030eeb52087b9dbb27b7aec79ca5249369f6ce7b20a5684ce38d9f4595a21c2fda-chan#8: Failing HTLC 18446744073709551615 due to peer death
2022-01-26T05:07:37.8775287Z lightningd-1: 2022-01-26T04:47:48.770Z **BROKEN** 030eeb52087b9dbb27b7aec79ca5249369f6ce7b20a5684ce38d9f4595a21c2fda-chan#8: Neither origin nor in?
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
727b486d49 connectd: don't received useless peer fd if we're told to send final msg.
We don't need the connection to ourselves, just to free it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
ca08f27d54 connectd: remove second gossip fd.
Now we only send and receive gossip messages on this fd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
3c5d27e3e9 subdaemons: remove gossipd fd from per-peer daemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
1c71c9849b connectd: handle custom messages.
This is neater than what we had before, and slightly more general.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON_RPC: `sendcustommsg` now works with any connected peer, even when shutting down a channel.
2022-02-08 11:15:52 +10:30
Rusty Russell
8782d39476 connectd: handle onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
50eccb6a12 connectd: handle pings and pongs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON_RPC: `ping` now works with connected peers, even without a channel.
2022-02-08 11:15:52 +10:30
Rusty Russell
bba468a51c connectd: temporarily have two fds to gossipd.
We want to stream gossip through this, but currently connectd treats the
fd as synchronous.  While we work on getting rid of that, it's easiest to
have two fds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
4dc72dd829 dualopend: tell lightningd about new channel as soon as it's locked in.
Once we send funding_locked, gossipd could start seeing channel_updates
from the peer (which get sent so we can use the channel in routehints
even before it's announcable).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
e841e69b1b channeld: send channel updates and announcements via lightningd.
We're weaning per-peer daemons off having a direct gossipd connection.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
e8554c862a channeld: keep local copy of latest channel_update for errors.
Now we don't ask gossipd, but lightningd keeps channeld up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
5065bd6fc2 lightningd: use our cached channel_update for errors instead of asking gossipd.
We also no longer strip the type off: everyone handles both forms, and
Eclair doesn't strip (and it's easier!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
c34639252b lightningd: tell gossipd when we use the channel_update.
This way it can flush it if it was pending.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
51d23ffcd3 gossipd: infrastructure to tell lightningd about local channel updates.
We want it to keep the latest, so it can make its own error msgs without
asking us.  This installs (but does not use!) the message handler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
a8bed542f7 lightningd: start gossipd after channels are loaded.
This is in preparation for gossipd feeding us the latest channel_updates,
rather than having lightningd and channeld query gossipd when it wants
to send an onion error with an update included.

This means gossipd will start telling us the updates, so we need the
channels loaded first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
niftynei
9ebdb71ec0 lightningd/Make: fixup reference to wallet headers
There is no "wallet_lib_headers" variable in wallet/Makefile
2022-02-07 13:02:09 +10:30
niftynei
6349d89326 coin_mvt: tiny, dont import lightningd header 2022-02-07 13:02:09 +10:30
niftynei
081ef05dd1 listchannels: add funding_outnum to listchannels
Changelog-Added: JSONRPC: `listchannels` now includes the `funding_outnum`
2022-02-07 13:02:09 +10:30
Rusty Russell
de28bbd792 closingd, lightningd: use bitcoin_tx_2of2_input_witness_weight
This fixes lightningd's chronic weight underestimate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: closingd: more accurate weight estimation helps mutual closing near min/max feerates.
2022-01-27 12:22:36 +01:00
Rusty Russell
169c113d15 pytest: test that our closingd tx weight estimate is correct.
Compare against lightningd's, and the actual tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-27 12:22:36 +01:00
niftynei
ff4ae8b5f4 coin_moves: add an 'originating_account' field
If a coin move concerns an external account, it's really useful to know
which 'internal' account initiated the transfer.

We're about to add a notification for withdrawals, so we can use this to
track wallet pushes to outside addresses

Changelog-Added: JSONRPC: `coin_movement` to 'external' accounts now include an 'originating_account' field
2022-01-26 13:34:45 +10:30
Rusty Russell
4a4f85dd3f subd: fix waitpid properly.
lightningd would race with the subd destructor to do the waitpid(),
resulting in UNUSUAL log messages, but also us missing if a plugin
was killed via a signal.

We can also get rid of the gratuitous waitpid() in test_subdaemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-25 06:26:52 +10:30
Rusty Russell
beed4fcccc lightningd: fix backwards test in sigchld.
This would never trigger, since test was backward.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-25 06:26:52 +10:30
Rusty Russell
d042cbc344 patch dual-open-control-double-notify-fix.patch 2022-01-20 15:24:06 +10:30
Rusty Russell
d51fb5207a msg_queue: don't allow magic MSG_PASS_FD message for peers.
msg_queue was originally designed for inter-daemon comms, and so it has
a special mechanism to mark that we're trying to send an fd.  Unfortunately,
a peer could also send such a message, confusing us!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
39c93ee6e5 connectd: get addresses from lightningd, not gossipd.
It's weird to have connectd ask gossipd, when lightningd can just do it
and hand all the addresses together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
741f44725a patch lightningd-peer-fds.patch 2022-01-20 15:24:06 +10:30
Rusty Russell
6115ed02e8 subdaemons: don't stream gossip_store at all.
We now let gossipd do it.

This also means there's nothing left in 'struct per_peer_state' to
send across the wire (the fds are sent separately), so that gets
removed from wire messages too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
029d65cf2e connectd: serve gossip_store file for the peer.
We actually intercept the gossip_timestamp_filter, so the gossip_store
mechanism inside the per-peer daemon never kicks off for normal connections.

The gossipwith tool doesn't set OPT_GOSSIP_QUERIES, so it gets both, but
that only effects one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
7a514112ec connectd: do dev_disconnect logic.
As connectd handles more packets itself, or diverts them to/from gossipd,
it's the only place we can implement the dev_disconnect logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
9c0bb444b7 per_peer_state: remove struct crypto_state
Now that connectd does the crypto, no need to hand around crypto_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
71f736678f lightningd: make sure gossipd knows before we update blockheight.
Without this, we can get spurious failures from lnprototest, which
is waiting for lightningd to acknowledge the blockheight in getinfo:

```
2021-12-21T00:56:21.460Z DEBUG   lightningd: Adding block 109: 57a7bd3ade3a88a899e5b442075e9722ccec07e0205f9a913b304a3e2e038e26
2021-12-21T00:56:21.470Z DEBUG   lightningd: Adding block 110: 11a280eb76f588e92e20c39999be9d2baff016c3c6bac1837b649a270570b7dd
2021-12-21T00:56:21.479Z DEBUG   lightningd: Adding block 111: 02977fc9529b2ab4e0a805c4bc1bcfbff5a4e6577a8b31266341d22e204a1d27
2021-12-21T00:56:21.487Z DEBUG   lightningd: Adding block 112: 2402f31c5ddfc9e847e8bbfb7df084d29a5d5d936a4358c109f2f4cf9ea8d828
2021-12-21T00:56:21.496Z DEBUG   lightningd: Adding block 113: 5a561fe9423b4df33f004fc09985ee3ef38364d692a56a8b27ecbc6098a16d39
2021-12-21T00:56:21.505Z DEBUG   lightningd: Adding block 114: 4502f5ec23c89177872846848848322e8fa6c3fb6f5eb361194e4cd47596dfe9
2021-12-21T00:56:21.511Z DEBUG   02f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9-gossipd: Ignoring future channel_announcment for 109x1x0 (current block 108)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Michael Schmoock
019b31c522 options: only allow one DNS announcement 2022-01-02 16:01:38 +01:00
Rusty Russell
ae4669f77f common/coin_mvt: make it clear we're using Lightning HRP not BIP 173.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 14:36:55 +10:30
Rusty Russell
425a7af512 common/coin_mvt: clean up API a little.
1. tal_strndup(.., str, strlen(str)) == tal_strdup()
2. tal_strdup also takes(), so document that.
3. Avoid passing 'struct sha256' on the stack: use ptr.
4. Generally, structures shouldn't keep pointers to things they don't own.
   In this case, mvt->node_id.
5. Make payment_hash a pointer, since NULL is more natural than an all-zero
   hash.

And add NON_NULL_ARGS() to the functions; it's cumbersome, but make it
fairly clear what params are optional.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 14:36:55 +10:30