Commit Graph

1236 Commits

Author SHA1 Message Date
Rusty Russell
2da5244e83 jsonrpc: make error codes an enum.
This allows GDB to print values, but also allows us to use them in
'case' statements.  This wasn't allowed before because they're not
constant terms.

This also made it clear there's a clash between two error codes,
so move one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: Error code from bcli plugin changed from 400 to 500.
2022-09-19 10:18:55 +09:30
Rusty Russell
7fa1364645 build: allow DEVELOPER builds with -Og and gcc 9.4.0
Removes some warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 10:18:55 +09:30
Rusty Russell
e8ef42b741 plugin: wire JSON id for commands which caused hooks to fire.
Most obvious one is the "connect" hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
ea7903f69a lightningd: trace JSON id prefixes through sendrawtx.
First, merge the _ahf_ and non-ahf interfaces.
Second, remove the always-NULL txs->cmd field.

Then, add optional id_prefix for bitcoind_sendrawx, so if it's
triggered by a command (e.g. "withdraw") it's shown correctly in logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Michael Schmoock
532544ce4f gossipd: rename remote_addr to discovered_ip within gossipd
This is cleaner because, the `remote_addr` and `discovered_ip` are
related but two different things.

Within connectd and lightningd we use the peers `remote_addr` feature
to validate (and guess a port) to be used for IP discovery.

Also when a peer reports us a `remote_addr`, this is given to the plugin API
via the `peer_connected` hook. The network port here is not modified for
godd reason! This can be used i.e. to detect if we are behind a NAT.

But once lightningd figures enough peers report the same `remote_addr`,
it sets the port to the selected network and tells gossipd to use that for
`node_announcement` updates.

Hence, within gossipd, there is no (should not be) `remote_addr`.

Changelog-None
2022-09-15 13:30:06 +09:30
Rusty Russell
4ca6b36439 lightningd: refuse to upgrade db on non-released versions by default.
This is a good sanity check that users understand that if they upgrade
to master mid-cycle they can't go back!

Suggested-by: @wtogami
Changelog-Added: Config: `--database-upgrade=true` required if a non-release version wants to (irrevocably!) upgrade the db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-15 13:25:58 +09:30
Christian Decker
246e1fb0b3 wallet: Set the completed_at timestamp when updating the status 2022-09-14 13:14:10 +02:00
Christian Decker
cb3ee0ac2e wallet: Load and value completed_at timestamp from DB 2022-09-14 13:14:10 +02:00
Christian Decker
daeec66bd7 db: Add completed_at field to payments
We'll want to use these to track durations for `sendpays` and `pays`.
2022-09-14 13:14:10 +02:00
Rusty Russell
e853cdc3ff db: fix sqlite3 code which manipulates columns.
Because it used internal routines, it didn't pass operations through the
db hook!  So make it use the generic routines, with the twist that they
are not translated.

And when we use this in a migration hook, we're actually in a
transaction.

This, in turn, introduces an issue: we need to be outside a transaction
to "PRAGMA foreign_keys = OFF", but completing the transaction when
there is a db hook actually enters the io loop, freeing the tmpctx!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell
d7aa2749c3 db: fix migrations which write to db.
valgrind noticed that this was uninitialized when I tried a complex
migration.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 14:00:41 +02:00
Rusty Russell
1b30ea4b82 doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready.  I change that everywhere, and try to fix up the
comments.

Also the `alias` field is called `short_channel_id`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
2022-09-12 09:34:52 +09:30
Rusty Russell
15751ea1b8 lightningd: do inline parsing for listforwards status parameter
We can do this now the function is cleaned up.

Always better to do the work inside param() since then `check`
gets the benefit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
6c33f7db65 common: remove unused parameter "allow_deprecated" from parse_wireaddr_internal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
bcabb3825f Makefile: Revert ba7d4a8f6b (make-schema: don't include tools/fromschema.py in SHASUMS)
1. If the tool changes, you need to regenerate since the output may
   change.

2. This didn't just filter that out, ignored all but the first
   dependency, which made bisecting the bookkeeper plugin a nightmare:
   it didn't regenerate the .po file, causing random crashes.

If we want this, try $(filter-out tools/fromschema.py) instead.  But I
don't think we want that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-08 00:36:51 +03:00
Rusty Russell
44d9e8d9c5 Remove names of parameters of callbacks which confuse gcc.
We annotate them with UNNEEDED, which is legal but weird, but it
makes gcc (at least 11.2.0) complain about shadowing:

	https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106424

I simply removed the names.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-31 12:18:28 +03:00
Rusty Russell
6fe570820e Remove general shadowed variables.
We shadow local variables in several places: generally, these changes
make the code clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-31 12:18:28 +03:00
niftynei
4e7f89f211 signpsbt: don't crash if HSM doesn't like your psbt, just return err
Changelog-Changed: JSON-RPC: `signpsbt` no longer crashes if it doesn't like what your PSBT is
2022-08-13 12:57:03 -05:00
niftynei
4984014578 signpsbt: add utxo info to inputs
If you build a PSBT externally from CLN and attempt to sign for the
output, we would crash. Now we don't crash.

Changelog-Changed: JSON-RPC: `signpsbt` will now add redeemscript + witness-utxo to the PSBT for an input that we can sign for, before signing it.

Fixes #5499 ?
2022-08-13 12:57:03 -05:00
Rusty Russell
eb006dcadd wallet: fix incorrect column-width access.
Postgresql actually checks, and fails.

It's unclear why this field is an INTEGER (and u32) when it's a BIGINT 
in db here (it's an INTEGER in the channels table, just a BIGINT in the
channel_funding_inflights table).

Changelog-Fixed: db: postgresql crash on startup when dual-funding lease open is pending with "s32 field doesn't match size: expected 4, actual 8"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-10 10:34:12 -05:00
Rusty Russell
22ff007d64 connectd: control connect backoff from lightningd.
We used to tell connectd to remember our connect delay, and hand it
back (increased if necessary).

Instead, simply record when we last tried to connect.  If it was less
than 10 minutes ago, double delay (up to 5 minutes max), otherwise
reset delay to 1 second.

This covers all scenarios: whether we reconnect then immediately
disconnect, or never successfully connect, it doesn't matter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5453
2022-07-28 15:08:44 +09:30
niftynei
aa7ffb78bd wallet: resolve crash when blockheight is null
We weren't ignoring the 'txindex' field
2022-07-28 12:08:18 +09:30
Rusty Russell
e96eb07ef4 lightningd: test that hsm_secret is as expected, at startup.
If you get the wrong hsm_secret, your node_id will change, and
peers won't know who you are, bitcoind will reject your transaction
signatures, and other madness.

Catch this as soon as it happens, by storing our node_id in the db.

Suggested-by: @cdecker, @fiatjaf
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Config: `lightningd` will refuse to start with the wrong node_id (i.e. hsm_secret changes).
2022-07-20 19:28:33 +09:30
Rusty Russell
a3c4908f4a lightningd: don't explicitly tell connectd to disconnect, have it do it on sending error/warning.
Connectd already does this when we *receive* an error or warning, but
now do it on send.  This causes some slight behavior change: we don't
disconnect when we close a channel, for example (our behaviour here
has been inconsistent across versions, depending on the code).

When connectd is told to disconnect, it now does so immediately, and
doesn't wait for subds to drain etc.  That simplifies the manual
disconnect case, which now cleans up as it would from any other
disconnection when connectd says it's disconnected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
Rusty Russell
e15e55190b lightningd: provide peer address for reconnect if connect fails.
It usually works out due to other reconnections, but I noticed this
diagnosing another test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
Rusty Russell
d31420211a connectd: add counters to each peer connection.
This allows us to detect when lightningd hasn't seen our latest
disconnect/reconnect; in particular, we would hit the following pattern:

1. lightningd says to connect a subd.
2. connectd disconnects and reconnects.
3. connectd reads message, connects subd.
4. lightningd reads disconnect and reconnect, sends msg to connect to subd again.
5. connectd asserts because subd is alreacy connected.

This way connectd can tell if lightningd is talking about the previous
connection, and ignoere it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
Rusty Russell
41b379ed89 lightningd: hand fds to connectd, not receive them from connectd.
Before this patch:
1. connectd says it's connected (peer_connected)
2. we tell connectd we want to talk about each channel (peer_make_active)
3. connectd gives us an fd for each channel, and we connect it to a subd (peer_active)
4. OR, connectd says it sent something about a channel we didn't tell it about, with an fd (peer_active)

Now:
1. connectd says it's connected (peer_connected)
2. we start all appropriate subds and tell connectd to what channels/fds (peer_connect_subd).
3. if connectd says it sent something about a channel we didn't tell it about, we either tell
   it to hang up (peer_final_msg), or connect a new opening daemon (peer_connect_subd).

This is the minimal-size patch, which is why we create socket pairs in
so many places to use the existing functions.  Many cleanups are
possible, since the new flow is so simple.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
Rusty Russell
eff53495db lightningd: make "is peer connected" a tristate.
First, connectd tells us the peer has connected, and we call the connected hook,
and if it says it's fine, we are actually connected and we fire off notifications.

Of course, we could be disconnected while in the connected hook, and that would
mean we tell people about a connection which is no longer current.

Make this clear with a tristate: if we're not marked disconnected by
the time the hooks finish, we're good.  It also gives us a cleaner
"connect" command return when we connected but disconnected before
processing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
Rusty Russell
912ac25270 lightningd: remove 'connected' flag from channel structure.
It's directly a product of "does it have a current owner subdaemon"
and "does that subdaemon talk to peers", so create a helper function
which just evaluates that instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-18 20:50:04 -05:00
Rusty Russell
36a29fbfbc lightningd/json.h: remove.
There are hardly any lightningd-specific JSON functions: all that's left
are the feerate ones, and there's already a comment that we should have
a lightningd/feerate.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
401f1debc5 common: clean up json routine locations.
We have them split over common/param.c, common/json.c,
common/json_helpers.c, common/json_tok.c and common/json_stream.c.

Change that to:
* common/json_parse (all the json_to_xxx routines)
* common/json_parse_simple (simplest the json parsing routines, for cli too)
* common/json_stream (all the json_add_xxx routines)
* common/json_param (all the param and param_xxx routines)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
adi2011
6ba8abb0de lightningd: Add RPC for populating DB with stub channels and set an error on reconnecting 2022-07-14 12:24:48 -05:00
adi2011
eca844eb36 channel: Add struct scb_chan in channel and making last tx optional. 2022-07-14 12:24:48 -05:00
adi2011
e42ba8366b common: Add scb_wire for serializing the static_chan_backup 2022-07-14 12:24:48 -05:00
niftynei
ba7d4a8f6b make-schema: don't include tools/fromschema.py in SHASUMS
We were including the entire list of prerequisites when generating a
shastamp, which for schemas includes the `tools/fromschema.py` doc.

This meant all of our shasums were updating anytime this tool file
changed.

Instead, we just include the first prerequisite.

See: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
2022-07-14 12:24:48 -05:00
Rusty Russell
3a1a7eb93f wallet: allow saving forwarding scid even if we don't have amount.
They're not logically connected: we can know where they wanted to
go, but we didn't send it.

Where possible, it's the scid *they asked for*; otherwise, it's the
scid or fallback to the alias, but do this in the *caller*, not by
overriding inside wallet_forwarded_payment_add.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Rusty Russell
32af92145b update-mocks: handle missing deprecated_apis.
This expands update-mocks to be able to handle (simple!) missing
symbols which are not functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-09 09:59:52 +09:30
Christian Decker
252ccfa7ab db: Store the local alias for forwarded incoming payments
Not only can the outgoing edge be a zeroconf channel, it can also be
the incoming channel. So we revert to the usual trick of using the
local alias if the short_channel_id isn't known yet.

We use the LOCAL alias instead of the REMOTE alias even though the
sender likely used the REMOTE alias to refer to the channel. This is
because we control the LOCAL alias, and we keep it stable during the
lifetime of the channel, whereas the REMOTE one could change or not be
there yet.
2022-07-04 22:14:06 +02:00
Christian Decker
2dc86bf29b db: Store the alias if that's all we got in a forward 2022-07-04 22:14:06 +02:00
Christian Decker
de1c0b51f0 zeroconf: Add alias_remote and alias_local to channel and DB
`alias_local` is generated locally and sent to the peer so it knows
what we're calling the channel, while `alias_remote` is received by
the peer so we know what to include in routehints when generating
invoices.
2022-07-04 22:14:06 +02:00
Vincenzo Palazzo
7ff62b4a00 lightnind: removeDEFAULT_PORT global definition
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-06-28 06:09:01 +09:30
Rusty Russell
37403e471c pytest: add (failing) test for db with old Tor v2 address.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-24 10:24:39 +09:30
Rusty Russell
36a2491a89 json: fix up msat amounts in non-_msat fields.
We had json_add_amount_msat_only(), which was designed to be used to
print out msat fields, if we had sats.

However, we misused it, so split it into the three different cases:
1. json_add_amount_sat_msat: We are using it correctly, with a field called
   xxx_msat.
2. json_add_amount_sats_deprecated: We were using it wrong, so deprecate
   the old field and create a new one which does end in _msat.
3. json_add_sats: we were using it to hand sats as a JSON parameter to an
   interface, where "XXXsat".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `rbf_channel` and `openchannel2` hooks `their_funding` (use `their_funding_msat`)
Changelog-Deprecated: Plugins: `openchannel2` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `funding_satoshis` (use `funding_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `channel_reserve_satoshis` (use `channel_reserve_msat`)
Changelog-Deprecated: Plugins: `channel_opened` notification `amount` (use `funding_msat`)
Changelog-Deprecated: JSON-RPC: `listtransactions` `msat` (use `amount_msat`)
Changelog-Deprecated: Plugins: `htlc_accepted` `forward_amount` (use `forward_msat`)
2022-06-21 06:52:35 +09:30
Jon Griffiths
572942c783 psbt: use DER encoded + sighash byte for PSBT_IN_PARTIAL_SIG items
Per BIP-0171, the signature map is of pubkey to "The signature as would
be pushed to the stack from a scriptSig or witness".

Fixes 5298

Changelog-Fixed: PSBT: Fix signature encoding to comply with BIP-0171.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2022-06-09 18:28:35 +02:00
Rusty Russell
f078e54e98 lightningd: remove various deprecated JSON fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: removed `listtransactions` `outputs` `satoshis` field (deprecated v0.10.1)
Changelog-Removed: JSON-RPC: removed `listpeers` `channels` deprecated fields (deprecated v0.10.1)
Changelog-Removed: JSON-RPC: removed `listpeers` `channels` `closer` now omitted, rather than `null` (deprecated v0.10.1)
2022-05-18 10:15:36 +09:30
Rusty Russell
d18817a525 wallet: don't get so upset on orphaned HTLC.
User grubman on IRC reported a crash due to new HTLC checks:

```
2022-04-05T01:15:17.707Z **BROKEN** wallet: Missing preimage for orphaned HTLC; replacing with zeros
...
2022-04-05T01:15:44.950Z **BROKEN** lightningd: check_already_failed:Both failed and succeeded?
2022-04-05T01:15:45.020Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v0.11.0rc1-modded)
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0xaaaab8f7f8bb
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: common/daemon.c:46 (crashdump) 0xaaaab8f7f91b
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffa4a315bf
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffa45c6cd8
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffa45b3a27
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/log.c:821 (fatal_vfmt) 0xaaaab8f3e1b7
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/log.c:829 (fatal) 0xaaaab8f3e24b
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/htlc_end.c:87 (corrupt) 0xaaaab8f325d3
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/htlc_end.c:175 (htlc_out_check) 0xaaaab8f32ee3
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1471 (check_already_failed) 0xaaaab8f53ea7
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1575 (onchain_failed_our_htlc) 0xaaaab8f54aab
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:411 (handle_missing_htlc_output) 0xaaaab8f4037b
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:544 (onchain_msg) 0xaaaab8f409bf
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/subd.c:556 (sd_msg_read) 0xaaaab8f5f8ef
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0xaaaab8fda197
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0xaaaab8fda76f
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0xaaaab8fda82f
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0xaaaab8fdc77f
2022-04-05T01:15:45.020Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0xaaaab8f37857
2022-04-05T01:15:45.021Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1181 (main) 0xaaaab8f3bcff
2022-04-05T01:15:45.021Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffa45b3d4f
2022-04-05T01:15:45.021Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xaaaab8f1ed33
```

Since no HTLCs are likely to still be pending since v0.6.1, we can remove that compat
code.  Indeed, this happens on my node: we've not loaded the incoming HTLC because
it's resolved.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-05 15:39:38 +09:30
Rusty Russell
e47786da04 pay/sendpay: also store description in case bolt11 uses description_hash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-04 20:26:04 +09:30
Rusty Russell
bf4d9e30d2 fundpsbt: deprecate reserve=true/false usage.
Make it always a number; this makes the JSON request specification
simpler.  We allowed a number since v0.10.1.

(reserve=True is the default anyway, so usually it can be omitted:
reserve=False becomes reserve=0).

Changelog-Deprecated: JSON-RPC: `fundpsbt`/`utxopsbt` `reserve` must be a number, not bool (for `true` use 72/don't specify, for `false` use 0).  Numbers have been allowed since v0.10.1.
2022-04-02 09:46:01 +10:30
Rusty Russell
7e789be0ea doc: update BOLTs to latest master.
Just typo fixes and the like.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
3b536400f9 specs: update CSV files.
Regenerate from current BOLTS via `make extract-bolt-csv`

1. The remote_addr field was added manually into peer_wire.csv: this
   needs to be a patch otherwise it vanishes on regen.
2. We never brought into the channel_disabled fields, because it was
   too much hassle (we never actually generate this!).  Do it now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30