Commit graph

3667 commits

Author SHA1 Message Date
Rusty Russell
3079afb024 lightningd: add delforward command.
Changelog-Added: JSON-RPC: `delforward` command to delete listforwards entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell
7420a7021f lightningd: add listhtlcs to list all the HTLCs we know about.
Using `listfowards` for this wrong; expose this directly if people
care (and unlike listforwards, which could be deleted, we have to
remember these while the channel is still open!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listhtlcs` new command to list all known HTLCS.
2022-09-22 15:19:46 +02:00
Rusty Russell
311807ff1f lightningd: add in_htlc_id / out_htlc_id to listforwards.
And document that we never know payment_hash.

Changelog-Added: JSON-RPC: `listforwards` now shows `in_htlc_id` and `out_htlc_id`
Changelog-Changed: JSON-RPC: `listforwards` now never shows `payment_hash`; use `listhtlcs`.
2022-09-22 15:19:46 +02:00
Rusty Russell
2022e4a7a9 wallet: simplify payments lookup so sqlite3 uses index.
Filtering by status is rare, so we can do it in the caller; just let sqlite3
filter by payment_hash.

With ~650,000 payments in db:

Before:
```
129/300000 complete 5.60/sec (33078 invs, 169 pays, 0 retries) in 30 seconds. 19 hours-14 hours remaining.
201/300000 complete 7.20/sec (43519 invs, 241 pays, 0 retries) in 40 seconds. 16 hours-11 hours remaining.
257/300000 complete 5.60/sec (54568 invs, 289 pays, 0 retries) in 50 seconds. 16 hours-14 hours remaining.
305/300000 complete 4.80/sec (65772 invs, 337 pays, 0 retries) in 60 seconds. 16 hours-17 hours remaining.
361/300000 complete 5.60/sec (75875 invs, 401 pays, 0 retries) in 70 seconds. 16 hours-14 hours remaining.
```

After:
```
760/300000 complete 40.00/sec (19955 invs, 824 pays, 0 retries) in 20 seconds. 2 hours-2 hours remaining.
1176/300000 complete 41.60/sec (30082 invs, 1224 pays, 0 retries) in 30 seconds. 2 hours-119 minutes remaining.
1584/300000 complete 40.80/sec (40224 invs, 1640 pays, 0 retries) in 40 seconds. 2 hours-2 hours remaining.
1984/300000 complete 40.00/sec (49938 invs, 2048 pays, 0 retries) in 50 seconds. 2 hours-2 hours remaining.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-22 15:19:46 +02:00
Rusty Russell
17858c9490 lightningd: deprecated "delexpiredinvoice", put functionality in autoclean plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `delexpiredinvoice`: use `autoclean-once`.
2022-09-22 15:19:46 +02:00
Rusty Russell
0868fa9f1e lightningd: allow extra tlv types in non-experimental mode.
The old `experimental-accept-extra-tlv-types` is now `accept-htlc-tlv-types`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `accept-htlc-tlv-types` lets us accept unknown even HTLC TLV fields we would normally reject on parsing (was EXPERIMENTAL-only `experimental-accept-extra-tlv-types`).
2022-09-22 17:19:11 +09:30
Christian Decker
67467213cb opening: Add dev-allowdustreserve option to opt into dust reserves
Technically this is a non-conformance with the spec, hence the `dev`
flag to opt-in, however I'm being told that it is also implemented in
other implementations. I'll follow this up with a proposal to the spec
to remove the checks we now bypass.
2022-09-21 11:25:47 +02:00
Christian Decker
5a54f450bd openingd: Pass reserve down to openingd when funding 2022-09-21 11:25:47 +02:00
Christian Decker
8d6423389a openingd: Wire reserve value through to openingd 2022-09-21 11:25:47 +02:00
Christian Decker
9a97f8c154 plugin: Add reserve to openchannel hook result
Changelog-Added: plugin: The `openchannel` hook may return a custom absolute `reserve` value that the peer must not dip below.
2022-09-21 11:25:47 +02:00
Christian Decker
5c1de8029a openingd: Add reserve to fundchannel and multifundchannel
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel` and `fundchannel_start` now accept a `reserve` parameter to indicate the absolute reserve to impose on the peer.
2022-09-21 11:25:47 +02:00
Greg Sanders
248d60d7bd Don't report redundant feerates to subdaemons 2022-09-19 19:10:35 +02:00
Rusty Russell
701dd3dcef memleak: remove exclusions from memleak_start()
Add memleak_ignore_children() so callers can do exclusions themselves.

Having two exclusions was always such a hack!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell
3380f559f9 memleak: simplify API.
Mainly renaming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell
2da5244e83 jsonrpc: make error codes an enum.
This allows GDB to print values, but also allows us to use them in
'case' statements.  This wasn't allowed before because they're not
constant terms.

This also made it clear there's a clash between two error codes,
so move one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: Error code from bcli plugin changed from 400 to 500.
2022-09-19 10:18:55 +09:30
Rusty Russell
bbe1711e16 lightningd: use jcon logging for commands where possible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
caecd1ee0a lightningd: don't log JSON ids as debug, use log io.
They are cute, sure, but they do spam the logs.

@Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
e8ef42b741 plugin: wire JSON id for commands which caused hooks to fire.
Most obvious one is the "connect" hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
eceb9f4328 lightningd: wire plugin command JSON id through to plugin commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
ea7903f69a lightningd: trace JSON id prefixes through sendrawtx.
First, merge the _ahf_ and non-ahf interfaces.
Second, remove the always-NULL txs->cmd field.

Then, add optional id_prefix for bitcoind_sendrawx, so if it's
triggered by a command (e.g. "withdraw") it's shown correctly in logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
a9557d5194 lightningd: derive JSONRPC ids from incoming id (append /cln:<method>#NNN).
Usually the calls are spontanous, so it's just "cln:<method>#NNN", but
json_invoice() calls listincoming, and json_checkmessage calls
listnodes, so those become "cli:invoice-<pid>/cln:listincoming#NNN".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
8fcf880e0f lightningd: explicitly remember if JSON id was a string.
This lets us use 'cmd->id' as an unquoted string (for building
new ids!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
ed3f700991 lightningd: use string as json req ids when we create them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
99f2019a24 lightningd: add jsonrpc_request_start_raw instead of NULL method.
Since we want to use methodname to create id, don't overload it
for a raw request.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
2d7cf153ad lightningd: log JSON request ids.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Michael Schmoock
532544ce4f gossipd: rename remote_addr to discovered_ip within gossipd
This is cleaner because, the `remote_addr` and `discovered_ip` are
related but two different things.

Within connectd and lightningd we use the peers `remote_addr` feature
to validate (and guess a port) to be used for IP discovery.

Also when a peer reports us a `remote_addr`, this is given to the plugin API
via the `peer_connected` hook. The network port here is not modified for
godd reason! This can be used i.e. to detect if we are behind a NAT.

But once lightningd figures enough peers report the same `remote_addr`,
it sets the port to the selected network and tells gossipd to use that for
`node_announcement` updates.

Hence, within gossipd, there is no (should not be) `remote_addr`.

Changelog-None
2022-09-15 13:30:06 +09:30
Michael Schmoock
c8ab8192ca peer_control: getinfo show correct port on discovered IPs
Changelog-Fixed: peer_control: getinfo shows the correct port on discovered IPs
2022-09-15 13:30:06 +09:30
Rusty Russell
4ca6b36439 lightningd: refuse to upgrade db on non-released versions by default.
This is a good sanity check that users understand that if they upgrade
to master mid-cycle they can't go back!

Suggested-by: @wtogami
Changelog-Added: Config: `--database-upgrade=true` required if a non-release version wants to (irrevocably!) upgrade the db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-15 13:25:58 +09:30
fiatjaf
1ef8fb7ef8 rename block_processed to block_added 2022-09-14 13:50:38 -05:00
fiatjaf
9b33a921f0 Add plugin notification topic "block_processed".
Changelog-Added: Plugins: Added notification topic "block_processed".
2022-09-14 13:50:38 -05:00
Rusty Russell
023a688e3f lightningd: fix spurious leak report.
We can (and probably should!) allocate this off tmpctx rather than
off the response.

Also, json_add_invoice doesn't actually match the normal pattern,
so fix that.


```
E           ValueError: 
E           Node errors:
E           Global errors:
E            - Node /tmp/ltests-_vbj8az8/test_pay_fail_unconfirmed_channel_1/lightning-2/ has memory leaks: [
E               {
E                   "backtrace": [
E                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
E                       "wallet/invoices.c:81 (wallet_stmt2invoice_details)",
E                       "wallet/invoices.c:697 (invoices_get_details)",
E                       "wallet/wallet.c:2946 (wallet_invoice_details)",
E                       "lightningd/invoice.c:1296 (json_add_invoices)",
E                       "lightningd/invoice.c:1370 (json_listinvoices)",
E                       "lightningd/jsonrpc.c:625 (command_exec)",
E                       "lightningd/jsonrpc.c:753 (rpc_command_hook_final)",
E                       "lightningd/plugin_hook.c:280 (plugin_hook_call_)",
E                       "lightningd/jsonrpc.c:841 (plugin_hook_call_rpc_command)",
E                       "lightningd/jsonrpc.c:938 (parse_request)",
E                       "lightningd/jsonrpc.c:1035 (read_json)",
E                       "ccan/ccan/io/io.c:59 (next_plan)",
E                       "ccan/ccan/io/io.c:407 (do_plan)",
E                       "ccan/ccan/io/io.c:417 (io_ready)",
E                       "ccan/ccan/io/poll.c:453 (io_loop)",
E                       "lightningd/io_loop_with_timers.c:22 (io_loop_with_timers)",
E                       "lightningd/lightningd.c:1194 (main)",
E                       "../csu/libc-start.c:308 (__libc_start_main)"
E                   ],
E                   "label": "wallet/invoices.c:81:struct invoice_details",
E                   "parents": [
E                       "common/json_stream.c:41:struct json_stream",
E                       "ccan/ccan/io/io.c:91:struct io_conn **NOTLEAK**",
E                       "lightningd/lightningd.c:107:struct lightningd"
E                   ],
E                   "value": "0x55a57a77de08"
E               }
E           ]
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-14 13:43:27 -05:00
Christian Decker
cb3ee0ac2e wallet: Load and value completed_at timestamp from DB 2022-09-14 13:14:10 +02:00
niftynei
1980ba420b notif: dont send balance snapshot for not yet opened channel
We were double counting channel lease fees because we were double firing
the channel open event sequence (so to speak). If we don't report
balances for unopened channels, we don't have this problem?

Changelog-Changed: Plugins: `balance_snapshot` notification does not send balances for channels that aren't locked-in/opened yet
2022-09-14 15:59:37 +09:30
Michael Schmoock
e0d6f3ceb1 connectd: DNS Bolt7 #911 no longer EXPERIMENTAL
Changelog-Changed: Bolt7 #911 DNS annoucenent support is no longer EXPERIMENTAL
2022-09-13 06:42:20 +09:30
Rusty Russell
375215a141 lightningd: more graceful shutdown.
Be more graceful in shutting down: this should fix the issue where
bookkeeper gets upset that its commands are rejected during shutdown,
and generally make things more graceful.

1. Stop any new RPC connections.
2. Stop any per-peer daemons (channeld, etc).
3. Shut down plugins.
4. Stop all existing RPC connections.
5. Stop global daemons.
6. Free up peer, chanen HTLC datastructures.
7. Close database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: RPC operations are now still available during shutdown.
2022-09-12 14:00:41 +02:00
Rusty Russell
4ca1203eb8 doc: include recent BOLT recommendation on grace period.
This includes the recommendation that we use 10 minute grace period,
so add quotes to where we use that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
5b7f14a7cb channeld/dualopend/lightningd: use channel_ready everywhere.
This alters the billboard, but that's a human-readable thing so not
noted in CHANGELOG.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listpeers` `status` now refers to "channel ready" rather than "funding locked" (BOLT language change for zeroconf channels)
Changelog-Added: JSON-RPC: `channel_opened` notification `channel_ready` flag.
Changelog-Deprecated: JSON-RPC: `channel_opened` notification `funding_locked` flag (use `channel_ready`: BOLTs namechange).
2022-09-12 09:34:52 +09:30
Rusty Russell
1b30ea4b82 doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready.  I change that everywhere, and try to fix up the
comments.

Also the `alias` field is called `short_channel_id`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
2022-09-12 09:34:52 +09:30
Rusty Russell
6cf3d47505 offers: remove backwards-compatiblity invoice_request signatures.
We changed the field name in v0.11.0, so this breaks compat with
v0.10.2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
29264e83fb lightningd: remove use_proxy_always parameter to plugin init.
Changelog-Removed: Plugins: plugin init `use_proxy_always` (deprecated v0.10.2)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
733ce81bd4 plugins: require usage for plugin APIs.
Changelog-Removed: JSON-RPC: plugins must supply `usage` parameter (deprecated v0.7)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
15751ea1b8 lightningd: do inline parsing for listforwards status parameter
We can do this now the function is cleaned up.

Always better to do the work inside param() since then `check`
gets the benefit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
a45ec78c36 lightningd: don't allow old listforwards arg order.
Changelog-Removed: Old order of the `status` parameter in the `listforwards` rpc command (deprecated in v0.10.2)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
43b037ab0b lightningd: always require "jsonrpc": "2.0" in request.
Changelog-Removed: JSONRPC: RPC framework now requires the `"jsonrpc"` property inside the request (deprecated in v0.10.2)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
bfe342c64b lightningd: remove double-wrapped rpc_command hook.
Somehow we missed this deprecation, found by grep.

Changelog-Removed: JSON API: Removed double wrapping of `rpc_command` payload in `rpc_command` JSON field (deprecated v0.8.2)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
6c33f7db65 common: remove unused parameter "allow_deprecated" from parse_wireaddr_internal.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Michael Schmoock
c6858748bb cleanup: fix mixed indentation of json_getinfo
This one got badly messed up over time. I know we usually don't fix
these to have easier git-bisect. I can remove this commit if required.
2022-09-11 18:09:13 +02:00
Michael Schmoock
ec95c7c18c peer_control: fix getinfo showing unannounced addr
Currently discovered IPs are only announced when we don't have any
usable addresses detected or configured already. However, the cli
command `getinfo` still showed theses unannounced addr as if they
were announced.

Changelog-Fixed: peer_control: getinfo showing unannounced addresses.
2022-09-11 18:09:13 +02:00
Rusty Russell
44d9e8d9c5 Remove names of parameters of callbacks which confuse gcc.
We annotate them with UNNEEDED, which is legal but weird, but it
makes gcc (at least 11.2.0) complain about shadowing:

	https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106424

I simply removed the names.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-31 12:18:28 +03:00
Rusty Russell
6fe570820e Remove general shadowed variables.
We shadow local variables in several places: generally, these changes
make the code clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-31 12:18:28 +03:00