I noticed that our subtables were not being cleaned, despite being "ON
DELETE CASCADE". This is because foreign keys were not enabled, but
then I got foreign key errors: rowid cannot be a foreign key anyway!
So create a real "rowid" column. We want "ON DELETE CASCADE" for
nodes and channels (and other tables in future) where we update
partially.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This *would* be a 1-line change (add it to Makefile) except that we
previously assumed a "list" prefix on commands.
These use the default refreshing, but they could be done better using
the time-range parameters.
Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
For now, we ignore every deprecated field, but put in the logic so
that future deprecations will work as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This requires us to rename "index" fields, rename fields if we have a
sub-object, and create sub-tables if we have an array, and handle the
fact that some listX commands don't contain array X (listsendpays
contains "payments").
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. When we receive a commando command from a remote using the `filter`
field, use it.
2. Add a `filter` parameter to `commando` to send it: this is usually
more efficient than using filtering locally.
Of course, older remote nodes will ignore the filter, but that's
harmless.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `commando` now supports `filter` as a parameter (for send and receive).
This was reported a while ago: now do it properly.
Fixes: #5637
Changelog-Fixed: Plugins: `commando` now responds to remote JSON calls with the correct JSON `id` field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We change the libplugin API so commando can provide its own ID base.
This id chaining enables much nicer diagnostics!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't do this yet, so we add deprecated to those test (until next
patch!).
Changelog-Deprecated: plugins: `commando` JSON commands without an `id` (see doc/lightningd-rpc.7.md for how to construct a good id field).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids any confusion between primitive and string ids, and in
particular stops an issue with commando once it starts chaining ids,
that weird ids can be double-escaped and commando will not recognize
the response, leaving the client hanging. It's the client's fault for
using a weird id, but it's still rude (and triggered by our tests!).
It also makes substituting the id in passthrough simpler, FTW.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a minimal fix: we wait until all plugins reply from init before
continuing. Really large or busy nodes can have other things monopolize
lightningd, then the timer goes off and we blame the plugin (which has
responded, we just haven't read it yet!).
The real answer is to have some timeouts only advance when we're idle,
or have them low-priority so we only activate them when we're idle (this
doesn't apply to all timers: some are probably important!). But
this is a minimal fix for -rc3.
Fixes: https://github.com/ElementsProject/lightning/issues/5736
Changelog-Fixed: plugins: on large/slow nodes we could blame plugins for failing to answer init in time, when we were just slow.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We no longer use offers for "I want to send you money", but we'll use
invoice_requests directly. Create a new table for them, and
associated functions.
The "localofferid" for "pay" and "sendpay" is now "localinvreqid".
This is an experimental-only option, so document the change under
experimental only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `pay` and `sendpay` `localofferid` is now `localinvreqid`.
1. Remove the very concept of ONION_REPLY_SIZE, instead make it a
local variable in create_onionreply().
2. Use the proper fromwire_ primitives in unwrap_onionreply() so we
don't have to do explicit length checks.
3. Make fromwire_tal_arrn() return NULL if it fails to pull, instead of
a zero-length allocation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: we now correctly decrypt non-256-length onion errors (we always forwarded them fine, now we actually can parse them).
Otherwise what the hook sees is actually a lie, and if it sets it
we might override it.
The side effect is that we add an explicit "forward_to" field, and
allow hooks to override it. This lets a *hook* control channel
choice explicitly.
Changelod-Added: Plugins: `htlc_accepted_hook` return can specify what channel to forward htlc to.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
CI is really slow: it sees all three expire at once. But making the
timeouts too long is painful in non-VALGRIND, so I ended up making it
conditional.
```
# First it expires.
wait_for(lambda: only_one(l3.rpc.listinvoices('inv1')['invoices'])['status'] == 'expired')
# Now will get autocleaned
wait_for(lambda: l3.rpc.listinvoices('inv1')['invoices'] == [])
> assert l3.rpc.autoclean_status()['autoclean']['expiredinvoices']['cleaned'] == 1
E assert 3 == 1
tests/test_plugin.py:2975: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's more natural: we will eventually support dynamic config variables,
so this will be quite nice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `autoclean` can now delete old forwards, payments, and invoices automatically.
And take the opportunity to rename l0 and l1 in the tests to the
more natural l1 l2 (since we add l3).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And document that we never know payment_hash.
Changelog-Added: JSON-RPC: `listforwards` now shows `in_htlc_id` and `out_htlc_id`
Changelog-Changed: JSON-RPC: `listforwards` now never shows `payment_hash`; use `listhtlcs`.
This one directly contains the scids of the channels involved, not
references, so can outlive the channels. As a side-effect, however,
it now never lists `payment_hash`. Having it listed (via join) is not
possible as it is a *string* in the channels table, and difficult
anyway because of channel aliases.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They can set their name explicitly, but if they don't we extract it from argv[0].
We also set it around callbacks, so it will be expanded by default.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids having to escape | or &, though we still allow that for
the deprecation period.
To detect deprecated usage, we insist that alternatives are *always*
an array (which could be loosened later), but that also means that
restrictions must *always* be an array for now.
Before:
```
# invoice, description either A or B
lightning-cli commando-rune '["method=invoice","pnamedescription=A|pnamedescription=B"]'
# invoice, description literally 'A|B'
lightning-cli commando-rune '["method=invoice","pnamedescription=A\\|B"]'
```
After:
```
# invoice, description either A or B
lightning-cli commando-rune '[["method=invoice"],["pnamedescription=A", "pnamedescription=B"]]'
# invoice, description literally 'A|B'
lightning-cli commando-rune '[["method=invoice"],["pnamedescription=A|B"]]'
```
Changelog-Deprecated: JSON-RPC: `commando-rune` restrictions is always an array, each element an array of alternatives. Replaces a string with `|`-separators, so no escaping necessary except for `\\`.
JSON needs to escape \, since it can't be in front of anything unexpected,
so no \|. So we need to return \\ to \, and in theory handle \n etc.
Changelog-Fixed: JSON-RPC: `commando-rune` now handles \\ escapes properly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Be more graceful in shutting down: this should fix the issue where
bookkeeper gets upset that its commands are rejected during shutdown,
and generally make things more graceful.
1. Stop any new RPC connections.
2. Stop any per-peer daemons (channeld, etc).
3. Shut down plugins.
4. Stop all existing RPC connections.
5. Stop global daemons.
6. Free up peer, chanen HTLC datastructures.
7. Close database.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: RPC operations are now still available during shutdown.
