Commit graph

2527 commits

Author SHA1 Message Date
Rusty Russell
0db01c882f pytest: fix flake in test_sendcustommsg
We assume that because we've told l3 to shut down, l2 already sees it
as disconnected.  But CI is ...slow... today!

```
        # `l3` is disconnected and we can't send messages to it
>       assert(not l2.rpc.listpeers(l3.info['id'])['peers'][0]['connected'])
E       assert not True

tests/test_misc.py:2218: AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-21 12:15:48 +02:00
Christian Decker
774d16a72e openingd: Fail if dust and max_htlcs result in 0output commitment tx
Prior to this we might end up with a commitment transaction without
any outputs, if combined with `--dev-allowdustreserve`. Otherwise the
reserve being larger than dust means the funder could not drop its
direct output to be below dust.

Reported-by: Rusty Russell <@rustyrussell>
2022-09-21 11:25:47 +02:00
Christian Decker
493a0dfcd4 pytest: Exercise all dust zeroreserve case
This just shows we need to add that constraint.

Reported-by: Rusty Russell <@rustyrussell>
2022-09-21 11:25:47 +02:00
Christian Decker
bdda62e1a4 pytest: Add test for mixed zeroreserve funding
Tests that we can have zeroreserve nodes accept and open new channels
with non-zeroreserve nodes, without throwing errors
2022-09-21 11:25:47 +02:00
Christian Decker
54b4baabb0 opening: Add dev-allowdustreserve option to opt into dust reserves
Technically this is a non-conformance with the spec, hence the `dev`
flag to opt-in, however I'm being told that it is also implemented in
other implementations. I'll follow this up with a proposal to the spec
to remove the checks we now bypass.
2022-09-21 11:25:47 +02:00
Christian Decker
c5b2aee5c6 pytest: Add a zeroreserve test 2022-09-21 11:25:47 +02:00
Rusty Russell
3380f559f9 memleak: simplify API.
Mainly renaming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-19 11:34:42 +09:30
Rusty Russell
caecd1ee0a lightningd: don't log JSON ids as debug, use log io.
They are cute, sure, but they do spam the logs.

@Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
f1f2c1322d contrib/pyln-client: construct JSON ID correctly.
They can set their name explicitly, but if they don't we extract it from argv[0].

We also set it around callbacks, so it will be expanded by default.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
d360075d22 libplugin: use string ids correctly.
Build them from the command which caused them, and take plugin name
as basename with extension stripped.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
e8ef42b741 plugin: wire JSON id for commands which caused hooks to fire.
Most obvious one is the "connect" hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
eceb9f4328 lightningd: wire plugin command JSON id through to plugin commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
ea7903f69a lightningd: trace JSON id prefixes through sendrawtx.
First, merge the _ahf_ and non-ahf interfaces.
Second, remove the always-NULL txs->cmd field.

Then, add optional id_prefix for bitcoind_sendrawx, so if it's
triggered by a command (e.g. "withdraw") it's shown correctly in logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
a9557d5194 lightningd: derive JSONRPC ids from incoming id (append /cln:<method>#NNN).
Usually the calls are spontanous, so it's just "cln:<method>#NNN", but
json_invoice() calls listincoming, and json_checkmessage calls
listnodes, so those become "cli:invoice-<pid>/cln:listincoming#NNN".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
8fcf880e0f lightningd: explicitly remember if JSON id was a string.
This lets us use 'cmd->id' as an unquoted string (for building
new ids!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
ed3f700991 lightningd: use string as json req ids when we create them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Rusty Russell
8711241535 lightning-cli: use cli:<method>-<pid> for all requests.
This is the format we should standardize on.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-16 12:31:45 +09:30
Michael Schmoock
c8ab8192ca peer_control: getinfo show correct port on discovered IPs
Changelog-Fixed: peer_control: getinfo shows the correct port on discovered IPs
2022-09-15 13:30:06 +09:30
Rusty Russell
4ca6b36439 lightningd: refuse to upgrade db on non-released versions by default.
This is a good sanity check that users understand that if they upgrade
to master mid-cycle they can't go back!

Suggested-by: @wtogami
Changelog-Added: Config: `--database-upgrade=true` required if a non-release version wants to (irrevocably!) upgrade the db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-15 13:25:58 +09:30
fiatjaf
1ef8fb7ef8 rename block_processed to block_added 2022-09-14 13:50:38 -05:00
fiatjaf
9b33a921f0 Add plugin notification topic "block_processed".
Changelog-Added: Plugins: Added notification topic "block_processed".
2022-09-14 13:50:38 -05:00
Rusty Russell
a6d4756d08 commando: make rune alternatives a JSON array.
This avoids having to escape | or &, though we still allow that for
the deprecation period.

To detect deprecated usage, we insist that alternatives are *always*
an array (which could be loosened later), but that also means that
restrictions must *always* be an array for now.

Before:

```
# invoice, description either A or B
lightning-cli commando-rune '["method=invoice","pnamedescription=A|pnamedescription=B"]'
# invoice, description literally 'A|B'
lightning-cli commando-rune '["method=invoice","pnamedescription=A\\|B"]'
```

After:

```
# invoice, description either A or B
lightning-cli commando-rune '[["method=invoice"],["pnamedescription=A", "pnamedescription=B"]]'
# invoice, description literally 'A|B'
lightning-cli commando-rune '[["method=invoice"],["pnamedescription=A|B"]]'
```

Changelog-Deprecated: JSON-RPC: `commando-rune` restrictions is always an array, each element an array of alternatives.  Replaces a string with `|`-separators, so no escaping necessary except for `\\`.
2022-09-14 17:46:43 +02:00
Rusty Russell
d57d87ea3a commando: unmangle JSON.
JSON needs to escape \, since it can't be in front of anything unexpected,
so no \|.  So we need to return \\ to \, and in theory handle \n etc.

Changelog-Fixed: JSON-RPC: `commando-rune` now handles \\ escapes properly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-14 17:46:43 +02:00
Rusty Russell
897245e3b7 pytest: test for escapes in commando values.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-14 17:46:43 +02:00
Christian Decker
cb3ee0ac2e wallet: Load and value completed_at timestamp from DB 2022-09-14 13:14:10 +02:00
niftynei
3ad8347969 bkpr-test: maybe fix race in test_bookkeeping_closing_trimmed_htlcs
test_bookkeeping_closing_trimmed_htlcs fails to find 'all outputs
resolved' occassionally, seems like it's because the
OUR_DELAYED_TO_WALLET doesn't make it into the mempool before we start
mining blocks?

So here make sure there's something in the mempool before before we
start making new blocks.
2022-09-14 15:59:37 +09:30
niftynei
efad09f966 bkpr: confirm that replaying the open+lock-in txs at start is ok
Make sure that we're not issuing duplicate lease_fee events!
2022-09-14 15:59:37 +09:30
niftynei
c143914ebf bkpr: migration to delete any duplicate lease_fee entries
Clean up for #5557.

If you've got duplicate 'lease_fee' entries, we delete them!
2022-09-14 15:59:37 +09:30
niftynei
1980ba420b notif: dont send balance snapshot for not yet opened channel
We were double counting channel lease fees because we were double firing
the channel open event sequence (so to speak). If we don't report
balances for unopened channels, we don't have this problem?

Changelog-Changed: Plugins: `balance_snapshot` notification does not send balances for channels that aren't locked-in/opened yet
2022-09-14 15:59:37 +09:30
niftynei
8452d903b4 bkpr: failing test for bookkeeper crash
Reproduce crash for #5557!

If we record the channel open because bookkeeper was added after the
channel open request started but the channel confirms later, we end up
with re-recording any associated push or leased fees (paid or rcvd).

In the case where you've paid for these fees, your channel balance goes
negative and the node crashes the next time you call `listbalances`.

Reported-by: @chrisguida
2022-09-14 15:59:37 +09:30
Michael Schmoock
e0d6f3ceb1 connectd: DNS Bolt7 #911 no longer EXPERIMENTAL
Changelog-Changed: Bolt7 #911 DNS annoucenent support is no longer EXPERIMENTAL
2022-09-13 06:42:20 +09:30
Rusty Russell
375215a141 lightningd: more graceful shutdown.
Be more graceful in shutting down: this should fix the issue where
bookkeeper gets upset that its commands are rejected during shutdown,
and generally make things more graceful.

1. Stop any new RPC connections.
2. Stop any per-peer daemons (channeld, etc).
3. Shut down plugins.
4. Stop all existing RPC connections.
5. Stop global daemons.
6. Free up peer, chanen HTLC datastructures.
7. Close database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: RPC operations are now still available during shutdown.
2022-09-12 14:00:41 +02:00
Rusty Russell
5b7f14a7cb channeld/dualopend/lightningd: use channel_ready everywhere.
This alters the billboard, but that's a human-readable thing so not
noted in CHANGELOG.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listpeers` `status` now refers to "channel ready" rather than "funding locked" (BOLT language change for zeroconf channels)
Changelog-Added: JSON-RPC: `channel_opened` notification `channel_ready` flag.
Changelog-Deprecated: JSON-RPC: `channel_opened` notification `funding_locked` flag (use `channel_ready`: BOLTs namechange).
2022-09-12 09:34:52 +09:30
Rusty Russell
1b30ea4b82 doc: update BOLTs to bc86304b4b0af5fd5ce9d24f74e2ebbceb7e2730
This contains the zeroconf stuff, with funding_locked renamed to
channel_ready.  I change that everywhere, and try to fix up the
comments.

Also the `alias` field is called `short_channel_id`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `funding_locked` is now called `channel_ready` as per latest BOLTs.
2022-09-12 09:34:52 +09:30
Rusty Russell
6cf3d47505 offers: remove backwards-compatiblity invoice_request signatures.
We changed the field name in v0.11.0, so this breaks compat with
v0.10.2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-12 09:34:52 +09:30
Rusty Russell
c4203e7de6 pyln-client: allow 'msat' fields to be 'null'
This happens with deprecated-apis and listconfigs, breaking some
python plugins!

Fixes: #5546
Fixes: #5563
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-07 11:01:36 +09:30
Matt Morehouse
0d5808b6f6 pytest: fix test_channel_state_change_history
The test fails because the closed channel is deleted by the time we
check the state change history.

It is unnecessary to mine any blocks after the close, since the state
changes up to CLOSINGD_COMPLETE occur without onchain changes.
2022-08-31 12:23:43 +03:00
Alex Myers
0abe2e3af1 pytest: Add debugging to test_gossip_store_compact_on_load
This flake has been difficult to reproduce, so let's dump the gossip
store to aid in debugging. See issue #5410.

Changelog-None
2022-08-26 07:03:10 +09:30
adi2011
8f78a76d1a tests/test_misc.py: check logs for already existing channel. 2022-08-19 10:27:09 -04:00
Christian Decker
65549a2931 ld: Fix a log message assuming that the channel->scid was set
This is no longer true after we introduce `zeroconf`, so use the alias
local alias instead if not set.

Signed-off-by: Christian Decker <@cdecker>
2022-08-16 14:52:52 -05:00
Christian Decker
ab1ca7f159 pytest: Reproduce a crash when we have multiple channels and 0conf
This is a case where we assume that the `channel->scid` is set, which
is no longer true with `zeroconf`

Changelog-None: Zeroconf was not yet released at the time the discovery was made

Reported-by: Yaacov Akiba Slama <@yaslama>
Reported-by: Roei Erez <@roeierez>
2022-08-16 14:52:52 -05:00
niftynei
4984014578 signpsbt: add utxo info to inputs
If you build a PSBT externally from CLN and attempt to sign for the
output, we would crash. Now we don't crash.

Changelog-Changed: JSON-RPC: `signpsbt` will now add redeemscript + witness-utxo to the PSBT for an input that we can sign for, before signing it.

Fixes #5499 ?
2022-08-13 12:57:03 -05:00
Ken Sedgwick
0878002fe6 Fix derived_secret, use correct size of secretstuff.derived secret
[ Updated tests to match -- RR]
2022-08-10 12:41:27 -05:00
Rusty Russell
b173b29346 pytest: test loading lease_chan_max_msat from channel_funding_inflights
It crashes with:

```
s32 field doesn't match size: expected 4, actual 8
```

Reported-by: @zerofeerouting
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-10 10:34:12 -05:00
niftynei
9d3bf4a1b5 bkpr: let channel reconnect, flake? 2022-08-10 10:30:45 -05:00
niftynei
72a30fc750 bkpr: dont flake, wait til pay done before mining blocks 2022-08-10 10:30:45 -05:00
niftynei
23cd58402a bkpr: create accounts for zero sat channels
we weren't making records for 'missed' accounts that had a zero balance
at snapshot time (if peer opens channel and is unused)

Fixes: #5502
Reported-By: https://github.com/niftynei/cln-logmaid
2022-08-10 10:30:45 -05:00
Rusty Russell
9543204b79 pytest: don't use bogus scids for first hop of route.
This was a legacy from when it was redundant: with multiple channels, it
no longer is!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:33:24 -05:00
Rusty Russell
8a9ce55345 lightningd: don't route private channels via real scid.
Again, we should use the real channel_type, but we approximate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: private channels will only route using short-channel-ids if channel opened with option_scid_alias-supporting peer.
2022-08-09 16:31:36 -05:00
Rusty Russell
cfe6b06fb5 lightnind: use aliases in routehints for private channels.
We *should* remember the channel type, since this is only required
if they set the channel_type to include option_scid_alias.

However, since we support channel upgrade, channel_type really needs
a new table.  I have a patch for that, from my abandoned original
"fastopen" branch for aliases, but it's too big a chance for rc2 IMHO.

Meanwhile, we allow exposeprivatechannels's scids to be either real or
the aliases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: invoice routehints will use fake short-channel-ids for private channels if channel opened with option_scid_alias-supporting peer.
2022-08-09 16:31:36 -05:00