Commit Graph

12177 Commits

Author SHA1 Message Date
Ken Sedgwick
0878002fe6 Fix derived_secret, use correct size of secretstuff.derived secret
[ Updated tests to match -- RR]
2022-08-10 12:41:27 -05:00
Rusty Russell
eb006dcadd wallet: fix incorrect column-width access.
Postgresql actually checks, and fails.

It's unclear why this field is an INTEGER (and u32) when it's a BIGINT 
in db here (it's an INTEGER in the channels table, just a BIGINT in the
channel_funding_inflights table).

Changelog-Fixed: db: postgresql crash on startup when dual-funding lease open is pending with "s32 field doesn't match size: expected 4, actual 8"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-10 10:34:12 -05:00
Rusty Russell
b173b29346 pytest: test loading lease_chan_max_msat from channel_funding_inflights
It crashes with:

```
s32 field doesn't match size: expected 4, actual 8
```

Reported-by: @zerofeerouting
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-10 10:34:12 -05:00
niftynei
9d3bf4a1b5 bkpr: let channel reconnect, flake? 2022-08-10 10:30:45 -05:00
niftynei
72a30fc750 bkpr: dont flake, wait til pay done before mining blocks 2022-08-10 10:30:45 -05:00
niftynei
256044081f bkpr: remove duplicate log stmt 2022-08-10 10:30:45 -05:00
niftynei
23cd58402a bkpr: create accounts for zero sat channels
we weren't making records for 'missed' accounts that had a zero balance
at snapshot time (if peer opens channel and is unused)

Fixes: #5502
Reported-By: https://github.com/niftynei/cln-logmaid
2022-08-10 10:30:45 -05:00
Rusty Russell
1ff8e1bacb doc: note that setchannel maxhtlc/minhtlc only apply to *forwards*.
Reported-by: Warren Togami
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:38:57 -05:00
Rusty Russell
fcba09dc33 doc: document that sendonion doesn't have to specify (but can!) the first_hop channel.
This was always the case, though it was ignored.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:33:24 -05:00
Rusty Russell
054339e0cb lightningd: obey first hop channel id.
Reported-by: Warren Togami
Changelog-Changed: `sendpay` and `sendonion` now obey the first hop "channel" short_channel_id, if specified.
Changelog-Deprecated: `sendpay` and `sendonion` could take a bogus first hop "channel" short_channel_id
2022-08-09 16:33:24 -05:00
Rusty Russell
9543204b79 pytest: don't use bogus scids for first hop of route.
This was a legacy from when it was redundant: with multiple channels, it
no longer is!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:33:24 -05:00
Rusty Russell
8a9ce55345 lightningd: don't route private channels via real scid.
Again, we should use the real channel_type, but we approximate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: private channels will only route using short-channel-ids if channel opened with option_scid_alias-supporting peer.
2022-08-09 16:31:36 -05:00
Rusty Russell
cfe6b06fb5 lightnind: use aliases in routehints for private channels.
We *should* remember the channel type, since this is only required
if they set the channel_type to include option_scid_alias.

However, since we support channel upgrade, channel_type really needs
a new table.  I have a patch for that, from my abandoned original
"fastopen" branch for aliases, but it's too big a chance for rc2 IMHO.

Meanwhile, we allow exposeprivatechannels's scids to be either real or
the aliases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: invoice routehints will use fake short-channel-ids for private channels if channel opened with option_scid_alias-supporting peer.
2022-08-09 16:31:36 -05:00
Rusty Russell
b479e9a9fa pytest: test that we implement option_scid_alias privacy.
Spoiler: we don't! :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:31:36 -05:00
Rusty Russell
25b4249f54 doc: fix decode schema for bolt11 routehints.
I decode a routehint in the next patch, and it barfed:

```
>       assert only_one(l1.rpc.decode(inv['bolt11'])['routes'])['short_channel_id'] == alias23

tests/test_opening.py:1515: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:321: in wrapper
    return self.call(name, payload=args)
contrib/pyln-testing/pyln/testing/utils.py:691: in call
    schemas[1].validate(res)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Validator(schema={'$schema': 'http://json-...ft-07/schema#', 'allOf': [{'if': {'properties': {'type': {'enum': [...], ...iption': 'if this is f... diagnostics!', 'type': 'boolean'}}, 'required': ['type', 'valid'], ...}, format_checker=None)
args = ({'amount_msat': 10msat, 'created_at': 1659923931, 'currency': 'bcrt', 'description': 'desc', ...},), kwargs = {}
error = <ValidationError: "1msat is not of type 'u32'">

    def validate(self, *args, **kwargs):
        for error in self.iter_errors(*args, **kwargs):
>           raise error
E           jsonschema.exceptions.ValidationError: 1msat is not of type 'u32'
E           
E           Failed validating 'type' in schema['allOf'][6]['then']['properties']['routes']['items']['items']['properties']['fee_base_msat']:
E               {'description': 'the base fee for payments', 'type': 'u32'}
E           
E           On instance['routes'][0][0]['fee_base_msat']:
E               1msat

```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-09 16:31:36 -05:00
niftynei
016ce2b925 build: ignore docker/image build directories
Changelog-None
2022-08-09 11:57:18 +09:30
niftynei
3fcf60ab7c bkpr: track channel rebalances, display in listincome
Track rebalances, and report income events for them.

Previously `listincome` would report:
	- invoice event, debit, outgoing channel
	- invoice_fee event, debit, outgoing channel
	- invoice event, credit, inbound channel

Now reports:
	- rebalance_fee, debit, outgoing channel
	(same value as invoice_fee above)

Note: only applies on channel events; if a rebalance falls to chain
we'll use the older style of accounting.

Changelog-None
2022-08-09 11:57:18 +09:30
Christian Decker
da0b651803 pay: Use safe list traversal when completing suspended payments
When traversing the list we call `command_finished` which modifies the
list we are traversing. This ensures we don't end up advancing in the
list iteration.

Reported-by: Rusty Russell <@rustyrussell>
2022-08-08 11:52:29 -05:00
Christian Decker
65a449e2c3 pay: Remove use-after-free bug
Technically this is a use-after-free since `command_finished` frees
the `cmd` which is also the parent of `p`, so reset it early. All
paths lead to `command_finished` so setting it early is ok.

Reported-by: Rusty Russell <@rustyrussell>
2022-08-08 11:52:29 -05:00
Christian Decker
093933b14d pay: Do not replay results against payments that were not suspended
Suspended payments now have the same groupid as the actual attempt,
this allows us to identify pay calls that were suspended due to us and
terminate only those.

Changelog-Fixed pay: Fixed a crash when `pay` was called multiple times while an attempt was already in progress.
2022-08-08 11:52:29 -05:00
Christian Decker
aca9c7d49c pay: Annotate suspended payments with the groupid they mirror
This should prevent us from accidentally completing a payment twice,
when replaying the result of an actual attempt against pay call that
was suspended due to it still being pending.
2022-08-08 11:52:29 -05:00
Rusty Russell
80a6d9b58e lightningd: set the channel_type feature.
AFAICT we should have been doing this since we started sending and
receiving it, but didn't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now advertize the `option_channel_type` feature (which we actually supported since v0.10.2)
2022-08-08 11:49:56 -05:00
Rusty Russell
d5d4e7b019 common/features: add channel_type definition, neaten header.
Put all the normal spec-defined ones together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-08 11:49:56 -05:00
Greg Sanders
e0a48c2aa7 am_opener unused 2022-08-04 14:18:34 -05:00
Vincenzo Palazzo
ffb3217ea8 docs: clarify the different way to build cln
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-08-04 11:28:00 -05:00
Rusty Russell
5260ea2911 pytest: make sure we never break channels in multhtlc test.
It's perfectlty safe to extend the timeout to 100 blocks, so let's do
it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:16:01 -05:00
Rusty Russell
93303ffdad pytest: change multihtlc topology for simpler testing.
We were seeing flakes due to channel closures on intermidiary nodes if
the blocks came too fast.

If we use a star topology it's actually what we want and it's much simpler to
figure out what's going on.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:16:01 -05:00
Rusty Russell
b5ee5e7fb1 pytest: simplify test_onchain_multihtlc_our_unilateral/their_unilateral
We don't actually care how it does it, just that it ends the HTLCs,
so simplify this logic which tries to match it exactly.

This also fixes the flake where we would sometimes close the upstream
channels, simply because we now wait at least one second per block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:16:01 -05:00
Rusty Russell
66d8ce7c8c pytest: clarify test_onchain_multihtlc_our_unilateral / test_onchain_multihtlc_their_unilateral
Use the names, not calculations on an array.  It's simply clearer,
especially when debugging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:16:01 -05:00
Christian Decker
2632cddfe4 pay: Fix a memory leak when retrying getroute 2022-08-04 11:14:45 -05:00
Rusty Russell
645b1b505b lightningd: fix funding_locked in channel_opened notification.
Previously, "funding_locked" was always "true"!

(It's actually been wrong since its introduction in v0.7.3!)

Changelog-Fixed: Plugins: `channel_opened` notification `funding_locked` field is now accurate: was always `true`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 13:08:25 +09:30
Seth For Privacy
498f9b75f2 Correct basics of backup plugin usage 2022-08-03 21:22:41 -05:00
Rusty Russell
4ffae2a8c6 CHANGELOG.md: note the Great Msat Migration!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:47:51 +09:30
Rusty Russell
cb496acb49 CHANGELOG.md: neaten entries.
1. Unbalanced backticks and missing space in bookkeeper line.
2. Rephrase emergency backup to not overpromise.
3. "logging" and "lightnignd" are not correct title, use "Config".
4. JSON-RPC not JSONRPC in one place.
5. Don't include empty EXPERIMENTAL section (I checked, it really was empty!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:47:51 +09:30
Rusty Russell
aea2ec5c0d CHANGELOG.md: add in CHANGELOG from 0.11.2 branch.
And manually remove those which were claimed for 0.12.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-04 11:47:51 +09:30
niftynei
071b1bc4f1 pyln: update versions to v0.12.0 2022-08-02 10:00:26 -07:00
niftynei
31ba3007bd changelog: v0.12.0rc1 release notes
Changelog for v0.12.0rc1

Changelog-None
2022-08-02 10:00:26 -07:00
niftynei
75ccce7808 devtools: if there's a message in the API call, print and exit
If you've got bad credentials, you'll get an error message.
2022-08-02 10:00:26 -07:00
niftynei
dfa325dc4d bkpr: make unit tests not fail if !HAVE_SQLITE3
We rely on sqlite3 being present to run unit tests for some bookkeeper
tests; here we effectively disable these tests if not available

Fixes report in #4928
Reported-By: @whitslack
2022-08-02 11:52:57 +09:30
Rusty Russell
2971b2af79 bkpr: insert obscure 60s pop references.
The initial snapshots on an already-running lightningd are expected to
be unbalanced, but this shouldn't cause users to long for the green,
green grass of home.

This controls the Art of Noise.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-08-01 09:01:08 -07:00
niftynei
3445882ee4 bkpr: use long-uint not size_t for time_t
Reported-By: @endothermicdev
2022-07-31 21:53:05 +09:30
niftynei
4e503f7d0a bkpr/listpeeers: add lease_fees back to funds; separate out in listpeers
First, how we record "our_funds" and then apply pushes vs lease_fees
(for liquidity ad buys/sales) was exactly opposite.

For pushes we were reporting the total funded into the channel, with the
push representing how much we'd later moved to the peer.

For lease_fees we were rerporting the total in the channel, with the
push representing how much was already moved to the peer.

We fix this (from a view perspective) by re-adding lease fees to what's
reported in the channel funding totals. Since this is now new behavior
(for leased channel values), we added new fields so we can take the old
field names thru a deprecation cycle.

We also make it possible to differentiate btw a push and a lease_fee
(before they were all the same), by adding to new fields to `listpeers`:
`fee_paid_msat` and `fee_rcvd_msat`.

This allows us to avoid math in the bookkeeper, instead we just pick
the numbers out directly and record them.

Fixes #5472

Changelog-Added: JSON-RPC: `listpeers` now has a few new fields for `funding` (`remote_funds_msat`, `local_funds_msat`, `fee_paid_msat`, `fee_rcvd_msat`).
Changelog-Deprecated: JSON-RPC: `listpeers`.`funded` fields `local_msat` and `remote_msat` are now deprecated.
2022-07-31 21:53:05 +09:30
niftynei
a675f4c24e balance_snapshot: emit balances for channels that are awaiting_lockin
9cad7d6a6a changed the behavior of `channel_active`, which slightly
broke how our balance snapshots work (we need info about channels that
aren't on-chain yet).

This patches adds AWAITING_UNILATERAL back in.
2022-07-31 21:53:05 +09:30
William Casarin
10d66c25c4 commando-rune: show warning when creating runes with no restrictions
Changelog-Added: Show warning when creating runes with no restrictions
Signed-off-by: William Casarin <jb55@jb55.com>
2022-07-29 08:46:14 -07:00
CC
78804d9ea8 fix doc: deschashonly 2022-07-29 16:33:06 +09:30
Rusty Russell
8c9fa457ba pytest: fix flake in test_gossip_timestamp_filter
```
        # 0x0100 = channel_announcement
        # 0x0102 = channel_update
        # (Node announcement may have any timestamp)
        types = Counter([m[0:4] for m in msgs])
        assert types['0100'] == 1
>       assert types['0102'] == 2
E       assert 1 == 2

tests/test_gossip.py:324: AssertionError
```

Examining the logs shows that we ask l4 for timestamps
"first_timestamp=1658892115 timestamp_range=13", and the timestamp on
the missing update is exactly `1658892128`.

So round the end time up by 1 for filtering.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-28 15:08:44 +09:30
Rusty Russell
22ff007d64 connectd: control connect backoff from lightningd.
We used to tell connectd to remember our connect delay, and hand it
back (increased if necessary).

Instead, simply record when we last tried to connect.  If it was less
than 10 minutes ago, double delay (up to 5 minutes max), otherwise
reset delay to 1 second.

This covers all scenarios: whether we reconnect then immediately
disconnect, or never successfully connect, it doesn't matter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5453
2022-07-28 15:08:44 +09:30
Rusty Russell
9cad7d6a6a lightningd: don't consider AWAITING_UNILATERAL to be "active".
It's not active: we don't want to connect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-28 15:08:44 +09:30
Rusty Russell
a259698906 pytest: test that we don't try to reconnect in AWAITING_UNILATERAL.
We would just send an error, and it's annoying.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-28 15:08:44 +09:30
niftynei
6e9af1ef3e bkpr: cleanup csv_safe_str 2022-07-28 12:08:18 +09:30