Commit Graph

2066 Commits

Author SHA1 Message Date
Rusty Russell
aeeaa14430 pytest: add reconnect and restart to test_multichan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
8458994a7f pytest: test closing one multichannel works as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
75596b3e0f lightningd: use a better channel if available to next hop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
debc1b90d3 lightningd: remove checks which prevent us from opening multiple channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now support opening multiple channels with the same peer.
2022-03-23 13:20:12 +10:30
Rusty Russell
21e1d68e3b lightningd: remove (most) functions to search channels by status.
This is generally verboten now, since there can be multiple.  There are a
few exceptions:

1. We sometimes want to know if there are *any* active channels.
2. Some dev commands still take peer id when they mean channel_id.
3. We still allow peer id when it's fully determined.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `close` by peer id will fail if there is more than one live channel (use `channel_id` or `short_channel_id` as id arg).
2022-03-23 13:20:12 +10:30
Rusty Russell
cb5dc48cab lightningd: make setchannelfee handle multiple channels per peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
90be2cc104 lightningd: remove some "single active channel" assumptions.
Generally this means converting a lazy "peer_active_channel(peer)" call
into an explicit iteration.

1. notify_feerate_change: call all channels (ignores non-active ones anyway).
2. peer_get_owning_subd remove unused function.
3. peer_connected hook: don't save channel, do lookup and iterate channels.
4. In json_setchannelfee "all" remove useless call to peer_active_channel
   since we check state anyway, and iterate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
7de7b7be61 lightningd: use channel_id when a peer is activated.
Rather than intuiting whether this is a new channel / active channel,
use the channel_id.  This simplifies things and makes them explicit,
and prepares for multiple live channels per peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
2424b7dea8 connectd: hold peer until we're interested.
Either because lightningd tells us it wants to talk, or because the peer
says something about a channel.

We also introduce a behavior change: we disconnect after a failed open.
We might want to modify this later, but we it's a side-effect of openingd
not holding onto idle connections.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
deecedb033 connectd: tell lightningd when disconnect is complete.
This avoids races in our tests where we assume it's sync (and is kind
of nicer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
10e36e073c openingd: disconnect from peer when an error occurs.
openingd currently holds the connection to idle peers, but we're about
to change that: it will only look after peers which are actively
opening a connection.  We can start this process by disconnecting
whenever we have a negotiation failure.

We could stay connected if we wanted to, but that would be up to
connectd to decide.  Right now it's easier if we disconnect from any
idle peer once it's been active.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
0cba062318 pytest: add test to check we notice remote disconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
fcd0b2eb42 connectd: prepare for multiple subd connections.
We still always have 1, but the infrastructure is now in place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
5704653d4c setchannel: don't let them advertize htlc_maximum_msat larger than capacity.
And check for the obvious setting min > max.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
999c734bb5 setchannel: add minhtlc
Suggested by @m-schmook, I realized that if we append it later I'll
never get it right: I expect parameters min and max, not max and min!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: you can now alter the `htlc_minimum_msat` and `htlc_maximum_msat` your node advertizes.
2022-03-22 18:45:41 +10:30
Rusty Russell
f29890ed66 lightningd: check htlc_maximum_msat of channels for routehints.
We still use the channel hint here (as it's the only option), we just
warn about lack of capacity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
1751b1becc pytest: add checks that pay and getroute respect htlc_maximum_msat.
We need to add some, since our internal representations of
htlc_maximum_msat round up, and we need to disable mpp which succeeds
in getting a payment through by splitting.

We also allow dev_routes to suppress invoice routehints altogether.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
42f91ff2fa lightningd: deprecate setchannelfee, use setchannel in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `setchannelfee` (use `setchannel`).
2022-03-22 18:45:41 +10:30
Rusty Russell
43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Rusty Russell
072c4711ec pytest: convert test_sendonion_rpc to modern TLV onion.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell
45143cc731 pytest: Remove onion test vectors containing legacy onions.
I thought about fixing them up, but really these should be in
lnprototest anyway.  Turns out they're from the spec, so we should
actually fix them up there.

I moved the vector files into contrib/pyln-proto, since that still
needs them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell
0db05f6e9c lightningd: opt_var_onion is now a compulsory feature.
We're about to drop support for legacy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Michael Schmoock
ef84d6eec5 chore: remove EXPERIMENTAL for rfc #917 remote_addr 2022-03-11 16:42:45 +10:30
Michael Schmoock
07fbc7ef13 pytest: IP address discovery updates node_announcement 2022-03-11 16:42:45 +10:30
Michael Schmoock
f1981461ef connectd: ignore private remote_addr on non-DEVELOPER builds
When compiled without DEVELOPER this will now filter out `remote_addr` that
come from localhost. The testcase checks for DEVELOPER to test for correct
function of `remote_addr`.

Also, I renamed "test_connect" to "test_connect_basic" so it can be started
without all the other tests in that file that start with "test_connect..."
2022-03-11 16:42:45 +10:30
Michael Schmoock
e92176248e chore: fix typo announcable -> announceable
"announcable" is a common misspelling of "announceable", see:

https://en.wiktionary.org/wiki/announcable
2022-03-11 16:42:45 +10:30
Christian Decker
af4eed3787 cln-plugin: Make hooks asynchronous 2022-03-10 10:21:41 +10:30
Christian Decker
a7ef38732f cln-plugin: Make rpcmethod handlers async 2022-03-10 10:21:41 +10:30
Christian Decker
60e773239c cln-plugin: Add notification subscriptions and hooks to the plugins
For now hooks are treated identically to rpcmethods, with the
exception of not being returned in the `getmanifest` call. Later on we
can add typed handlers as well.
2022-03-10 10:21:41 +10:30
Christian Decker
8c6af21169 cln-plugin: Add support for synchronous RPC methods
Changelog-Experimental: cln-plugin: Added support for non-async RPC method passthrough (async support coming soon)
2022-03-10 10:21:41 +10:30
Christian Decker
4aba119733 pytest: Use valgrind target suppressions instead of skipping tests
Having a list of very targeted suppressions allows us to still run the
majority of tests with valgrind checking, and not fail when Rust does
some trickery. This is for example the case with `std::sync::Once`
which uses `num_procs` calling out to the cgroups subsystem, sometimes
with a null path.

Suggested-by: Rusty Russell <@rustyrussell>
2022-03-10 10:21:41 +10:30
Christian Decker
a1555623bc pytest: Mark Rust-dependent tests as skipped with VALGRIND
`valgrind` reports seems to flag some memory accesses that are ok in
the Rust standard library, which we can consider false positives for
our purposes:

```Valgrind error file: valgrind-errors.69147
==69147== Syscall param statx(file_name) points to unaddressable byte(s)
==69147==    at 0x4B049FE: statx (statx.c:29)
==69147==    by 0x2E2DA0: std::sys::unix::fs::try_statx (weak.rs:139)
==69147==    by 0x2D7BD5: <std::fs::File as std::io::Read>::read_to_string (fs.rs:784)
==69147==    by 0x2632CE: num_cpus::linux::Cgroup::param (linux.rs:214)
==69147==    by 0x263179: num_cpus::linux::Cgroup::quota_us (linux.rs:203)
==69147==    by 0x263002: num_cpus::linux::Cgroup::cpu_quota (linux.rs:188)
==69147==    by 0x262C01: num_cpus::linux::load_cgroups (linux.rs:149)
==69147==    by 0x26289D: num_cpus::linux::init_cgroups (linux.rs:129)
==69147==    by 0x26BD88: core::ops::function::FnOnce::call_once (function.rs:227)
==69147==    by 0x26B749: std::sync::once::Once::call_once::{{closure}} (once.rs:262)
==69147==    by 0x139717: std::sync::once::Once::call_inner (once.rs:419)
==69147==    by 0x26B6D5: std::sync::once::Once::call_once (once.rs:262)
==69147==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==69147==
==69147== Syscall param statx(buf) points to unaddressable byte(s)
==69147==    at 0x4B049FE: statx (statx.c:29)
==69147==    by 0x2E2DA0: std::sys::unix::fs::try_statx (weak.rs:139)
==69147==    by 0x2D7BD5: <std::fs::File as std::io::Read>::read_to_string (fs.rs:784)
==69147==    by 0x2632CE: num_cpus::linux::Cgroup::param (linux.rs:214)
==69147==    by 0x263179: num_cpus::linux::Cgroup::quota_us (linux.rs:203)
==69147==    by 0x263002: num_cpus::linux::Cgroup::cpu_quota (linux.rs:188)
==69147==    by 0x262C01: num_cpus::linux::load_cgroups (linux.rs:149)
==69147==    by 0x26289D: num_cpus::linux::init_cgroups (linux.rs:129)
==69147==    by 0x26BD88: core::ops::function::FnOnce::call_once (function.rs:227)
==69147==    by 0x26B749: std::sync::once::Once::call_once::{{closure}} (once.rs:262)
==69147==    by 0x139717: std::sync::once::Once::call_inner (once.rs:419)
==69147==    by 0x26B6D5: std::sync::once::Once::call_once (once.rs:262)
==69147==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==69147==

```
2022-03-10 10:21:41 +10:30
Christian Decker
fbcb4c33ad cln-plugin: Populate the options when we get an init call 2022-03-10 10:21:41 +10:30
Christian Decker
249fa8675a cln-plugin: Add options to the getmanifest call 2022-03-10 10:21:41 +10:30
Christian Decker
fe21b89b56 pytest: Add a test for the cln-plugin logging integration 2022-03-10 10:21:41 +10:30
niftynei
ecb19ba6f2 coin_mvt: report mutual close outputs also
It's better to report every single utxo on close so we know when
to mark a channel account as definitively closed.
2022-03-05 15:12:12 +10:30
niftynei
7a00277b43 coin_mvt: mark coins destined for wallet w/ originating acct
Only shows up on delayed to us outputs, but nice to have anyway.

It's missing for channel index destined deposits, maybe nice to add at
some point in the future; right now you can figure out which close a
wallet deposit comes from via the channel close txid
2022-03-05 15:12:12 +10:30
niftynei
7bfbe1492a test_close: sync l1 to the blockchain, so that the events are there
Flake on CI coming from `penalty_in/outhtlc` due to the fact that all
the events haven't arrived for the check.

If we wait to sync `l1` as well as `l2`, this should resolve the flake.

        # We use a subset of tags in expected_2 that are used in expected_1
>       tags = check_utxos_channel(l1, [channel_id], expected_1)

tests/test_closing.py:726:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/utils.py:321: in check_utxos_channel
    txid = matchup_events(u_set, evs, chans, tag_list)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

u_set = [[{'account_id': 'external', 'blockheight': 104, 'coin_type': 'bcrt', 'credit': '100000000msat', ...}, None]]
evs = [('external', ['penalty'], None, None), ('external', ['penalty'], None, None), ('external', ['penalty'], None, None)]
chans = ['2722a5fe49a8b5fa4004c19828f3e903632ac02712c6fe78ebea418daad2691f']
tag_list = {'0': '892c64c7d8c8f15d7bdcdcde34b615817d273d2e33d9d775cc9ff38e8e3deeb2', 'A': '1e69d2aa8d41eaeb78fec61227c02a6303e9f32898c10440fab5a849fea52227', 'B': '0e11e2ca01bf5f30b4c54522af172af055dcff8e3810f80a5069a2394cad74b5'}

    def matchup_events(u_set, evs, chans, tag_list):
>       assert len(u_set) == len(evs) and len(u_set) > 0
E       AssertionError
2022-03-01 16:19:34 -06:00
Simon Vrouwe
175e0b54b3 testing: fix flake in test_ping_timeout
[ Folded 2 fixups --RR ]
2022-03-01 09:38:40 +10:30
Simon Vrouwe
84bead9396 pyln-testing: in LightningNode.openchannel, make wait_for_announce more reliable
it now waits for 'alias' in node_announcement, not just block confirms.
more cleanup
2022-03-01 09:38:40 +10:30
Simon Vrouwe
2d06c38997 plugin-funder: fix typos in option lease-fee-base-msat and funder-fuzz-percent
Changelog-Experimental: option `--lease-fee-base-msat` renamed to `--lease-fee-base-sat`

Changelog-Experimental: option `--lease-fee-base-msat` deprecated and will be removed next release
2022-03-01 09:38:40 +10:30
Simon Vrouwe
9bbb52097e lightningd: fix parsing of option dev-force-features
It did not pick-up the last (bolt11) bits set.
2022-03-01 09:38:40 +10:30
Simon Vrouwe
32fb8a42fd testing: cleanup in test_plugin_shutdown 2022-03-01 09:38:40 +10:30
Christian Decker
86b83e473b pytest: Remove 3 stress-tests
These tests have proven to be:

 a) very expensive, as they spin up many nodes, and perform long setup
 b) are not testing anything specific, they just fuzz functionality
    that is already tested otherwise
 c) have not helped pinpoint any issues in living memory
 d) are very flaky, making for really bad signal-to-noise, so much
    that devs usually just restart without even looking at the logs
 e) even if we were to look at the logs, we'd be unable to reproduce
    due to the inherent randomness involved in these tests
 f) are really noisy neighbors, causing other tests to flake as well,
    further muddying the water

All in all, these tests are a waste of time, and source of
frustration.

[ Cleaned up python unused imports --RR ]
Changelog-None
2022-02-26 14:36:50 +10:30
Rusty Russell
1da9b30b9a gossipd: don't send updates in error messages for unannounced channels.
This restores the behaviour prior to `lightningd: use our cached
channel_update for errors instead of asking gossipd.`, where gossipd
would refuse to give us channel_updates for unannounced channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-23 14:47:39 +10:30
Rusty Russell
01f2ca4fe7 pytest: remove test_htlc_rexmit_while_closing as too flaky.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-23 14:46:10 +10:30
Michael Schmoock
6db97b4235 pytest: check for remote_addr 2022-02-22 05:45:47 +10:30
Rusty Russell
d0c7e18995 bitcoind: importmulti fails (bitcoin master), use importdescriptors
But this requires a watch-only wallet, and python-bitcoinlib doesn't support
multiple wallets, so we need to unload the original one, but then we need
to generate a block, so that can't generate a new address, so we need
an address arg to generate_block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-20 19:29:44 +10:30
Rusty Russell
c4ec1d576e pytest: fix test_statictor_onions
It wasn't binding to .onion:<l2.port>, but .onion:9735.  Test both cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-20 19:29:44 +10:30
Rusty Russell
00cbe6959b pytest: YA bitcoind API break.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-20 19:29:44 +10:30