Commit Graph

2080 Commits

Author SHA1 Message Date
Christian Decker
f5147bbf1e pytest: Add a test for incorrect credentials
If we aren't using the correct certificates we should reject the
connections during the mTLS connection setup. This test tries to
connect with the wrong client cert to the node, and the server will
reject it.
2022-03-30 12:15:55 +10:30
Christian Decker
27e468d2ae grpc-plugin: Generate mTLS certificates and use them in grpc 2022-03-30 12:15:55 +10:30
Christian Decker
d221c9b491 pytest: Add a test for the grpc plugin
Currently still unencrypted, but will get its mTLS authentication in
the next commits.
2022-03-30 12:15:55 +10:30
Rusty Russell
20392ae526 connectd: restore obs2 onion support.
I removed these prematurely: we *haven't* had a release since
introducing them!

This consists of reverting d15d629b8b
"plugins/fetchinvoice: remove obsolete string-based API." and
plugins/fetchinvoice: remove obsolete string-based
API. "onion_messages: remove obs2 support."

Some minor changes due to updated fromwire_tlv API since they
were removed, but not much.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: REVERT: Removed backwards compat with onion messages from v0.10.1.
2022-03-29 10:55:12 +10:30
Rusty Russell
aad4495f56 delinvoice: allow desconly arg to only remove the description.
Means that field is now optional in JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `delinvoice` has a new parameter `desconly` to remove description.
2022-03-29 10:04:16 +10:30
Rusty Russell
ccaf04d268 invoice: add deschashonly parameter.
LNURL wants this so they can include images etc in descriptions.

Replaces: #4892
Changelog-Added: JSON-RPC: `invoice` has a new parameter `deschashonly` to put hash of description in bolt11.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-29 10:04:16 +10:30
Rusty Russell
7829f2eb06 onion_messages: remove obs2 support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Removed backwards compat with onion messages from v0.10.1.
2022-03-25 13:55:44 +10:30
Rusty Russell
65f5bb2638 pytest: test for compat code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
5aa108f176 pytest: fix flake in test_pay.py::test_setchannel_state
The second disconnect could fail, if slow enough:

```
>       l1.rpc.disconnect(l2.info['id'], force=True)

tests/test_pay.py:2067:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:690: in disconnect
    return self.call("disconnect", payload)
contrib/pyln-testing/pyln/testing/utils.py:639: in call
    res = LightningRpc.call(self, method, payload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pyln.testing.utils.PrettyPrintingLightningRpc object at 0x7fb139ce87d0>
method = 'disconnect'
payload = {'force': True, 'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d'}

...
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: disconnect, payload: {'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d', 'force': True}, error: {'code': -1, 'message': 'Peer not connected'}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-24 09:53:38 +10:30
Vincenzo Palazzo
53806d1abd cli: make the command line more user friendly.
Also has to fix up tests.

Changelog-Fixed: cli doesn't required anymore to confirm the password if the `hsm_secret` is already encrypted.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30
Rusty Russell
be75518559 pytest: fix bad gossip flake in test_multifunding_v1_v2_mixed
This was missed in e8d2176e6b.

```
>           raise ValueError(str(errors))
E           ValueError: 
E           Node errors:
E            - lightningd-2: had bad gossip messages
E            - lightningd-3: had bad gossip messages
E           Global errors:

contrib/pyln-testing/pyln/testing/fixtures.py:201: ValueError

...

0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 105x1x2 (current block 104)
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Bad gossip order: WIRE_CHANNEL_UPDATE before announcement 105x1x2/0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
bcf3cef96c pytest: fix flake in test_connection.py::test_opener_feerate_reconnect
Make sure it sees disconnect before reconnect, otherwise the next command
fails since we're now disconnected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
2a80400a0f pytest: fix test_connection.py::test_funding_close_upfront fake
Breaks when there are no peers:

```
>       _fundchannel(l1, l2, amt_normal, None)

tests/test_connection.py:1564: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_connection.py:1535: in _fundchannel
    wait_for(lambda: not has_normal_channels(l2, l1))
contrib/pyln-testing/pyln/testing/utils.py:88: in wait_for
    while not success():
tests/test_connection.py:1535: in <lambda>
    wait_for(lambda: not has_normal_channels(l2, l1))
tests/test_connection.py:1527: in has_normal_channels
    for c in only_one(l1.rpc.listpeers(l2.info['id'])['peers'])['channels']])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

arr = []

    def only_one(arr):
        """Many JSON RPC calls return an array; often we only expect a single entry
        """
>       assert len(arr) == 1
E       AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
486b1b2481 pytest: fix flake in test_funding_fail
We may not see a disconnect instantly:

```
>       assert len(l2.rpc.listpeers()['peers']) == 0
E       assert 1 == 0
E         +1
E         -0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
aeeaa14430 pytest: add reconnect and restart to test_multichan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
8458994a7f pytest: test closing one multichannel works as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
75596b3e0f lightningd: use a better channel if available to next hop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
debc1b90d3 lightningd: remove checks which prevent us from opening multiple channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now support opening multiple channels with the same peer.
2022-03-23 13:20:12 +10:30
Rusty Russell
21e1d68e3b lightningd: remove (most) functions to search channels by status.
This is generally verboten now, since there can be multiple.  There are a
few exceptions:

1. We sometimes want to know if there are *any* active channels.
2. Some dev commands still take peer id when they mean channel_id.
3. We still allow peer id when it's fully determined.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `close` by peer id will fail if there is more than one live channel (use `channel_id` or `short_channel_id` as id arg).
2022-03-23 13:20:12 +10:30
Rusty Russell
cb5dc48cab lightningd: make setchannelfee handle multiple channels per peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
90be2cc104 lightningd: remove some "single active channel" assumptions.
Generally this means converting a lazy "peer_active_channel(peer)" call
into an explicit iteration.

1. notify_feerate_change: call all channels (ignores non-active ones anyway).
2. peer_get_owning_subd remove unused function.
3. peer_connected hook: don't save channel, do lookup and iterate channels.
4. In json_setchannelfee "all" remove useless call to peer_active_channel
   since we check state anyway, and iterate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
7de7b7be61 lightningd: use channel_id when a peer is activated.
Rather than intuiting whether this is a new channel / active channel,
use the channel_id.  This simplifies things and makes them explicit,
and prepares for multiple live channels per peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
2424b7dea8 connectd: hold peer until we're interested.
Either because lightningd tells us it wants to talk, or because the peer
says something about a channel.

We also introduce a behavior change: we disconnect after a failed open.
We might want to modify this later, but we it's a side-effect of openingd
not holding onto idle connections.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
deecedb033 connectd: tell lightningd when disconnect is complete.
This avoids races in our tests where we assume it's sync (and is kind
of nicer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
10e36e073c openingd: disconnect from peer when an error occurs.
openingd currently holds the connection to idle peers, but we're about
to change that: it will only look after peers which are actively
opening a connection.  We can start this process by disconnecting
whenever we have a negotiation failure.

We could stay connected if we wanted to, but that would be up to
connectd to decide.  Right now it's easier if we disconnect from any
idle peer once it's been active.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
0cba062318 pytest: add test to check we notice remote disconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
fcd0b2eb42 connectd: prepare for multiple subd connections.
We still always have 1, but the infrastructure is now in place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
5704653d4c setchannel: don't let them advertize htlc_maximum_msat larger than capacity.
And check for the obvious setting min > max.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
999c734bb5 setchannel: add minhtlc
Suggested by @m-schmook, I realized that if we append it later I'll
never get it right: I expect parameters min and max, not max and min!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: you can now alter the `htlc_minimum_msat` and `htlc_maximum_msat` your node advertizes.
2022-03-22 18:45:41 +10:30
Rusty Russell
f29890ed66 lightningd: check htlc_maximum_msat of channels for routehints.
We still use the channel hint here (as it's the only option), we just
warn about lack of capacity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
1751b1becc pytest: add checks that pay and getroute respect htlc_maximum_msat.
We need to add some, since our internal representations of
htlc_maximum_msat round up, and we need to disable mpp which succeeds
in getting a payment through by splitting.

We also allow dev_routes to suppress invoice routehints altogether.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
42f91ff2fa lightningd: deprecate setchannelfee, use setchannel in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `setchannelfee` (use `setchannel`).
2022-03-22 18:45:41 +10:30
Rusty Russell
43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Rusty Russell
072c4711ec pytest: convert test_sendonion_rpc to modern TLV onion.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell
45143cc731 pytest: Remove onion test vectors containing legacy onions.
I thought about fixing them up, but really these should be in
lnprototest anyway.  Turns out they're from the spec, so we should
actually fix them up there.

I moved the vector files into contrib/pyln-proto, since that still
needs them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell
0db05f6e9c lightningd: opt_var_onion is now a compulsory feature.
We're about to drop support for legacy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Michael Schmoock
ef84d6eec5 chore: remove EXPERIMENTAL for rfc #917 remote_addr 2022-03-11 16:42:45 +10:30
Michael Schmoock
07fbc7ef13 pytest: IP address discovery updates node_announcement 2022-03-11 16:42:45 +10:30
Michael Schmoock
f1981461ef connectd: ignore private remote_addr on non-DEVELOPER builds
When compiled without DEVELOPER this will now filter out `remote_addr` that
come from localhost. The testcase checks for DEVELOPER to test for correct
function of `remote_addr`.

Also, I renamed "test_connect" to "test_connect_basic" so it can be started
without all the other tests in that file that start with "test_connect..."
2022-03-11 16:42:45 +10:30
Michael Schmoock
e92176248e chore: fix typo announcable -> announceable
"announcable" is a common misspelling of "announceable", see:

https://en.wiktionary.org/wiki/announcable
2022-03-11 16:42:45 +10:30
Christian Decker
af4eed3787 cln-plugin: Make hooks asynchronous 2022-03-10 10:21:41 +10:30
Christian Decker
a7ef38732f cln-plugin: Make rpcmethod handlers async 2022-03-10 10:21:41 +10:30
Christian Decker
60e773239c cln-plugin: Add notification subscriptions and hooks to the plugins
For now hooks are treated identically to rpcmethods, with the
exception of not being returned in the `getmanifest` call. Later on we
can add typed handlers as well.
2022-03-10 10:21:41 +10:30
Christian Decker
8c6af21169 cln-plugin: Add support for synchronous RPC methods
Changelog-Experimental: cln-plugin: Added support for non-async RPC method passthrough (async support coming soon)
2022-03-10 10:21:41 +10:30
Christian Decker
4aba119733 pytest: Use valgrind target suppressions instead of skipping tests
Having a list of very targeted suppressions allows us to still run the
majority of tests with valgrind checking, and not fail when Rust does
some trickery. This is for example the case with `std::sync::Once`
which uses `num_procs` calling out to the cgroups subsystem, sometimes
with a null path.

Suggested-by: Rusty Russell <@rustyrussell>
2022-03-10 10:21:41 +10:30
Christian Decker
a1555623bc pytest: Mark Rust-dependent tests as skipped with VALGRIND
`valgrind` reports seems to flag some memory accesses that are ok in
the Rust standard library, which we can consider false positives for
our purposes:

```Valgrind error file: valgrind-errors.69147
==69147== Syscall param statx(file_name) points to unaddressable byte(s)
==69147==    at 0x4B049FE: statx (statx.c:29)
==69147==    by 0x2E2DA0: std::sys::unix::fs::try_statx (weak.rs:139)
==69147==    by 0x2D7BD5: <std::fs::File as std::io::Read>::read_to_string (fs.rs:784)
==69147==    by 0x2632CE: num_cpus::linux::Cgroup::param (linux.rs:214)
==69147==    by 0x263179: num_cpus::linux::Cgroup::quota_us (linux.rs:203)
==69147==    by 0x263002: num_cpus::linux::Cgroup::cpu_quota (linux.rs:188)
==69147==    by 0x262C01: num_cpus::linux::load_cgroups (linux.rs:149)
==69147==    by 0x26289D: num_cpus::linux::init_cgroups (linux.rs:129)
==69147==    by 0x26BD88: core::ops::function::FnOnce::call_once (function.rs:227)
==69147==    by 0x26B749: std::sync::once::Once::call_once::{{closure}} (once.rs:262)
==69147==    by 0x139717: std::sync::once::Once::call_inner (once.rs:419)
==69147==    by 0x26B6D5: std::sync::once::Once::call_once (once.rs:262)
==69147==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==69147==
==69147== Syscall param statx(buf) points to unaddressable byte(s)
==69147==    at 0x4B049FE: statx (statx.c:29)
==69147==    by 0x2E2DA0: std::sys::unix::fs::try_statx (weak.rs:139)
==69147==    by 0x2D7BD5: <std::fs::File as std::io::Read>::read_to_string (fs.rs:784)
==69147==    by 0x2632CE: num_cpus::linux::Cgroup::param (linux.rs:214)
==69147==    by 0x263179: num_cpus::linux::Cgroup::quota_us (linux.rs:203)
==69147==    by 0x263002: num_cpus::linux::Cgroup::cpu_quota (linux.rs:188)
==69147==    by 0x262C01: num_cpus::linux::load_cgroups (linux.rs:149)
==69147==    by 0x26289D: num_cpus::linux::init_cgroups (linux.rs:129)
==69147==    by 0x26BD88: core::ops::function::FnOnce::call_once (function.rs:227)
==69147==    by 0x26B749: std::sync::once::Once::call_once::{{closure}} (once.rs:262)
==69147==    by 0x139717: std::sync::once::Once::call_inner (once.rs:419)
==69147==    by 0x26B6D5: std::sync::once::Once::call_once (once.rs:262)
==69147==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==69147==

```
2022-03-10 10:21:41 +10:30
Christian Decker
fbcb4c33ad cln-plugin: Populate the options when we get an init call 2022-03-10 10:21:41 +10:30
Christian Decker
249fa8675a cln-plugin: Add options to the getmanifest call 2022-03-10 10:21:41 +10:30
Christian Decker
fe21b89b56 pytest: Add a test for the cln-plugin logging integration 2022-03-10 10:21:41 +10:30
niftynei
ecb19ba6f2 coin_mvt: report mutual close outputs also
It's better to report every single utxo on close so we know when
to mark a channel account as definitively closed.
2022-03-05 15:12:12 +10:30