Commit Graph

2092 Commits

Author SHA1 Message Date
Rusty Russell
e01abf0b34 bolt11: support payment_metadata.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
2526e804f7 doc: big BOLT update to incorporate warnings language.
We do this (send warnings) in almost all cases anyway, so mainly this
is a textual update, but there are some changes:

1. Send ERROR not WARNING if they send a malformed commitment secret.
2. Send WARNING not ERROR if they get the shutdown_scriptpubkey wrong (vs upfront)
3. Send WARNING not ERROR if they send a bad shutdown_scriptpubkey (e.g. p2pkh in future)
4. Rename some vars 'err' to 'warn' to make it clear we send a warning.

This means test_option_upfront_shutdown_script can be made reliable, too,
and it now warns and doesn't automatically close channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
9f06a59e3c shutdown: don't allow shutdown to p2pkh or p2sh addresses for anchor outputs.
This doesn't have an effect now (except in experimental mode), but it
will when we support anchors.  So we deprecate the use of those in the
close command too.

For experimental mode we have to avoid using p2pkh; adapt that test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `shutdown` no longer allows p2pkh or p2sh addresses.
2022-04-02 09:40:18 +10:30
Rusty Russell
2f7f7ec255 lightningd: add "style" to listforwards.
Suggested-by: @t-bast
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listforwards` has new entry `style`, currently "legacy" or "tlv".
2022-03-31 19:38:05 +10:30
Rusty Russell
116a77f1be lightningd: Restore forwarding of legacy onions.
Partial revert of 43a833e405
"lightningd: remove support for legacy onion format."; we restore the
ability to decode legacy onions for forwarding, but not to generate them.
(We don't accept them properly since making payment_secret compulsory
anyway, so no real change there!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: ... but we still forward legacy HTLC onions for now.
2022-03-31 19:38:05 +10:30
Rusty Russell
141d4ef675 pytest: test legacy onion acceptance.
Using a canned sendonion call from an old version (v0.10.2), where I
forced it to use legacy onion for l2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 19:38:05 +10:30
Rusty Russell
5033e22835 pytest: make test_onchain_close_upstream more accurately reflect report.
1. Don't use dust HTLCs.
2. Make l3 unresponsive, like report.
3. Make l2-l3 fail because we time out on successive HTLC.

We use sendpay rather than pay, because pay can do multiple attempts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell
1e08c3b882 pytest: simple "does onchaind on missing HTLC cause upstream failure" test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-31 13:40:27 +10:30
Rusty Russell
0c3aa355db tests: add test that pay gets update correct.
This is the part which works: we have another old PR (or simply
uncomment the FIXME section) for the routehint case.

Closes: #4781
See-also: #4808
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-30 14:18:39 +10:30
Rusty Russell
ae5b98a727 pytest: fix flake due to cln-grpc starting before "public key" message.
e.g.
```
lightningd-1: 2022-03-28T11:02:12.476Z DEBUG   plugin-cln-grpc: add_pem_file processed 1 valid and 0 invalid certs
lightningd-1: 2022-03-28T11:02:12.478Z DEBUG   plugin-cln-grpc: Connecting to \"lightning-rpc\" and serving grpc on 0.0.0.0:36331
lightningd-1: 2022-03-28T11:02:12.478Z DEBUG   connectd: REPLY WIRE_CONNECTD_ACTIVATE_REPLY with 0 fds
lightningd-1: 2022-03-28T11:02:12.478Z INFO    lightningd: --------------------------------------------------
lightningd-1: 2022-03-28T11:02:12.478Z INFO    lightningd: Server started with public key
```

Which means we don't see it, since start() swallows it:

```
>               raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E               TimeoutError: Unable to find "[re.compile('serving grpc on 0.0.0.0:')]" in logs.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-30 12:15:55 +10:30
Christian Decker
09ee28cb51 cln-grpc: Do not start unless a grpc-port is specified
For now we don't want to autostart.

Suggested-by: Rusty Russell <@rustyrussell>
2022-03-30 12:15:55 +10:30
Christian Decker
dd66c85fcb grpc-plugin: Make the grpc port to listen on configurable
Changelog-Added: cln-grpc-plugin: The plugin can be configured to listen on a specific port using the `grpc-port` option
2022-03-30 12:15:55 +10:30
Christian Decker
f5147bbf1e pytest: Add a test for incorrect credentials
If we aren't using the correct certificates we should reject the
connections during the mTLS connection setup. This test tries to
connect with the wrong client cert to the node, and the server will
reject it.
2022-03-30 12:15:55 +10:30
Christian Decker
27e468d2ae grpc-plugin: Generate mTLS certificates and use them in grpc 2022-03-30 12:15:55 +10:30
Christian Decker
d221c9b491 pytest: Add a test for the grpc plugin
Currently still unencrypted, but will get its mTLS authentication in
the next commits.
2022-03-30 12:15:55 +10:30
Rusty Russell
20392ae526 connectd: restore obs2 onion support.
I removed these prematurely: we *haven't* had a release since
introducing them!

This consists of reverting d15d629b8b
"plugins/fetchinvoice: remove obsolete string-based API." and
plugins/fetchinvoice: remove obsolete string-based
API. "onion_messages: remove obs2 support."

Some minor changes due to updated fromwire_tlv API since they
were removed, but not much.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: REVERT: Removed backwards compat with onion messages from v0.10.1.
2022-03-29 10:55:12 +10:30
Rusty Russell
aad4495f56 delinvoice: allow desconly arg to only remove the description.
Means that field is now optional in JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `delinvoice` has a new parameter `desconly` to remove description.
2022-03-29 10:04:16 +10:30
Rusty Russell
ccaf04d268 invoice: add deschashonly parameter.
LNURL wants this so they can include images etc in descriptions.

Replaces: #4892
Changelog-Added: JSON-RPC: `invoice` has a new parameter `deschashonly` to put hash of description in bolt11.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-29 10:04:16 +10:30
Rusty Russell
7829f2eb06 onion_messages: remove obs2 support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Removed backwards compat with onion messages from v0.10.1.
2022-03-25 13:55:44 +10:30
Rusty Russell
65f5bb2638 pytest: test for compat code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
5aa108f176 pytest: fix flake in test_pay.py::test_setchannel_state
The second disconnect could fail, if slow enough:

```
>       l1.rpc.disconnect(l2.info['id'], force=True)

tests/test_pay.py:2067:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:690: in disconnect
    return self.call("disconnect", payload)
contrib/pyln-testing/pyln/testing/utils.py:639: in call
    res = LightningRpc.call(self, method, payload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pyln.testing.utils.PrettyPrintingLightningRpc object at 0x7fb139ce87d0>
method = 'disconnect'
payload = {'force': True, 'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d'}

...
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: disconnect, payload: {'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d', 'force': True}, error: {'code': -1, 'message': 'Peer not connected'}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-24 09:53:38 +10:30
Vincenzo Palazzo
53806d1abd cli: make the command line more user friendly.
Also has to fix up tests.

Changelog-Fixed: cli doesn't required anymore to confirm the password if the `hsm_secret` is already encrypted.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30
Rusty Russell
be75518559 pytest: fix bad gossip flake in test_multifunding_v1_v2_mixed
This was missed in e8d2176e6b.

```
>           raise ValueError(str(errors))
E           ValueError: 
E           Node errors:
E            - lightningd-2: had bad gossip messages
E            - lightningd-3: had bad gossip messages
E           Global errors:

contrib/pyln-testing/pyln/testing/fixtures.py:201: ValueError

...

0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Ignoring future channel_announcment for 105x1x2 (current block 104)
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-gossipd: Bad gossip order: WIRE_CHANNEL_UPDATE before announcement 105x1x2/0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
bcf3cef96c pytest: fix flake in test_connection.py::test_opener_feerate_reconnect
Make sure it sees disconnect before reconnect, otherwise the next command
fails since we're now disconnected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
2a80400a0f pytest: fix test_connection.py::test_funding_close_upfront fake
Breaks when there are no peers:

```
>       _fundchannel(l1, l2, amt_normal, None)

tests/test_connection.py:1564: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_connection.py:1535: in _fundchannel
    wait_for(lambda: not has_normal_channels(l2, l1))
contrib/pyln-testing/pyln/testing/utils.py:88: in wait_for
    while not success():
tests/test_connection.py:1535: in <lambda>
    wait_for(lambda: not has_normal_channels(l2, l1))
tests/test_connection.py:1527: in has_normal_channels
    for c in only_one(l1.rpc.listpeers(l2.info['id'])['peers'])['channels']])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

arr = []

    def only_one(arr):
        """Many JSON RPC calls return an array; often we only expect a single entry
        """
>       assert len(arr) == 1
E       AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
486b1b2481 pytest: fix flake in test_funding_fail
We may not see a disconnect instantly:

```
>       assert len(l2.rpc.listpeers()['peers']) == 0
E       assert 1 == 0
E         +1
E         -0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
aeeaa14430 pytest: add reconnect and restart to test_multichan.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
8458994a7f pytest: test closing one multichannel works as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
75596b3e0f lightningd: use a better channel if available to next hop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
debc1b90d3 lightningd: remove checks which prevent us from opening multiple channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now support opening multiple channels with the same peer.
2022-03-23 13:20:12 +10:30
Rusty Russell
21e1d68e3b lightningd: remove (most) functions to search channels by status.
This is generally verboten now, since there can be multiple.  There are a
few exceptions:

1. We sometimes want to know if there are *any* active channels.
2. Some dev commands still take peer id when they mean channel_id.
3. We still allow peer id when it's fully determined.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `close` by peer id will fail if there is more than one live channel (use `channel_id` or `short_channel_id` as id arg).
2022-03-23 13:20:12 +10:30
Rusty Russell
cb5dc48cab lightningd: make setchannelfee handle multiple channels per peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
90be2cc104 lightningd: remove some "single active channel" assumptions.
Generally this means converting a lazy "peer_active_channel(peer)" call
into an explicit iteration.

1. notify_feerate_change: call all channels (ignores non-active ones anyway).
2. peer_get_owning_subd remove unused function.
3. peer_connected hook: don't save channel, do lookup and iterate channels.
4. In json_setchannelfee "all" remove useless call to peer_active_channel
   since we check state anyway, and iterate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
7de7b7be61 lightningd: use channel_id when a peer is activated.
Rather than intuiting whether this is a new channel / active channel,
use the channel_id.  This simplifies things and makes them explicit,
and prepares for multiple live channels per peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
2424b7dea8 connectd: hold peer until we're interested.
Either because lightningd tells us it wants to talk, or because the peer
says something about a channel.

We also introduce a behavior change: we disconnect after a failed open.
We might want to modify this later, but we it's a side-effect of openingd
not holding onto idle connections.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
deecedb033 connectd: tell lightningd when disconnect is complete.
This avoids races in our tests where we assume it's sync (and is kind
of nicer).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
10e36e073c openingd: disconnect from peer when an error occurs.
openingd currently holds the connection to idle peers, but we're about
to change that: it will only look after peers which are actively
opening a connection.  We can start this process by disconnecting
whenever we have a negotiation failure.

We could stay connected if we wanted to, but that would be up to
connectd to decide.  Right now it's easier if we disconnect from any
idle peer once it's been active.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
0cba062318 pytest: add test to check we notice remote disconnects.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
fcd0b2eb42 connectd: prepare for multiple subd connections.
We still always have 1, but the infrastructure is now in place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-23 13:20:12 +10:30
Rusty Russell
5704653d4c setchannel: don't let them advertize htlc_maximum_msat larger than capacity.
And check for the obvious setting min > max.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
999c734bb5 setchannel: add minhtlc
Suggested by @m-schmook, I realized that if we append it later I'll
never get it right: I expect parameters min and max, not max and min!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: you can now alter the `htlc_minimum_msat` and `htlc_maximum_msat` your node advertizes.
2022-03-22 18:45:41 +10:30
Rusty Russell
f29890ed66 lightningd: check htlc_maximum_msat of channels for routehints.
We still use the channel hint here (as it's the only option), we just
warn about lack of capacity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
1751b1becc pytest: add checks that pay and getroute respect htlc_maximum_msat.
We need to add some, since our internal representations of
htlc_maximum_msat round up, and we need to disable mpp which succeeds
in getting a payment through by splitting.

We also allow dev_routes to suppress invoice routehints altogether.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-22 18:45:41 +10:30
Rusty Russell
42f91ff2fa lightningd: deprecate setchannelfee, use setchannel in tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `setchannelfee` (use `setchannel`).
2022-03-22 18:45:41 +10:30
Rusty Russell
43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Rusty Russell
072c4711ec pytest: convert test_sendonion_rpc to modern TLV onion.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell
45143cc731 pytest: Remove onion test vectors containing legacy onions.
I thought about fixing them up, but really these should be in
lnprototest anyway.  Turns out they're from the spec, so we should
actually fix them up there.

I moved the vector files into contrib/pyln-proto, since that still
needs them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Rusty Russell
0db05f6e9c lightningd: opt_var_onion is now a compulsory feature.
We're about to drop support for legacy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-18 09:20:11 +10:30
Michael Schmoock
ef84d6eec5 chore: remove EXPERIMENTAL for rfc #917 remote_addr 2022-03-11 16:42:45 +10:30
Michael Schmoock
07fbc7ef13 pytest: IP address discovery updates node_announcement 2022-03-11 16:42:45 +10:30