Commit graph

14841 commits

Author SHA1 Message Date
Christian Decker
a7f4c28f5e pay: Remove the waitheight paymod
It is getting replaced with a paymod that adjusts the chain height
used to offset all calculations from.
2024-05-08 23:18:27 -05:00
Christian Decker
c95b70d52a pytest: Add a test for the height mismatch logic 2024-05-08 23:18:27 -05:00
Lagrang3
7335777595 renepay: fix some memory leaks 2024-05-08 22:32:13 -05:00
Lagrang3
7b18056b09 renepay: fix sources 2024-05-08 22:32:13 -05:00
Lagrang3
985a01b41d renepay: wake-up result collector every 50msec 2024-05-08 22:32:13 -05:00
Lagrang3
014e32ca2f renepay: a bit more verbose information in logs 2024-05-08 22:32:13 -05:00
Lagrang3
e97e6ede41 renepay: update test files 2024-05-08 22:32:13 -05:00
Lagrang3
16d00e09ef renepay: bug fix, local channel information
Listpeerchannels would update the local channel information setting the
liquidity in the outgoing channel to known_min=known_max=capacity,
when in fact it should be known_min=known_max=spendable.
2024-05-08 22:32:13 -05:00
Lagrang3
4adf9b4080 renepay: bug fix on the routebuilder
The route-builder checks the liquidity bounds of each route one at a
time. Every route that satisfy the contraints is recorded in the
uncertainty network and produces an HTLC burden on the channels it uses,
so that the following routes cannot count on the same liquidity twice.
2024-05-08 22:32:13 -05:00
Lagrang3
da00cae30b renepay: remove payment from route
Routes contain only routing information and the payment they're linked
to can be obtained through the payment_hash. We remove the dependency of
route building routines from the payment itself. In order to make
plain payment information available we define a payment_info structure.
2024-05-08 22:32:13 -05:00
Lagrang3
ca2b4e54ae renepay: add disabledmap abstraction
Define a new object called disabledmap that carries information about
the disabled channels and nodes.
2024-05-08 22:32:13 -05:00
Lagrang3
96bd0e455b renepay: refactor routefail
- use switch case over all possible WIRE_* errors,
- remove the virtual machine for routefail, use a simple two step
  solution: 1. update the gossip and 2. handle error cases
2024-05-08 22:32:13 -05:00
Lagrang3
cafc08be1d renepay: disable channels not in chan_extra_map
Expand the disabled set to include channels that are not present in the
chan_extra_map/uncertainty network.
2024-05-08 22:32:13 -05:00
Lagrang3
d2c44fc3df renepay: add a test for routehints 2024-05-08 22:32:13 -05:00
Lagrang3
761c295199 renepay: uncertainty network update revisited
- Update the uncertainty network with the gossmap+local_gossmods,
- ignore channels that fail to give their capacity.
2024-05-08 22:32:13 -05:00
Lagrang3
9f13621de9 renepay: pay selfpayments on spot
Resolve a selfpayment right from the result of `sendpay` instead of
waiting for the notification.
2024-05-08 22:32:13 -05:00
Lagrang3
fa199e21d2 renepay: minor fixes 2024-05-08 22:32:13 -05:00
Lagrang3
0e2660733f renepay: update unit tests 2024-05-08 22:32:13 -05:00
Lagrang3
13a2828b12 renepay: add a test for concurrent payments 2024-05-08 22:32:13 -05:00
Lagrang3
1ee157f02b renepay: update the Makefile 2024-05-08 22:32:13 -05:00
Lagrang3
cc00d509b1 renepay: add a helper module json.c 2024-05-08 22:32:13 -05:00
Lagrang3
73c6142965 renepay: add error codes
Using enum renepay_errorcode simplifies the low level API of chan_extra and flow.
We can extract information about the nature of a function call failure
from its return value.
2024-05-08 22:32:13 -05:00
Lagrang3
7584629696 renepay: add routefail.c
Routefail consist mainly of an API `routefail_start` that handles the
failure of a forwarding request (encoded in a route). Internally there
is a routefail datastructure that goes through a series of execution
steps, eg. updating the gossmap, updating the uncertainty network, etc.
2024-05-08 22:32:13 -05:00
Lagrang3
480dec58f2 renepay: add routetracker.c
Routetracker is a structure that is used to follow the progress of the
route forwarding requests.
2024-05-08 22:32:13 -05:00
Lagrang3
84651fa4ba renepay: add routebuilder.c
Routebuilder constitute the module that builds the routes we try in the
payment. The public API is simply `get_routes`, it replaces the similar
interface for pay_flows.
2024-05-08 22:32:13 -05:00
Lagrang3
6766acbcdf renepay: payment state machine (mods)
Add a new implementation of the payment state machine.
This is based in the `pay` plugin concept of payment modifiers,
but here we take it to the next level.
The payment goes through a virtual machine that includes calling
functions and evaluating conditions.
2024-05-08 22:32:13 -05:00
Lagrang3
4b9fe23293 renepay: the plugin executable is main.c
- move the plugin executable from pay.c to main.c,
- adapt the plugin main program to handle concurrent pay calls,
2024-05-08 22:32:13 -05:00
Lagrang3
190649585f renepay: THE plugin global structure
The plugin global structure is now defined in payplugin.h
2024-05-08 22:32:13 -05:00
Lagrang3
7ffb722b27 renepay: refactor uncertainty API
The uncertainty structure is updated based on the result of a route,
previously chan_extra and pay_flow were used instead.
2024-05-08 22:32:13 -05:00
Lagrang3
2a4a23a362 renepay: rename uncertainty_network for uncertanty 2024-05-08 22:32:13 -05:00
Lagrang3
4c91bc0ef0 renepay: abandon pay_flow for route
The role of the structure pay_flow is not taken by a new structure
called route.
2024-05-08 22:32:13 -05:00
Lagrang3
ba54d678bc renepay: concurrent pay calls
Refactor the payment structure to handle multiple pay requests for the
same or different invoices.
2024-05-08 22:32:13 -05:00
Lagrang3
3cbbaa7ab5 renepay: separate flow and chan_extra
Flow and chan_extra are two different concepts, we keep their
definitions and APIs separate.
2024-05-08 22:32:13 -05:00
Vincenzo Palazzo
e8372e7481 plugin: notify only the plugins with init state
This is fixing a bug that @chrisguida reported,
when we notify a plugin we also notify a plugin
that it is in the middle of initialization.

So imagine the following use case:

- 1. Plugin A sends get manifest
- 2. cln makes a check and generates a warning
- 3. Plugin A is subscribed to the warning or * notification
- 4. Core Lightning gets upset because it receives a warning message but it was waiting for a init.

I think it is still a bug in how Core Lightning handles the init, but also I think we should communicate with the plugin only if it is initialized.

```
lightningd-1 2024-04-23T23:20:34.154Z DEBUG   plugin-clnrest: Notification: {'warning': {'level': 'warn', 'time': '1713914433.818230531', 'timestamp': '2024-04-23T23:20:33.818Z', 'source': 'plugin-bookkeeper', 'log': \"topic 'utxo_deposit' is not a known notification topic\"}}
lightningd-1 2024-04-23T23:20:34.154Z DEBUG   hsmd: new_client: 0
lightningd-1 2024-04-23T23:20:34.154Z **BROKEN** plugin-test_libplugin: Did not receive 'init' yet, but got 'warning' instead
lightningd-1 2024-04-23T23:20:34.155Z INFO    plugin-test_libplugin: Killing plugin: exited before we sent init
lightningd-1 2024-04-23T23:20:34.155Z **BROKEN** plugin-test_libplugin: Plugin marked as important, shutting down lightningd!
lightningd-1 2024-04-23T23:20:34.155Z DEBUG   lightningd: io_break: lightningd_exit
lightningd-1 2024-04-23T23:20:34.155Z DEBUG   lightningd: io_loop: connectd_init
Time-out: can't find [re.compile('Server started with public key')] in logs
{'github_repository': 'ElementsProject/lightning', 'github_sha': '014c1eb383b0a65394cf166b3aa0174cf2077896', 'github_ref': 'refs/pull/7258/merge', 'github_ref_name': 'HEAD', 'github_run_id': 8808124001, 'github_head_ref': 'cguida/onchain_notif', 'github_run_number': 9395, 'github_base_ref': 'master', 'github_run_attempt': '1', 'testname': 'test_self_disable', 'start_time': 1713914432, 'end_time': 1713914612, 'outcome': 'fail'}
----------------------------- Captured stderr call -----------------------------
Did not receive 'init' yet, but got 'warning' insteadlightningd: lightningd/connect_control.c:767: connectd_init: Assertion `ret == ld->connectd' failed.
lightningd: FATAL SIGNAL 6 (version 014c1eb-modded)
0x555b1dc2d6f3 send_backtrace
	common/daemon.c:33
0x555b1dc2d8de crashdump
	common/daemon.c:75
0x7fda4584251f ???
	???:0
0x7fda458969fc ???
	???:0
0x7fda45842475 ???
	???:0
0x7fda458287f2 ???
	???:0
0x7fda4582871a ???
	???:0
0x7fda45839e95 ???
	???:0
0x555b1db98f8e connectd_init
	lightningd/connect_control.c:767
0x555b1dbb0307 main
	lightningd/lightningd.c:1249
0x7fda45829d8f ???
	???:0
0x7fda45829e3f ???
	???:0
0x555b1db6fd64 ???
	???:0
0xffffffffffffffff ???
	???:0
```

Reported-by: @chrisguida
Co-Developed-by: @chrisguida
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-05-08 21:57:37 -05:00
Rusty Russell
1c3312c8ac pytest: clean up tests/test_db.py::test_last_tx_psbt_upgrade
Ignore bad gossip, and use the broken_log option rather than manually filtering logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-08 21:54:50 -05:00
Rusty Russell
61cc3aa887 pytest: catch more broken messages in test_important_plugins.
This can happen, so allow them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-08 21:54:50 -05:00
Rusty Russell
92c28a5bcd Add missing docs and Changelog-Added line.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: new log level `trace` where we moved the very noisiest `debug` logs.
2024-05-08 21:05:49 -05:00
Ken Sedgwick
5b2c237718 drive-by: fix valgrind complaint about uninitialized memory 2024-05-08 21:05:49 -05:00
Ken Sedgwick
e93ad5d124 logging: move noisy gossip recv message to TRACE
```
lightningd-test   | 2024-04-26T13:07:01.581Z DEBUG   031c64a68e6d1b9e50711336d92b434c584ce668b2fae59ee688bd73713fee1569-gossipd: handle_recv_gossip: WIRE_CHANNEL_UPDATE
lightningd-test   | 2024-04-26T13:07:01.581Z DEBUG   031c64a68e6d1b9e50711336d92b434c584ce668b2fae59ee688bd73713fee1569-gossipd: handle_recv_gossip: WIRE_CHANNEL_UPDATE
lightningd-test   | 2024-04-26T13:07:01.582Z DEBUG   031c64a68e6d1b9e50711336d92b434c584ce668b2fae59ee688bd73713fee1569-gossipd: handle_recv_gossip: WIRE_CHANNEL_UPDATE
```
2024-05-08 21:05:49 -05:00
Ken Sedgwick
788e102482 logging: move two noisy plugin_hook logging entries to TRACE
These are very noisy at the debug level:
DEBUG   lightningd: Calling rpc_command hook of plugin clboss
DEBUG   lightningd: Plugin clboss returned from rpc_command hook call

As seen in https://github.com/ZmnSCPxj/clboss/issues/194
2024-05-08 21:05:49 -05:00
Ken Sedgwick
e3c600b0c8 logging: add TRACE between DEBUG and IO
By moving super-noisy debugging logging to new TRACE level we can
make long-term logging at DEBUG possible w/o removing any logging.

Addresses ([#7279])

Related Issues:
https://github.com/ElementsProject/lightning/issues/6927
https://github.com/chrisguida/smaug/issues/34
https://github.com/ZmnSCPxj/clboss/issues/194
2024-05-08 21:05:49 -05:00
Alex Myers
c3280676e8 lightningd: avoid crash from hsmd not signing anchor spend
Temporary patch to avoid issue #6973

[ Add much more logging! --RR ]

Changelog-Fixed: lightningd: avoid crash on signing failure when trying to spend anchor outputs.
2024-05-07 16:41:47 -05:00
Rusty Russell
a481dbcddd pytest: attempt to reproduce hsm signing failure.
This didn't trigger the bug, but worth explicitly testing: we spend a
to-remote output from a previous unilateral close, to spend an anchor
on a current unilateral close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-07 16:41:47 -05:00
ShahanaFarooqui
e38c828bf0 docker: Fix jq installation error for focal
`jq` is not available in the default repositories of the Ubuntu Focal image. To resolve this, adding the official jq package repository and then installing it.

Changelog-None.
2024-05-06 22:13:54 -05:00
Rusty Russell
153567d699 pytest: test that the preapprove hooks get called by check.
We add a dev flag so we can decline them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-06 20:51:19 -05:00
Rusty Russell
68c4b83ab1 pay: move preapproveinvoice command out so it can be called bu check.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `check` `keysend` now checks with HSM that it will approve it.
2024-05-06 20:51:19 -05:00
Rusty Russell
e1034eec43 keysend: move preapprovekeysend outside pay loop, so it can be called by check.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `check` `keysend` now checks with HSM that it will approve it.
2024-05-06 20:51:19 -05:00
Rusty Russell
c7339ea310 pytest: test various preapprove scenarios.
We wire through --dev options into the hsmd, and test preapprove accept and deby
with both old and new protocols.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-06 20:51:19 -05:00
Rusty Russell
bd6cf99e4d lightningd: extend preapproveinvoice and preapprovekeysend to check with HSM.
If they support it, we can actually ask hsmd when we are called in
check mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-06 20:51:19 -05:00
Rusty Russell
6ea95da342 hsmd: add variant of preapprove commands to have it check only, not do anything.
Apparently VLS actually does something when we preapprove: if caller is just
checking we want to tell it not to do that!

I put in a flag so we can test both old and new APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-06 20:51:19 -05:00