Commit Graph

1198 Commits

Author SHA1 Message Date
Rusty Russell
40d93c2af7 renepay: remove always-true "first_time" and "unlikely_ok" flags.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
5e5f592369 renepay: remove unused result member.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
e8cfb40709 renepay: put the entire hash in the key struct.
As recommended by your TODO, a bit simpler: we also make the hash function
return a ptr rather than the (now rather large) struct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
135180c1a0 renepay: don't re-parse bolt11 to get routehints.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
2ceac6c11d renepay: simplify JSON handling in notification_sendpay_success.
Use json_scan(), and use the new pay_flow_from_notification() routine.

Also, the tal_dup_or_null can be tal_dup, since &preimage is never NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
72b316924d renepay: simplify JSON handling.
We can use json_scan(), and share a routine to map the notification
to the pay_flow.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
fb50474efd renepay: get max group_id in single iteration.
Simplifies the logic since we bail out if there are two different
group ids in progress.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
3b8217bd72 renepay: merge struct renepay and struct payment into one.
There are a few fields in `struct renepay` which are genuinely
transient, but it makes the code much harder to follow than simply
having a single structure.

More cleanups will follow, but this is the minimal set.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
6d7cd1e729 renepay: make memleak simpler.
Simply tell it to scan the entire object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
0091dc4e6d renepay: move list_node to first member of struct payment.
Results in payments having a pointer to the start of the object,
which helps our memleak code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
008bd74c93 renepay: use more formal allocator pattern.
The general pattern for xxx_new is that it should populate all
fields, for encapsulation and so you never can have a half-formed
object.

This means a fair bit of work for now, but it pays off in the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
ca72a230f3 renepay: fix localmods.
You cannot refresh the gossmap with localmods applied, nor apply localmods
when others have applied localmods in the same process.

There are optimizations we could do, but for now always apply/unapply before
querying gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
0a9b3d6740 renepay: remove attempt limit.
Time is what users care about, so remove this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
b5273610b2 renepay: remove unused all_flows field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
5f8b77480c plugins: fix error report from bitcoin-cli exec failure.
We've stomped errno, so if exec fails we don't get a reliable result:

```
2023-08-07T17:58:45.713Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-09 19:53:47 +09:30
junderw
32b88a2340 Fix: Remove Sync requirements on Futures returned in the Rust plugin library.
See: https://github.com/bitcoindevkit/bdk/issues/1047#issuecomment-1660645669

In general, futures produced by most libraries in the ecosystem of Rust, and bounds placed
on users of famous runtimes like tokio and its spawn method all lack Sync requirements.

Because of this, anyone who creates a callback using any sort of library that returns a
non-Sync future (which most libraries fit this description) inside of it will get some
cryptic error messages (async error messages still leave a lot to be desired).

Removing these Sync requirements will make the library more useful.
2023-08-08 09:45:42 +09:30
Rusty Russell
54bcb10227 lightningd: fix bolt11 parsing in preapproveinvocie, sendonion, listsendpays and renepay
Since bolt11_decode now insists that any `lightning:` prefix be removed, we need
to make sure to use param_invstring not param_string for all bolt11 parameters:

```
2023-08-07T05:55:32.515Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v23.08rc1-21-g0bf5ee6)
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x55dd94934154
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x55dd949342e6
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 ((null)) 0x7f5cf5a3bcef
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:44 (__pthread_kill_implementation) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:78 (__pthread_kill_internal) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:89 (__GI___pthread_kill) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ../sysdeps/posix/raise.c:26 (__GI_raise) 0x7f5cf5a3bc45
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7f5cf5a227fb
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:92 (__assert_fail_base) 0x7f5cf5a2271a
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:101 (__GI___assert_fail) 0x7f5cf5a33595
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:734 (bolt11_decode_nosig) 0x55dd94929967
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:953 (bolt11_decode) 0x55dd9492a44f
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/pay.c:1730 (json_listsendpays) 0x55dd948d7d72
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:658 (command_exec) 0x55dd948b525b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:786 (rpc_command_hook_final) 0x55dd948b5876
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:285 (plugin_hook_call_) 0x55dd948f6446
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:874 (plugin_hook_call_rpc_command) 0x55dd948b5c77
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:984 (parse_request) 0x55dd948b6234
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:1090 (read_json) 0x55dd948b670f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55dd94ac9bf4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x55dd94aca823
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x55dd94aca865
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x55dd94accbff
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x55dd948b33c4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1332 (main) 0x55dd948ba429
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main) 0x7f5cf5a2350f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../csu/libc-start.c:381 (__libc_start_main_impl) 0x7f5cf5a235c8
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x55dd94881e74
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Fixes: #6524
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: broken in master since last release.
2023-08-07 18:46:48 +09:30
Rusty Russell
2042b50978 plugins/bcli: update minimum required bitcoind version.
Less than 22 is obsolete anyway, so we should increment this from 16.0 at least!

Closes: #6234
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 16:04:14 +09:30
Rusty Russell
84fe3dc940 build: more 32-bit printf fixes.
Reported-by: Shahana
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 11:52:40 +09:30
ShahanaFarooqui
18b928046b plugin/clnrest: Downgrading pyln-proto versino from 23.08 to 23.5.2
Bug fix: pip install from plugins/clnrest/requirements.txt was unable to find "not yet released" pyln-proto version 23.08.
2023-08-06 12:34:19 +09:30
Dusty Daemon
7c9f5e9847 renepay: Use macros instead of manual format types
Fixes 32-bit builds:

```
In file included from plugins/renepay/pay.c:5:
./plugins/renepay/pay_flow.h: In function 'fmt_payflow_key':
./plugins/renepay/pay_flow.h:54:17: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'u64' {aka 'long long unsigned int'} [-Werror=format=]
   54 |                 "key: groupid=%ld, partid=%ld, payment_hash=%s",
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   55 |                 k->groupid,k->partid,
      |                 ~~~~~~~~~~
      |                  |
      |                  u64 {aka long long unsigned int}
```

etc
2023-08-04 20:33:20 +09:30
Rusty Russell
67e43ea868 renepay: clean up ARC_ADDITIONAL_BITS.
It's expressed in bits, but really it's clearer as a quantity, given
how it's used.

Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:52 +09:30
Rusty Russell
15c8a6f6fe renepay: switch from arc_t to struct arc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:52 +09:30
Rusty Russell
b793dc9224 renepay: implement an alternative to a union with bitsets.
This was actually relying on little-endian; if we want this, we need to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:52 +09:30
Shahana Farooqui
c96d0c2643 plugin/clnrest: Updated pyln version to 23.08 in requirements.txt 2023-08-01 11:35:19 +09:30
Dusty Daemon
4628e3ace8 channeld: Code to implement splicing
Update the lightningd <-> channeld interface with lots of new commands to needed to facilitate spicing.

Implement the channeld splicing protocol leveraging the interactivetx protocol.

Implement lightningd’s channel_control to support channeld in its splicing efforts.

Changelog-Added: Added the features to enable splicing & resizing of active channels.
2023-07-31 21:00:22 +09:30
Dusty Daemon
50fe819f47 splicing: Add channel state AWAITING_SPLICE
Update gossip routiens and various other hecks on the channel state to consider AWAITING_SPLICE to be routable and treated similar to CHANNELD_NORMAL.

Small updates to psbt interface

Changelog-None
2023-07-31 21:00:22 +09:30
Lagrang3
3024afe524 renepay: more cleanups
- adopt "const <type> *"convention
- remove use_shadow option for some pyln tests
- show prob. information of flows into paynotes
- show prob. of success of entire payment flow in paynotes
- minflow: We were not releasing the memory of flow arrays when replacing
  them with a new canditate.
- use memleak_scan_obj in memleak_check
- replace u64 with size_t

Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-07-31 12:58:56 +09:30
Rusty Russell
b5da85e85d plugins/renepay/dijkstra: improve API to remove global.
The global is an *internal* hack because dijkstra_item_mover doesn't
take a context arg!  It should be used with care.

Easy, since all the accessors exist: we just hand in the struct dijkstra.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-31 12:58:56 +09:30
Lagrang3
79486c1e3b renepay: fixups after comments
- remove internal gheap checks
- add check for arc_t.chanidx overflow
- remove outdated comments
- check the delta flow bounds before augmenting along a path
- get_flow_paths uses a dynamic tal array instead of a list.
- fix a unit test that depended on the order of returned flows
- fix bug: lightnind doesn't like if I reuse the partid of a failed
  flow, therefore use a higher partid than any of the previous attempts.
- plugin_err instead of LOG_BROKEN if sendpay fails and we cannot get a
  an error code.
- fix wrong comments.
- remove the background timer.
- This is a bugfix. Previous to this the MCF network was built using the
knowledge of the min and max liquidity but it didn't take into account
pending HTLCs.
- Also remove the min_prob_success option but hardcode a 90% value.

Removing some options that are not relevant to the user, they're kept
for developer mode only:
- base_fee_penalty
- min_prob_success
- prob_cost_factor
- remove heap.h, not used

Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-07-31 12:58:56 +09:30
Lagrang3
d46990d301 renepay: add information decay linear in time
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-07-31 12:58:56 +09:30
Rusty Russell
b8ca193606 renepay: an experimental payment plugin
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
Changelog-Added: Plugins: `renepay`: an experimental pay plugin implementing Pickhardt payments (`renepay` and `renepaystatus`).
2023-07-31 12:58:56 +09:30
Shahana Farooqui
637f314bc6 plugin/clnrest: Improved readme and comments
- Adding runes info in readme
- Added extra comments in the code
- System exit on shutdown
- Replaced single quotes with double quotes

Changelog-None
2023-07-30 16:50:04 +09:30
Shahana Farooqui
3fab4cc4f3 plugin/clnrest: Updating package
- Adding package initializer for future proofing
- Updated Pyln version
2023-07-30 16:50:04 +09:30
niftynei
73a67fbb3e funder: don't re-reserve utxos on retries
This way unreserving the PSBT will work as intended, and we don't have
to keep track of how many times we've called reserved for any one input.

Technically we're supposed to not reserve inputs at *all* while doing
opens, this moves us slightly closer to that.
2023-07-30 15:20:04 +09:30
Christian Decker
5f2072869a pay: Fix a null-dereference introduced in #6428
The alias may not be set for non-alias channels after they
confirm. The other branch is safe because we only consider active
channels.

Changelog-None
Fixes #6450
2023-07-28 10:06:43 +09:30
Rusty Russell
b2c1ef73cb build: avoid printing PLUGIN_BASES on every build.
Debugging left over in 94205c094f.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-28 09:44:58 +09:30
Rusty Russell
0a727ae6c5 plugins/bcli: plug temporary leak on retry.
Caught by leak detection, we just re-assigned this when we retried: sure,
it's temporary, but it's technically a leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-26 06:58:38 +09:30
Rusty Russell
7ddebada90 plugins/bcli: fix leak report when bitcoind goes away.
I shut down bitcoind during a test, and bcli leak reports flooded in.
They're all temporary, but this fixes them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-26 06:58:38 +09:30
Rusty Russell
620135335d libplugin/pay: allow shortcut for self-pay.
This is the simplest solution, not the best, but there's significant risk in try to remove the "we have a path" assumption in the code pay code.

Includes removing a `tal_steal` which was incorrect: the buffer has the same lifetime as the plugin, so if we steal it then things get messy when we free the  struct payment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `pay` will now pay your own invoices if you try.
2023-07-26 06:55:48 +09:30
Rusty Russell
981d82c406 common/utils: add tal_strdup_or_null helper.
It's not that uncommon to want to pass NULL through, for optional strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-26 06:55:48 +09:30
Rusty Russell
b6d347a6d3 db: add optional column string helper, and make db_col amounts return sat/msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-26 06:55:48 +09:30
Shahana Farooqui
911d2c117e plugins/clnrest: Websocket Server
- Added Dependencies
- Updated WS client instructions
- WS Server
2023-07-25 19:20:16 +09:30
Shahana Farooqui
94205c094f plugins/clnrest: Update clnrest dependencies in scripts and Dockerfiles
Adding python3-json5 python3-flask python3-gunicorn flask_restx pyln-client
Also run `pip3 install -r plugins/clnrest/requirements.txt`
2023-07-25 19:20:16 +09:30
Shahana Farooqui
5e3ffc468b plugins/clnrest: removing notifications queue in favor of websocket server 2023-07-25 19:20:16 +09:30
Rusty Russell
2ab0b4af3e plugins/clnrest: disable ourselves unless rest-port is set.
This mirrors grpc's behavior, and avoids listening on ports without
explicit user permission.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-25 19:20:16 +09:30
Shahana Farooqui
abcfda133f plugin: On-boarding default python plugins
plugin: On-boarding default python plugins
2023-07-25 19:20:16 +09:30
Rusty Russell
a6701bf180 plugins/clnrest: fail gracefully if dependencies not installed.
json and sys are Python built-in packages, so we can do this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-25 19:20:16 +09:30
Shahana Farooqui
21160aa6a7 plugin: clnrest
plugin: clnrest
2023-07-25 19:20:16 +09:30
Rusty Russell
a219c99691 commando: pass through commands to lightningd.
The in-core commands are much more helpful with details on why runes failed, too!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-25 14:47:12 +09:30