Commit Graph

117 Commits

Author SHA1 Message Date
Christian Decker
de75d3ac0c mpp: Add CLI option to opt-out of multi-part payments
Several tests are not well-suited for mpp, so I added a CLI option to opt-out
of the MPP support at startup time.
2020-07-15 11:32:58 +02:00
Christian Decker
a287bbe55d mpp: Enable adaptive splitter
Changelog-Added: The adaptive multi-part payment modifier will split payments that are failing due to their size into smaller parts, and re-attempted.
2020-07-15 11:32:58 +02:00
Christian Decker
d0eb3a79eb paymod: Not having a result doesn't mean we failed at getroute
Specifically if we split, there is no result, but we shouldn't add a failure
message.
2020-07-15 11:32:58 +02:00
Christian Decker
b813974e13 mpp: Add the presplit MPP modifier
Changelog-Added: The MPP presplit modifier splits large payments into 10k satoshi parts to maximize chances of performing the payment and to obfuscate the overall amount being sent.
2020-07-15 11:32:58 +02:00
Christian Decker
5776a33116 paymod: Activate paymod and move legacy pay to legacypay command
As suggested during the paymod-03 review it is better to activate the new code
right away, and give users an escape hatch to use the legacy code instead. The
way I implemented it allows using either `legacypay` or `pay` and then set
`legacy` to switch to the other implementation.

Changelog-Added: JSON-RPC: The `pay` command now uses the new payment flow, the new `legacypay` command can be used to issue payment with the legacy code if required.

Suggested-by: Rusty Russell <@rustyrussell>
Suggested-by: ZmnSCPxj <@zmnscpxj>
2020-07-14 06:25:29 +09:30
Christian Decker
0aa5c197ec pytest: Fix tests broken by the pay and paystatus changes
This commit collects the changes required to the tests caused by the changes
to the `pay` and `paystatus` commands. They are also rather good hints as to
what these changes entail.
2020-07-07 23:25:45 +02:00
Christian Decker
2ac87c1490 paymod: Reorder paymod arguments to match pay
We've been adding modifiers and arguments out of order, and we need the
arguments order to match up if we want `paymod` to be a drop-in replacement.
2020-07-07 23:25:45 +02:00
Christian Decker
da8eb6fb4c paymod: Expose riskfactor and wire through to getroute 2020-07-07 23:25:45 +02:00
Christian Decker
5601a1ab21 paymod: Add a deadline to the pay command and retry modifier 2020-07-07 23:25:45 +02:00
Christian Decker
f557955515 paymod: Consolidate parsing RPC results in libplugin
We handle these in a number of different ways, and regularly get the parsing
and logic for optional fields wrong, so let's consolidate them here.
2020-07-07 23:25:45 +02:00
Christian Decker
cb00cbac7c paymod: Add support for the msatoshi override argument 2020-07-07 23:25:45 +02:00
Christian Decker
2331cd62e1 paymod: Add user-provided label back into the paystatus result 2020-07-07 23:25:45 +02:00
Christian Decker
215a0ada8b paymod: Update the route constraints as we apply routes and mods
These are primarily the fee and cltv constraints that we need to keep up to
date in order to give modifiers a correct view of what is and what isn't
allowed.
2020-07-07 23:25:45 +02:00
Christian Decker
935578e567 paymod: Exclude most expensive/slowest chan if limits are exceeded 2020-07-07 23:25:45 +02:00
Christian Decker
4648d2867f paymod: Compute fee and CLTV delta limits for a payment
So far we got away with not caring about these but since we're implementing
modifiers that impact these limits, we better keep track of them.
2020-07-07 23:25:45 +02:00
Christian Decker
e71bdf9ed8 paymod: Wrap the getroute request in a struct
Keeping the arguments in their non-serialized form allows us to amend them as
we apply modifiers and learn new information.
2020-07-07 23:25:45 +02:00
Christian Decker
1b057f025c paymod: Let the payment know the local ID
This is necessary for the routehint modifier.
2020-07-07 23:25:45 +02:00
Christian Decker
097cb8d3a0 paymod: Exclude nodes that we found to be faulty 2020-07-07 23:25:45 +02:00
Christian Decker
2f0e535b81 paymod: Add reason why a payment was attempted
This is a slight change in interface in that the string no longer spells out
that a specific node was excluded.
2020-07-07 23:25:45 +02:00
Christian Decker
3a35dd34ac paymod: Add the contents of paystatus
This proved to be rather difficult given the tight coupling of the old structs
and the output of the command.
2020-07-07 23:25:45 +02:00
Christian Decker
2e88249a7b paymod: Make the transition to FAILED a function call
We need to set some variables as well, and the transition to FAILED is a bit
special in that quite some modifiers will skip into FAILED.
2020-07-07 23:25:45 +02:00
Christian Decker
ddb0424a4d paymod: Remove the dummy payment modifier
It was there only for demonstration purposes, and is no longer useful.
2020-07-02 11:54:12 +02:00
Christian Decker
2acb86bfe3 paymod: Consider new payments when checking if an attempt is ongoing
Without this the flapping behavior tested in `test_pay_retry` would reappear.
2020-07-02 11:54:12 +02:00
Christian Decker
fbd8cf1495 paymod: Keep a copy of the serialized bolt11 in the root payment
We frequently query by the bolt11 string, so keeping it around saves us from
having to parse the query or serialize the parsed one.
2020-07-02 11:54:12 +02:00
Christian Decker
9d7a48d0f4 pay: Key the MPP payments based on the payment_hash not the bolt11
This allows us to drive the payment from outside, and still get the
aggregation that we want.
2020-07-02 11:54:12 +02:00
Christian Decker
5e3134083e paymod: Add a local_channel_hints modifier to collect local channels
We can have quite detailed information about our local channels, so call
`listpeers` before the `getroute` call on the root payment, to seed that
information in the channel_hints.
2020-07-02 11:54:12 +02:00
Christian Decker
b1e9f4923b paymod: Add the final cltv delta to the getroute call 2020-07-02 11:54:12 +02:00
Christian Decker
6fbfe24e3e paymod: Add a COMPAT_090 flag
Since we end up consolidating some of the return values for `pay` and
`paystatus` and change the public interface we need to add the compatibility
flag and guard the switchover behind it.
2020-07-02 11:54:12 +02:00
Christian Decker
b15876c112 paymod: Implement TLV onion payload generation 2020-07-01 12:19:02 +02:00
Christian Decker
c35df400b2 paymod: Maintain a list of current and past payments
We need to keep them around so we can inspect them later. We'll also need a
background cleanup every once in a while to free some memory. More on that in
a future commit.
2020-07-01 12:19:02 +02:00
Christian Decker
5c32e33aab paymod: Add a retry modifier that retries payments on failure
This is likely a bit of overkill for this type of functionality, but it is a
nice first use-case of how functionality can be compartmentalized into
modifiers. If makes swapping retry mechanisms in and out really simple.
2020-07-01 12:19:02 +02:00
Christian Decker
23b4dca3c7 paymod: Add a simple test-command to test the paymod state-machine
This commit can be reverted/skipped once we have implemented all the logic and
have feature parity with the normal `pay`. It's main purpose is to expose the
unfinished functionality to test it, without completely breaking the existing
`pay` command.
2020-07-01 12:19:02 +02:00
Rusty Russell
cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Antoine Poinsot
4bb7b4621f pay: respect maxfeepercent when choosing a shadow route
And the percentage of the initial amount, not the constently increasing
one !

Changelog-Fixed: pay: we now respect maxfeepercent, even for tiny amounts.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-05 14:10:42 +09:30
Christian Decker
e03acd9663 libplugin: Add features to plugin_main and getmanifest 2020-04-16 18:03:35 +09:30
ZmnSCPxj jxPCSnmZ
919d371fe8 common/json_stream.c: Implement a json_add_jsonstr to add already-JSON strings to json_stream objects.
Also incidentally fixes #3613

ChangeLog-none
2020-04-09 14:09:23 +09:30
lisa neigut
b51772ff68 pay: fix #3613; string output must be quoted
`json_add_member` requires quotes for string types

Changelog-Fixed: `pay` would crash on expired waits with tried routes
2020-04-08 15:31:52 +02:00
lisa neigut
3124d43d9f pay: check for null route in maybe_exclude
Fixes (?) #3607

Changelog-Fixed: Fix crash in pay when attempting to find cheaper route with exemptfee
2020-04-08 15:31:52 +02:00
Christian Decker
891adfca1e pay: Speed up pay and fix the shadow route construction
The shadow route algorithm is extending the route randomly using channels
adjacent to the current destination, in the hope to create a plausible route
extension. However, instead of only retrieving the channels adjacent to the
destination it was retrieving all channels in the entire topology, and
selecting a random channel from there. This resulted in a very large request
for all channels being processed, and then mostly not being used, but also in
shadow extensions to the path which were not plausible (they didn't extend the
real path, just random edges). This is fixed by restricting the call to
`listchannels` to the channels with the current destination as source.

On my laptop retrieving all channels in the current mainnet takes
approximately 1.2 seconds, and given the geometric series expansion of the 50%
extension probability this indeed would result in an overhead of 1.2 seconds
to the `pay` command. In contrast specifying a source results in an overhead
of ~30ms.

So good news everyone, your pay commands just shaved 1.17 seconds off their
runtime.

Changelog-Changed: pay: Improved the performance of the `pay`-plugin by limiting the `listchannels` when computing the shadow route.
Changelog-Fixed: pay: The `pay`-plugin was generating non-contiguous shadow routes
2020-04-04 16:24:57 +10:30
Rusty Russell
ca512e3cb2 plugins/pay: use feature set from lightningd to decode bolt11.
This means we correctly reject invoices with features we don't understand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
Rusty Russell
cf43e44378 common/features: don't use internal global.
Turns out that unnecessary: all callers can access the feature_set,
so make it much more like a normal primitive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
Rusty Russell
2aad3ffcf8 common: tal_dup_talarr() helper.
This is a common thing to do, so create a macro.

Unfortunately, it still needs the type arg, because the paramter may
be const, and the return cannot be, and C doesn't have a general
"(-const)" cast.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-27 14:16:16 +10:30
Vasil Dimov
89ceb273f5 wire: remove towire_double()
Before this patch we used to send `double`s over the wire by just
copying them. This is not portable because the internal represenation
of a `double` is implementation specific.

Instead of this, multiply any floating-point numbers that come from
the outside (e.g. JSONs) by 1 million and round them to integers when
handling them.

* Introduce a new param_millionths() that expects a floating-point
  number and returns it multipled by 1000000 as an integer.

* Replace param_double() and param_percent() with param_millionths()

* Previously the riskfactor would be allowed to be negative, which must
  have been unintentional. This patch changes that to require a
  non-negative number.

Changelog-None
2020-02-27 09:07:04 +10:30
Rusty Russell
5e2053feea pay: fix crash when we paystatus too soon.
If the attempt hasn't started yet, we can have an empty 'attempts' array
(while it's in listpeers, for example):

```
pay: plugins/pay.c:1457: json_paystatus: Assertion `tal_count(ps->attempts)' failed.
pay: FATAL SIGNAL 6 (version v0.8.1rc1-1-g946af3f)
0x55e895110543 send_backtrace
	common/daemon.c:39
0x55e8951105e9 crashdump
	common/daemon.c:52
0x7f92b0928f1f ???
	???:0
0x7f92b0928e97 ???
	???:0
0x7f92b092a800 ???
	???:0
0x7f92b091a399 ???
	???:0
0x7f92b091a411 ???
	???:0
0x55e895100012 json_paystatus
	plugins/pay.c:1457
0x55e895103aaa ld_command_handle
	plugins/libplugin.c:968
0x55e895103bd4 ld_read_json_one
	plugins/libplugin.c:1011
0x55e895103cd2 ld_read_json
	plugins/libplugin.c:1030
0x55e895124cbd next_plan
	ccan/ccan/io/io.c:59
0x55e89512583a do_plan
	ccan/ccan/io/io.c:407
0x55e895125878 io_ready
	ccan/ccan/io/io.c:417
0x55e895127a3e io_loop
	ccan/ccan/io/poll.c:445
0x55e895104593 plugin_main
	plugins/libplugin.c:1194
0x55e895100cbc main
	plugins/pay.c:1697
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-13 14:20:50 +10:30
darosior
b91433cb42 libplugin: use json_stream for all plugins' commands 2020-02-10 09:49:15 +10:30
darosior
2bff80e3de libplugin: use json_stream helpers for RPC calls
This adds helpers to start and send a jsonrpc request using json_stream
in order to benefit from the helpers.

This then simplifies existing plugins RPC requests by using json_stream
helpers.
2020-02-10 09:49:15 +10:30
darosior
b31e3b1541 libplugin: pass a pointer to plugin to send_outreq
autoclean needs to send outreqs from a timer cb, hence with cmd == NULL.
2020-02-04 13:24:32 +10:30
darosior
765626875e libplugin: use ccan/io for notifications 2020-02-04 13:24:32 +10:30
darosior
499dce1c38 libplugin: include the rpc conn into the global state
And rename the struct name, as it's now only used for RPC.
2020-02-04 13:24:32 +10:30
Vasil Dimov
55173a56b7 Use dedicated type for error codes
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.

To resolve this:

* Introduce an error code type with a known size:
  `typedef s32 errcode_t`.

* Change all error code macros to constants of type `errcode_t`.
  Constants also play better with gdb - it would visualize the name of
  the constant instead of the numeric value.

* Change all functions that take error codes to take the new type
  `errcode_t` instead of `int`.

* Introduce towire / fromwire functions to send / receive the newly added
  type `errcode_t` and use it instead of `towire_int()`.

In addition:

* Remove the now unneeded `towire_int()`.

* Replace a hardcoded error code `-2` with a new constant
  `INVOICE_EXPIRED_DURING_WAIT` (903).

Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
2020-01-31 06:02:47 +00:00