Commit Graph

206 Commits

Author SHA1 Message Date
Rusty Russell
39e4796ae3 json_command: command_fail_badparam helper.
It's common to want to complain a token is not what we expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 09:46:37 +09:30
Rusty Russell
74e960c20b pytest: increase channel sizes so they reach min capacity with option_anchor_outputs
With a feerate of 7500perkw and subtracting 660 sats for anchors, a
20,000 sat channel has capacity about 9800 sat, below our default:

	You gave bad parameters: channel capacity with funding 20000sat, reserves 546sat/546sat, max_htlc_value_in_flight_msat is 18446744073709551615msat, channel capacity is 9818sat, which is below 10000000msat

So bump channel amounts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-01 12:27:39 -05:00
Christian Decker
94d4d20d5f pytest: Stabilize test_mpp_interference_2 2020-08-30 20:03:42 +02:00
Christian Decker
e420bfd6a8 pytest: Skip test_lockup_drain if VALGRIND and not DEVELOPER
The test was failing very reliably, by not raising the expected exception. My
best guess is that it needs more synchronization to cause the issue.
2020-08-30 20:03:42 +02:00
Rusty Russell
4a9c162211 pytest: disable test_mpp_interference_2.
We're failing this too often: we'd fail it more but it's disabled
with VALGRIND (it shouldn't be: @slow_test removes VALGRIND if SLOW_MACHINE
is set).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-27 13:51:22 +02:00
Christian Decker
21d87f7075 pay: Implement simple presplit fix for ludicrous amounts
This is the simplest possible fix: increase the target amount until we get
the desired number of parts, while still bucketizing payments together that
are in approximately the same size.

The current logic puts all payments that are in the range x < amount <= 16*x
in the same bucket, making them harder to distinguish.

Changelog-Fixed: pay: The `presplit` modifier now supports large payments without exhausting the available HTLCs.
2020-08-27 10:19:21 +09:30
Christian Decker
3f6809c934 pytest: Reproduce ludicrous presplit issue
Huge payment + presplit modifier = too many HTLCs...
2020-08-27 10:19:21 +09:30
Christian Decker
8f028272f4 pytest: Stabilize test_pay_get_error_with_update 2020-08-27 09:45:35 +09:30
ZmnSCPxj jxPCSnmZ
ba3f38077e lightningd/invoice.c: Use round-robin channel selection.
Changelog-Changed: We now make MPP-aware routehints in invoices.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ
f50951a0d6 tests/test_pay.py: Motivating scenario for round-robin routehinting. 2020-08-26 09:29:46 +09:30
Rusty Russell
cbd0be5c0e EXPERIMENTAL_FEATURES: enable option_anchor_outputs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
8cf2fff91b pytest: generalize tests.
Anchor outputs break many assumptions in our tests:

1. Remove some hardcoded numbers in favor of a fee calc, so we only have to
   change in one place.
   FIXME: This should also be done for elements!
2. Do binary search to get feerate for a given closing fee.
3. Don't assume output #0: anchor outputs perturb them.
4. Don't assume we can make 1ksat channels (anchors cost 660 sats!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Christian Decker
8769f9ed93 pay: Fix final TLV payload if not going through MPP modifiers
Reported-by: ZmnSCPxj
Signed-off-by: Christian Decker <@cdecker>

Changelog-Fixed: pay: Correct a case where we put the sub-payment value instead of the *total* value in the `total_msat` field of a multi-part payment.
2020-08-13 12:50:16 +02:00
ZmnSCPxj jxPCSnmZ
29df517966 tests/test_pay.py: Provide test showing that blockheight disagreement causes us to advance routehints too aggressively.
The worst effect is that unpublished nodes are harder to pay, but
even published ones make us do unnecessary work, since we are
losing routehints from the published ones that could help us
actually route better to them.
2020-08-13 12:50:16 +02:00
Christian Decker
0dcd974d97 pytest: Reproduce #3915 2020-08-12 19:10:48 +02:00
Vincenzo Palazzo
1521c29fcf listpays mod 1: add destination inside the response when bolt11 is null
Changelog-Added: JSON-RPC: `listpays` now lists the `destination` if it was provided (e.g., via the `pay` plugin or `keysend` plugin)
2020-08-09 16:03:03 +02:00
Vincent
700897f06e listpays: fixed bolt11 null with keysend and update doc command
listpays: make doc-all missed
Changelog-Added: JSON-RPC: `listpays` can be used to query payments using the `payment_hash`
Changelog-Added: JSON-RPC: `listpays` now includes the `payment_hash`
2020-08-09 16:03:03 +02:00
Christian Decker
723b7223b7 pay: Add timestamp of first part to listpays
Changelog-Added: JSON-RPC: The result returned by `listpays` now includes the timestamp of the first part of the payment
2020-08-08 12:53:12 +02:00
Rusty Russell
01a82d38f7 pytest: add slow_test marker.
And when it's set, and we're SLOW_MACHINE, simply disable valgrind.

Since Travis (SLOW_MACHINE=1) only does VALGRIND=1 DEVELOPER=1 tests,
and VALGRIND=0 DEVELOPER=0 tests, it was missing tests which needed
DEVELOPER and !VALGRIND.

Instead, this demotes them to non-valgrind tests for SLOW_MACHINEs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell
fde353ab00 pytest: use get_nodes more widely.
I started replacing all get_node() calls, but got bored, so then just did the
tests which call get_node() 3 times or more.

Ends up not making a measurable speed difference, but it does make some
things neater and more standard.

Times with SLOW_MACHINE=1 (given that's how Travis tests):

Time before (non-valgrind):
	393 sec (had 3 failures?)
Time after (non-valgrind):
	410 sec

Time before (valgrind):
	890 seconds (had 2 failures)
Time after (valgrind):
	892 sec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
ZmnSCPxj jxPCSnmZ
4fde45669c plugins/pay.c: Fix the routehints/presplit conflict.
Changelog-Fixed: pay: Fixed a bug where routehints would be ignored if the payment exceeded 10,000 satoshi. This is particularly bad if the payee is only reachable via routehints in an invoice.
2020-08-06 17:36:58 +02:00
ZmnSCPxj jxPCSnmZ
a9b992a94a tests/test_pay.py: Add test for conflict between presplit and routehints paymods. 2020-08-06 17:36:58 +02:00
Rusty Russell
4517435810 pytest: fix flake in tests/test_pay.py::test_pay_exclude_node
```
        # Excludes channel, then ignores routehint which includes that, then
        # it excludes other channel.
>       assert len(status) == 2
E       assert 1 == 2
E         -1
E         +2
```

The invoice we use at the end has a routehint: 50% of the time it's
to l2 (which fails), 50% to l5 (which succeeds).

Change it to create invoice before channel with l5 so it does the
retry like we expect here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-29 11:42:09 +02:00
Christian Decker
311594b2cd jsonrpc: Add amount received by recipient to listpays result
We sum up the amounts just like we do with `amount_sent`, however we may have
parts that don't have that annotation, so make it optional.

Suggested-by: Rusty Russell <@rustyrussell>
2020-07-28 16:17:39 +09:30
Christian Decker
d12ea51e63 pytest: Replicate the missing amount_msat in listpays
Since we started using `sendonion` in the `pay` plugin we no longer
automatically have the `amount` annotation on (partial) payments. This
replicates the issue so we can fix it.

Reported-by: Rusty Russell <@rustyrussell>
2020-07-28 16:17:39 +09:30
Vincent
81fd552e84 plugins/pay: hand bolt11 arg to sendonion if we have one (i.e. for pay)
[ Extracted into standalone patch and comment added by RR ]
2020-07-27 13:11:14 +02:00
Vincent
f4f2853ded pytest: test to show that listpays doesn't remember bolt11 string.
[Separated into separate commit by RR]
2020-07-27 13:11:14 +02:00
Christian Decker
67e846fdbb pytest: Adjust test now that routehint order changed 2020-07-24 11:35:49 +02:00
Christian Decker
899a2e64b0 paymod: Randomly select a routehint, or none at random
The adaptive MPP test was showing an issue with always using a routehint, even
when it wasn't necessary: we would insist on routhing to the entrypoint of the
routehint, even through the actual destination. If a channel on that loop
would result being over capacity we'd slam below 0, and then increase again by
unapplying the route. The solution really is not to insist on routing through
a routehint, so we implement random skipping of routehints, and we rotate them
if we have multiples.
2020-07-23 10:14:21 +09:30
Vincent
54888d454b Fixed assertion in pay plugin
This PR includes the fix discussed on PR #3855. This fix was tested with the use case described inside the issue and worked.

Fixes: #3855

Changelog-None
2020-07-23 10:14:21 +09:30
ZmnSCPxj jxPCSnmZ
9b4fd11e98 tests/test_pay.py: Replicate #3855. 2020-07-23 10:14:21 +09:30
Rusty Russell
6951a9ea42 pytest: fix erroneous test_pay_retry result.
Seems like we get *4* failures, since failing to find a route counts now?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-23 10:14:21 +09:30
Rusty Russell
09eb7110e0 sendpay: insist that partid be an exact duplicate if in progress.
The test had part 1 and 2 backward, but still worked.  When I copied that to
*after* the test had succeeded, it complained.  It should always complain,
to catch bugs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-21 13:30:15 +02:00
Rusty Russell
73d5d96d2a sendpay: don't allow a new part payment if any part has succeeded.
This wasn't important before, but now we have MPP it's good to enforce.

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-21 13:30:15 +02:00
Christian Decker
de096eeed9 pytest: Disable test_pay_routeboost for valgrind due to timeout 2020-07-15 11:32:58 +02:00
Christian Decker
de90606490 pytest: Add an adaptive MPP test
This exercises something that is simply not possible without MPP, i.e., the
bundling of multiple paths to get sufficient capacity to perform the payment.
2020-07-15 11:32:58 +02:00
Christian Decker
dad23066dc pytest: Disable MPP sending for 4 tests
These mostly deal with exact HTLC counts, and fixed number of attempts to
conclusion, so the randomization that MPP adds is not desirable.
2020-07-15 11:32:58 +02:00
Christian Decker
b813974e13 mpp: Add the presplit MPP modifier
Changelog-Added: The MPP presplit modifier splits large payments into 10k satoshi parts to maximize chances of performing the payment and to obfuscate the overall amount being sent.
2020-07-15 11:32:58 +02:00
Christian Decker
02e44e9903 paymod: Set the STOPPED_RETRYING status code if we stopped retrying 2020-07-14 06:25:29 +09:30
Christian Decker
5776a33116 paymod: Activate paymod and move legacy pay to legacypay command
As suggested during the paymod-03 review it is better to activate the new code
right away, and give users an escape hatch to use the legacy code instead. The
way I implemented it allows using either `legacypay` or `pay` and then set
`legacy` to switch to the other implementation.

Changelog-Added: JSON-RPC: The `pay` command now uses the new payment flow, the new `legacypay` command can be used to issue payment with the legacy code if required.

Suggested-by: Rusty Russell <@rustyrussell>
Suggested-by: ZmnSCPxj <@zmnscpxj>
2020-07-14 06:25:29 +09:30
Christian Decker
01a475161d paymod: Implement keysend sending support as a native RPC command
This makes use of the payment modifier structure to just add the preimage to
the TLV payload for the last hop.

Changelog-Added: JSON-RPC: The `keysend` command allows sending to a node without requiring an invoice first.
2020-07-13 14:19:16 +02:00
Christian Decker
0aa5c197ec pytest: Fix tests broken by the pay and paystatus changes
This commit collects the changes required to the tests caused by the changes
to the `pay` and `paystatus` commands. They are also rather good hints as to
what these changes entail.
2020-07-07 23:25:45 +02:00
Christian Decker
2ac87c1490 paymod: Reorder paymod arguments to match pay
We've been adding modifiers and arguments out of order, and we need the
arguments order to match up if we want `paymod` to be a drop-in replacement.
2020-07-07 23:25:45 +02:00
Christian Decker
da8eb6fb4c paymod: Expose riskfactor and wire through to getroute 2020-07-07 23:25:45 +02:00
Christian Decker
5601a1ab21 paymod: Add a deadline to the pay command and retry modifier 2020-07-07 23:25:45 +02:00
Christian Decker
f557955515 paymod: Consolidate parsing RPC results in libplugin
We handle these in a number of different ways, and regularly get the parsing
and logic for optional fields wrong, so let's consolidate them here.
2020-07-07 23:25:45 +02:00
Christian Decker
cb00cbac7c paymod: Add support for the msatoshi override argument 2020-07-07 23:25:45 +02:00
Christian Decker
2331cd62e1 paymod: Add user-provided label back into the paystatus result 2020-07-07 23:25:45 +02:00
Christian Decker
935578e567 paymod: Exclude most expensive/slowest chan if limits are exceeded 2020-07-07 23:25:45 +02:00
Christian Decker
4648d2867f paymod: Compute fee and CLTV delta limits for a payment
So far we got away with not caring about these but since we're implementing
modifiers that impact these limits, we better keep track of them.
2020-07-07 23:25:45 +02:00