Commit Graph

1275 Commits

Author SHA1 Message Date
Rusty Russell
006ab1e367 pytest: test loading all plugins at once, including failing ones.
We modify the slow_init() so it doesn't go too slowly for this test.

This demonstrates a crash, where we currently try to fail a command
multiple times.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Rusty Russell
6441233d2b pytest: add test for a plugin which falls over outside a command.
This actually passes fine, but it's an interesting case to test.

Fixed-by: Darosior <darosior@protonmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-05 13:45:17 +09:30
Christian Decker
7b752e00d5 pyln: Set the bitcoin datadir when running tests
Telling `lightningd` to pass a `-datadir` to `bitcoin-cli` so it doesn't go
snooping where it doesn't belong (i.e., the user's home directory and config).

Changelog-None
Suggested-by: Simon Vrouwe <@SimonVrouwe>
Signed-off-by: Christian Decker <@cdecker>
2020-05-05 13:28:54 +09:30
rbndg
241fa00e97 plugin:added invoice creation event
New invoice_creation event triggered when an new invoice is created

Changelog-Added: plugin: New invoice_creation plugin event
2020-05-04 19:18:05 +02:00
Rusty Russell
7abff309e5 pytest: Fix flakiness in test_feerate_stress.
We occasionally get a temporary_channel_failure too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
Rusty Russell
4eb1233ccb lightningd: don't report spurious temporary_node_failure on local failures.
I noticed the following in logs for tests/test_connection.py::test_feerate_stress:

```
DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Failing HTLC 18446744073709551615 due to peer death
DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: local_routing_failure: 8194 (WIRE_TEMPORARY_NODE_FAILURE)
```

This is because it reports the (transient) node_failure error, because
our channel_failure message is incomplete.  Fix this wart up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
Rusty Russell
75496ad168 pytest: fix flaky test_peerinfo
If we don't wait for close tx to reach mempool, it might not get to
depth 100, and we don't get 'onchaind complete, forgetting peer'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
Rusty Russell
4df9b2c5f2 pytest: fix flake in test_onchain_different_fees()
We didn't wait until l2 processed the final state of HTLC #2, so
it might not include it when it drops onchain, leading to us only
getting 3 (not 4) sendrawtx calls.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-04 18:56:33 +09:30
lisa neigut
0e20e3c5e7 df: rename 'funder' to 'opener'
Previously we've used the term 'funder' to refer to the peer
paying the fees for a transaction; v2 of openchannel will make
this no longer true. Instead we rename this to 'opener', or the
peer sending the 'open_channel' message, since this will be universally
true in a dual-funding world.
2020-05-04 10:22:26 +09:30
Christian Decker
e7b555dd90 pytest: Stabilize test_penalty_{in,out}htlc tests
They were looking for specific amounts which proved to be rather flaky. Now
they look for specific outputs being available in the `listfunds` result after
everything was settled.
2020-05-01 18:20:33 -05:00
lisa neigut
f2aa770a3e tests: since features are longer now, so bytes read is longer
Since the node announcements now include the 55th bit flag for keysends,
the total amount of bytes read from disk is now +8
2020-04-28 10:39:07 -05:00
lisa neigut
d6d26dd056 features: split expected feature bits into node/peer sets
The new `keysend` plugin modifies the node features that we send to
peers. This commit breaks out the 'expected_features' we use for tests
to encompass this differentiation.
2020-04-28 10:39:07 -05:00
Christian Decker
9f298330b2 libplugin: It's featurebits, not features
The documentation was wrong, and I copied my mistake to `libplugin` where it
was then ignored instead of ORed into the node's featurebits. This fixes both.
2020-04-28 10:39:07 -05:00
Christian Decker
8c23f22591 pytest: Reproduce issue #3666
It seems we aren't registering the featurebits correctly.
2020-04-28 10:39:07 -05:00
Rusty Russell
bf053dcf7c plugins: simplify htlc_accepted hook payload-setting API.
As discussed with Christian, prepending the length to the payload returned
is awkward, but it's the only way to set a legacy payload.  As this will
be soon deprecated, simplify the external API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-16 18:03:35 +09:30
Rusty Russell
63441075b5 lightningd: allow htlc_accepted hook to replace onion payload.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-added: `htlc_accepted` hook can now offer a replacement onion `payload`.
2020-04-16 18:03:35 +09:30
Christian Decker
e03acd9663 libplugin: Add features to plugin_main and getmanifest 2020-04-16 18:03:35 +09:30
Christian Decker
568773daad pytest: Add a test for keysend
It currently uses a borrowed sending implementation from the noise plugin, but
we'll implement that functionality in the native keysend plugin next.
2020-04-16 18:03:35 +09:30
Rusty Russell
284bd7de0c pytest: fix flaky test_excluded_adjacent_routehint
We weren't actually waiting until l3 got the channel_update from l2,
so it might not be able to create the routehint.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-15 20:52:13 -05:00
Rusty Russell
1dc281c111 pytest: test for pay adjacent routehint crash.
When route returns a result which is too expensive, we try to figure out which
hop is most expensive to exclude it for next time.

If it's a single-hop route, we don't count it, since the first hop is free.
That's not usually a problem, since single-hop routes can't exceed our limits
(they're always "free"!).

But if we are using a routehint, the total cost could exceed our limits,
even if the start of the routehint is a single hop away.

This reproduces that test case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 14:13:26 -05:00
Rusty Russell
df31af5221 invoice: ignore dead-end heuristic on explicitly specified channels.
This makes testing easier, and makes sense: lightningd might not
*know* about other connected channels, depending on gossip, but if the
user specifies it we should obey it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON: `invoice` `exposeprivatechannels` now includes explicitly named channels even if they seem like dead-ends.
2020-04-14 14:13:26 -05:00
Rusty Russell
1d29228136 lightningd: make sendpay support blinding and enctlv fields (EXPERIMENTAL)
This is what actually lets us pay blinded invoices.

Unfortunately, our internal logic assumes every hop in a path has a
next `short_channel_id`, so we have to use a dummy.  This is
sufficient for testing, however.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell
bf2a42ada5 channeld: defer first update_fee until we have an HTLC to send.
Sending update_fee immediately after channel establishment seems to
upset LND, so work around it by deferring it.  The reason we increase
the fee after establishment is because now we might need to close the
channel in a hurry due to htlcs, but until there are htlcs that's
unnecessary.

Fixes: #3596
Changelog-Changed: Added workaround for lnd rejecting our commitment_signed when we send an update_fee after channel confirmed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-13 20:58:06 -05:00
Christian Decker
197a144505 plugin: Ensure RPC passthrough calls are terminated when plugin dies
We now track all pending RPC passthrough calls, and terminate them with an
error if the plugin dies.

Changelog-Fixed: JSON-RPC: Pending RPC method calls are now terminated if the handling plugin exits prematurely.
2020-04-14 11:07:55 +09:30
darosior
af76bce0fa pyln-client: send proper JSONRPC2 errors on failed call
Changelog-Added: pyln now sends proper error on bad calls to plugin methods

Co-Authored-By: Sergi Delgado Segura <sergi.delgado.s@gmail.com>
2020-04-11 12:17:51 +09:30
Christian Decker
8370a627bb invoice: Parse amounts without a multiplier as BTC not msats
The spec states that invoices with an amount, but lacking a multiplier, should
be interpreted as integer Bitcoin amounts:

   `amount`: optional number in that currency, followed by an optional
   `multiplier` letter. The unit encoded here is the 'social' convention of a
   payment unit -- in the case of Bitcoin the unit is 'bitcoin' NOT satoshis.

Suggested-by: Stefano Pellegrini <@St333p>
Signed-off-by: Christian Decker <@cdecker>
Changelog-Fixed: invoice: The invoice parser assumed that an amount without a multiplier was denominated in msatoshi instead of bitcoins.
2020-04-11 08:43:06 +09:30
Christian Decker
b68066e8e8 python: Consolidate requirements.txt files in a single place
We had them split according the separate use-cases:

 - testing
 - doc-gen
 - wire-gen

But that was causing new contributors to miss some dependencies when they
first got hacking. So this consolidates all of our own dependencies in a root
requirements.txt, with the notable exception of `pyln-client`, `pyln-testing`
and `pyln-proto` which are distributed as PyPI modules and therefore have
their own dependencies that need to be tracked in the module root.

Closes #3518
2020-04-09 15:14:06 +02:00
Vasil Dimov
158d2212c2 closingd: configurable closing fee negotiation step
When negotiating the transaction fee for closing a channel [1], we used
to always pick the middle of the range between our proposal and the
peer's proposal.

Introduce a new option `fee_negotiation_step` to the close command, so
the peer who initiates the close can choose his back off step.

Partially resolves https://github.com/ElementsProject/lightning/issues/3270

[1] https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#closing-negotiation-closing_signed

Changelog-Added: New optional parameter to the `close` command to control the closing transaction fee negotiation back off step
2020-04-07 13:52:48 +09:30
darosior
5aafef1484 pytest: skip some test when DEPRECATED_APIS is enabled. 2020-04-05 11:06:27 +09:30
Rusty Russell
28e3ffc66b plugins/fundchannel: make 'all' do the right thing for wumbo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
Rusty Russell
07a281faf8 lightningd: add large-channels / wumbo option.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-added: `large-channels` option to negotiate opening larger channels.
2020-04-03 13:13:21 +10:30
Rusty Russell
1d90f21833 lightningd / openingd: remove limits if we negotiate option_support_large_channel.
Note that now we check capacity once we've figured out which peer, which
broke a test (we returned "unknown peer" instead of "capacity exceeded"),
so we rework that too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-03 13:13:21 +10:30
Rusty Russell
7a95e90ee4 plugins: add feature_set to init object.
Shows what features we use in various contexts, including those added
by plugins in getmanifest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugin: `feature_set` object added to `init`
2020-04-03 13:13:21 +10:30
darosior
7c0af81c21 bcli: use a more urgent feerate for HTLCs and penalty transactions
A CONSERVATIVE/3 target for them.

Some noisy changes to the tests as we had to update the estimatesmartfee
mock.

Changelog-Changed: We now use a higher feerate for resolving onchain HTLCs and for penalty transactions
2020-04-01 23:02:47 -05:00
Rusty Russell
77e7beeb06 pytest: test the reply functionality (via blinded path) using a plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-02 14:32:38 +10:30
Rusty Russell
8b8cbb9397 channeld: handle encblob and blinding in messages.
This is based on https://github.com/lightningnetwork/lightning-rfc/blob/route-blinding/proposals/route-blinding.md

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-02 14:32:38 +10:30
Rusty Russell
e3dfba8944 pytest: test sendonionmessage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-02 14:32:38 +10:30
darosior
eead65350f wallet: allow to withdraw with unconfirmed utxos
Changelog-Fixed: Passing 0 as minconf to withdraw allows you to use unconfirmed transaction outputs, even if explicitly passed as the `utxos` parameter
2020-04-01 22:58:18 -05:00
Rusty Russell
afb76392e4 common/features: use bitmaps internally, have explicit init function.
This is to prepare for dynamic features, including making plugins first
class citizens at setting them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Rusty Russell
a430abf899 connectd: permit multiple descriptors of the same type.
This restriction was removed from the spec as of
86c2ebcc5973a4133d3ce4d80ae1c203061a1646.

We also fix up some strange formatting in that part of the documentation.

Changelog-changed: We now announce multiple addresses of the same type, if given.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Rusty Russell
5d4620484c lightningd: remove json_close force option.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON: `close` `force` parameter removed (deprecated in 0.7.2.1)
2020-03-30 12:47:01 +02:00
Rusty Russell
75838341a7 fundchannel/fundchannel_start: remove deprecated satoshi parameter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON: `fundchannel` and `fundchannel_start` `satoshi` parameter removed (renamed to `amount` in 0.7.3).
2020-03-30 12:47:01 +02:00
darosior
8a3295b6a8 chaintopology: Add the cost of HTLCs transactions to json_feerates
Changelog-Changed: "htlc_timeout_satoshis" and "htlc_success_satoshis" fields have been added to the `feerates` command.
2020-03-30 20:17:18 +10:30
darosior
5e72b22e80 bcli: adapt interface to the new fees estimation interface
We keep the same behaviour as lightningd before.
2020-03-30 20:17:18 +10:30
darosior
d4fe4073a4 lightning/bitcoind: adapt and batch fees estimations
This adapts our fee estimations requests to the Bitcoin backend to the
new semantic, and batch the requests.

This makes our request for fees much simpler, and leaves some more
flexibility for a plugin to do something smart (it could still lie before
but now it's explicit, at least.) as we don't explicitly request
estimation for a specific mode and a target.

Changelog-Changed: We now batch the requests for fee estimation to our Bitcoin backend.
Changelog-Changed: We now get more fine-grained fee estimation from our Bitcoin backend.
2020-03-30 20:17:18 +10:30
darosior
dce2e87928 chaintopology: better feerate targets differentiation
We kept track of an URGENT, a NORMAL, and a SLOW feerate. They were used
for opening (NORMAL), mutual (NORMAL), UNILATERAL (URGENT) transactions
as well as minimum and maximum estimations, and onchain resolution.

We now keep track of more fine-grained feerates:
- `opening` used for funding and also misc transactions
- `mutual_close` used for the mutual close transaction
- `unilateral_close` used for unilateral close (commitment transactions)
- `delayed_to_us` used for resolving our output from our unilateral close
- `htlc_resolution` used for resolving onchain HTLCs
- `penalty` used for resolving revoked transactions

We don't modify our requests to our Bitcoin backend, as the next commit
will batch them !

Changelog-deprecated: The "urgent", "slow", and "normal" field of the `feerates` command are now deprecated.
Changelog-added: The fields "opening", "mutual_close", "unilateral_close", "delayed_to_us", "htlc_resolution" and "penalty" have been added to the `feerates` command.
2020-03-30 20:17:18 +10:30
Michael Schmoock
0998164214 test: receivable_msat similar to spendable_msat 2020-03-27 12:33:23 +01:00
Michael Schmoock
80ff9c5b63 test: refactor lockup_drain add helpers drain and force_feerates 2020-03-27 12:33:23 +01:00
Christian Decker
e99ab03f79 pytest: Fix test_closing_fee regression in elements
Constants once again.
2020-03-24 09:52:33 +10:30
Christian Decker
9736c7bfd0 pytest: Make test_pay_retry less flaky for liquid-regtest 2020-03-24 09:52:33 +10:30