Commit Graph

1712 Commits

Author SHA1 Message Date
Christian Decker
2834aaced0 gci: Stabilize test_forward_event_notification 2021-01-29 10:29:09 +10:30
Christian Decker
62cb1c3fbc pytest: Stabilize test_forward_stats 2021-01-29 10:29:09 +10:30
Christian Decker
da2e956538 pytest: Stabilize test_routing_gossip
openchannel internally generates blocks, which may cause nodes to be
out of sync and ignore "future" channel announcements, resulting in
bad gossip.
2021-01-29 10:29:09 +10:30
Christian Decker
5ecaff65ee pytest: Give each run of the hsmtool its own pty 2021-01-29 10:29:09 +10:30
Christian Decker
1463797c61 pytest: Stabilize test_funding_close_upfront
Reconnections and unsynchronized states where causing us some issues.
2021-01-29 10:29:09 +10:30
Christian Decker
bbdf35c6fe pytest: Stabilize test_closing_negotiation_reconnect
The test was not considering that concurrent sendrawtx of the same tx
is not stable, and either endpoint will submit it first. Now just
checking state transitions and the mempool.
2021-01-29 10:29:09 +10:30
Christian Decker
52e82b76b6 pytest: Stabilize test_bad_onion 2021-01-29 10:29:09 +10:30
Christian Decker
3d4c111721 pytest: Stabilize test_multiple_channels
If we're quick (or the node is slow) we end up reconnecting before our
counterparty has realized the state transition, resulting in an
unexpected re-establish.
2021-01-29 10:29:09 +10:30
Christian Decker
542f3225e3 pytest: Parameterize process waits for hsmtool calls
We were sometimes waiting only 5 seconds, which is way too short on a
heavily loaded machine such as CI. Making it 30 seconds and collecting
it in a single place so we can adjust more easily.
2021-01-29 10:29:09 +10:30
Christian Decker
8cc62d76e4 pytest: Stabilize test_channel_{spendable,receivable}
They were using TIMEOUT / 2 which may be way too long (hit against
test timeout), so we use a still ludicrous 30 seconds instead.
2021-01-29 10:29:09 +10:30
Christian Decker
6384cadd69 pytest: Stabilize the negotiation tests
We also make the logic a bit nicer to read. The failure was due to
more than one status message being present if we look at the wrong
time:

```
arr = ['CLOSINGD_SIGEXCHANGE:We agreed on a closing fee of 20334 satoshi for tx:17f1e9d377840edf79d8b6f1ed0faba59bb307463461...9b98', 'CLOSINGD_SIGEXCHANGE:Waiting for another closing fee offer: ours was 20334 satoshi, theirs was 20332 satoshi,']                                                                                                   │

     def only_one(arr):
         """Many JSON RPC calls return an array; often we only expect a single entry
         """
 >       assert len(arr) == 1
 E       AssertionError
```
2021-01-29 10:29:09 +10:30
Christian Decker
04ed93f5f8 pytest: Stabilize test_funding_external_wallet_corners 2021-01-29 10:29:09 +10:30
Christian Decker
18483ca582 pytest: Disable test_v2_open if not developer
It requires `--dev-force-features` which isn't available without
`DEVELOPER=1`
2021-01-29 10:29:09 +10:30
Christian Decker
7962db821c pytest: Stabilize test_channel_state_changed_bilateral 2021-01-29 10:29:09 +10:30
Christian Decker
07f5054700 pytest: Stabilize test_setchannelfee_state
Synching with the blockchain was slower than our timeout...
2021-01-29 10:29:09 +10:30
Christian Decker
03449e3cf0 pytest: Stabilize test_gossip_persistence
We weren't waiting for the `dev_fail` transaction to hit the mempool,
throwing the results off.
2021-01-29 10:29:09 +10:30
Christian Decker
ae40c10bcb pytest: Stabilize test_onchain_timeout
The timeout on the pay future was too short under valgrind.
2021-01-29 10:29:09 +10:30
Christian Decker
c564f165fa pytest: Stabilize test_penalty_htlc_tx_timeout
We weren't waiting for the transactions to enter the mempool which
could cause all of our fine-tuned block counts to be off. Now just
waiting for the expected number of txs.
2021-01-29 10:29:09 +10:30
Christian Decker
c0f06f2779 pytest: Simplify and stabilize test_reconnect_no_update 2021-01-29 10:29:09 +10:30
Christian Decker
2b12cac31e pytest: Skip hsm encryption test if we don't have a TTY 2021-01-29 10:29:09 +10:30
Rusty Russell
001b5d6416 offers: make it a runtime option.
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
2021-01-13 14:45:36 +01:00
Rusty Russell
27c006f7aa libplugin: make init return a string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: init can return a non-NULL string to disable the plugin.
2021-01-13 14:45:36 +01:00
Rusty Russell
529ae0d766 plugins: allow plugins to disable themselves at startup.
By returning 'disable: <reason>' inside getmanifest or init result.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: plugins can now disable themselves by returning `disable`, even if marked important.
2021-01-13 14:45:36 +01:00
Rusty Russell
fc3e679c97 lightningd: control onion messages by experimental-onion-messages option.
Note that this also changes so the feature is not represented in channels,
reflecting the recent drafts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-onion-messages` enables send, receive and relay of onion messages.
2021-01-13 14:45:36 +01:00
niftynei
7e324177de df-tests: simultaneous openchannel_init (while in progress)
Reject a peer's request to open a channel while we're already in progress
2021-01-10 13:44:04 +01:00
niftynei
2c9ce2566c df-tests: test_lightningd_still_loading 2021-01-10 13:44:04 +01:00
niftynei
a357e75c5d df-tests: test_disconnect_half_signed + test_reconnect_signed 2021-01-10 13:44:04 +01:00
niftynei
fec1d9b707 df-tests: test_reconnect_openingd 2021-01-10 13:44:04 +01:00
niftynei
7eb6213fca df-tests: test_reconnect_sender_add1, others
We're still ignoring multifundchannel failure tests, but now we use the
'dual-fund' flag instead of a blanket true.
2021-01-10 13:44:04 +01:00
niftynei
d9c68a708a df-tests: test_reconnect_receiver_add 2021-01-10 13:44:04 +01:00
niftynei
96c42262ab df-tests: test_peerinfo 2021-01-10 13:44:04 +01:00
niftynei
135cd178f7 df-tests: test_channel_persistence 2021-01-10 13:44:04 +01:00
niftynei
a82bfa83ff df-tests: have the df_accepter plugin keep track of attempts
Test connection/reconnection handling for v2 opens. We needed to fixup
the accepter plugin so that we were freeing up inputs on
disconnect/failure.
2021-01-10 13:44:04 +01:00
niftynei
88b4b5b2bb df-tests: test_disconnect_opener 2021-01-10 13:44:04 +01:00
niftynei
3e330d7186 df-tests: test_forward_local_failed_stats 2021-01-10 13:44:04 +01:00
niftynei
973224dd14 df-tests: if we're using fundbalancechannel, set up plugin
nodes need to be using the accepter plugin, so they'll match funding.
we used to use push_msat for this, but v2 gets rid of it.

*sad trombone*
2021-01-10 13:44:04 +01:00
niftynei
f822a93cf4 df-tests: add setacceptmaxfunding to df_accepter plugin
Method for setting a ceiling on the fund matching capabilities of
df_accepter. Setting it to zero means we don't fund the channel anymore.
2021-01-10 13:44:04 +01:00
niftynei
e964c31369 df-tests: test_payment_success|failed_persistence 2021-01-10 13:44:04 +01:00
niftynei
09da171381 df-tests: test_io_logging
Move pid collection down to when dualopend is definitely dead
2021-01-10 13:44:04 +01:00
niftynei
609d509ca5 df-tests: test_channel_state_changed_unilateral 2021-01-10 13:44:04 +01:00
niftynei
ac88c72f1e df-tests: test_channel_state_changed_bilateral 2021-01-10 13:44:04 +01:00
niftynei
450a094d26 df-tests: test_coin_movement_notices 2021-01-10 13:44:04 +01:00
niftynei
923e1f5c80 df-tests: test_feature_set 2021-01-10 13:44:04 +01:00
niftynei
db65771f38 df-tests: test_plugin_feature_announce 2021-01-10 13:44:04 +01:00
niftynei
21b9bec93b df-tests: test_channel_state_change_history 2021-01-10 13:44:04 +01:00
niftynei
959ca793b0 df-tests: test_openchannel_hook_error_handling 2021-01-10 13:44:04 +01:00
niftynei
803d9701d3 df-tests: test_openchannel_hook_chaining 2021-01-10 13:44:04 +01:00
niftynei
b0a8b10b41 df-tests: test_openchannel_hook
Have it check both openchannel + openchannel2
2021-01-10 13:44:04 +01:00
niftynei
0d45823b82 df-tests: test_funding_reorg_private 2021-01-10 13:44:04 +01:00
niftynei
09834bbc8d df-tests: test_node_reannounce correct features 2021-01-10 13:44:04 +01:00
niftynei
c055198869 df-tests: test_fail_unconfirmed, duplicate with correct wires 2021-01-10 13:44:04 +01:00
niftynei
dfa0eb0f2f df-tests: test_shutdown_awaiting_lockin check correct state 2021-01-10 13:44:04 +01:00
niftynei
5c7cc15a75 df-tests: adjust logsearch/features for test_dataloss_protection 2021-01-10 13:44:04 +01:00
niftynei
e661d87440 df-tests: fixup expected features for dual-funding channels 2021-01-10 13:44:04 +01:00
niftynei
08200f03a7 df-tests: test_multifunding_v2_v1_mixed update to use flag
Now that we've got a fancy flag to enable dual funding, let's use it
2021-01-10 13:44:04 +01:00
niftynei
8fb97fd7e1 df-tests: updates for plugin hook_accepter helper
Fixes so that test_funding_close_upfront works with v2 opens
2021-01-10 13:44:04 +01:00
niftynei
f4b7904ce5 tests: don't enforce daemon name on opening logline 2021-01-10 13:44:04 +01:00
niftynei
9951638d5f df: add flag to signal to run v2 channel open tests 2021-01-10 13:44:04 +01:00
niftynei
99a621dd99 df-reconnects: allow tx-sigs in channeld iff we're reconnecting
There's a case where a dropped funding_locked will result in the peer
moving onto channeld, while we stay in dualopend. As we haven't
received their funding_locked, we retransmit tx_sigs, which channeld
will need to handle.

With the patch the peer drops it on the floor; the peer will resend
funding_locked on reconnect, which will correctly advance us to
channeld and CHANNELD_NORMAL
2021-01-10 13:44:04 +01:00
Rusty Russell
af46a4f57d fetchinvoice: allow amounts to be specified.
As per lastest revision of the spec, we can specify amounts in invoice
requests even if the offer already specifies it, as long as we exceed
the amount given.  This allows for tipping, and amount obfuscation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell
4bb05e46e9 offers: convert currency when they request an invoice.
Means a reshuffle of our logic: we want to multiply by quantity before
conversion for maximum accuracy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell
9681d491df offer: allow offers in other currencies if we can convert.
This avoids a footgun where they create an offer then we can't create
the invoice because they don't have a converter plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell
23af9d4972 onion_message: support variable-length onion messages.
Updated to the BOLT, and a few tweaks, and we can send giant onion_messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell
d867fab3a7 sendinvoice: call waitinvoice directly to on invoice payment.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
1aa7e8e011 offers: split offer send_invoice generation into new JSON command
We split `send_invoice` offers inoo offerout (for want of a better name).
This simplifies the API.

Also took the opportunity to move the `vendor` tag to immediately
follow `description` (our tests use arguments by keywords, so no
change there).

Suggested-by: shesek
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
55983827b6 fetchinvoice: try direct peers if we can't route.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
bdfcec3eb6 fetchinvoice: check we're in the period before attempting to fetch.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
dae477175c fetchinvoice: implement timeout.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
075c25fc08 plugins/fetchinvoice: handle sendinvoice timeout, error or payment.
If they pay the invoice, they don't bother replying; that's just for
errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
43f5304539 plugins/offer: allow offer to create send_invoice, refund and single_use offers.
refund offers are implied send_invoice offers.  And send_invoice offers
are implied single-use offers, though it can also make sense to have
a non-send_invoice offer be single-use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Christian Decker
6b6e0fe34c pytest: Mark test_lockup_drain as disabled
It's been causing me headaches for some time, and it's just getting
annoying. Will file an issue to track progress.
2021-01-08 19:28:30 +01:00
niftynei
d0dcb7cb42 tests: wait for drop to chain test_onchain_middleman_their_unilateral_in
this was catching the wrong 'sendrawtx exit 0' and consequently
not mining the close tx into a block
2021-01-08 19:28:30 +01:00
Rusty Russell
1a8978100e invoice: add ctlv option.
This is required if we want to create a "bouncer" plugin (in my copious free time!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `invoice` now takes an optional `cltv` parameter.
2021-01-08 10:45:56 +01:00
Christian Decker
4b2efd69d7 pay: Fix a crash when waitblockheight times out
Fixes #4309
Changelog-Fixed: pay: Fixed an issue where waiting for the blockchain height to sync could time out.
2021-01-08 09:19:24 +01:00
Christian Decker
ca204eacfa pytest: Reproduce waitblockheight timeout issue #4309 2021-01-08 09:19:24 +01:00
Christian Decker
70410b8ee8 pytest: Test LightningRpc and plugin command notification support 2021-01-07 08:32:40 +01:00
Christian Decker
84b3653606 pyln: Add command notification support for plugins
Changelog-Added: pyln-client: Plugin methods can now report progress or status via the `Request.notify` function
2021-01-07 08:32:40 +01:00
Christian Decker
a1d33c3781 pytest: Add a test for listinvoices
We now have some more logic in the query, so let's test it
exhaustively.
2021-01-06 18:48:06 +01:00
Antoine Poinsot
a4f07a3e78 tests/fuzz: add a fuzz target for hsm_encryption
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
506d524508 libfuzz: add a NULL-termination in to_string
It's more useful if we actually want to use the output as, well, a
string..

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
Antoine Poinsot
56c223886c lightning: confirm password on hsm_secret encryption
Changelog-changed: lightningd: the `--encrypted-hsm` now asks you to confirm your password when first set
Changelog-changed: hsmtool: the `encrypt` now asks you to confirm your password
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-06 13:50:01 +01:00
niftynei
0c9072272f df-plugin: add signonly to signing in df_accepter
We're the accepter, so we look up which inputs we need to sign and only
sign those.
2021-01-05 19:12:00 +01:00
niftynei
e3a1d1a7f3 pyln-client: to_whole_satoshi returns the rounded up satoshi value
A fractional satoshi value isn't really useful; rounding up loses
precision but that's why you called "whole satoshi", wasn't it?

Changelog-Changed: pyln-client: Millisatoshi has new method, `to_whole_satoshi`; *rounds value up* to the nearest whole satoshi
2021-01-05 19:12:00 +01:00
niftynei
cce0020568 pyln-client: add missing min_witness_weight to fundpsbt/utxopsbt
We need to use it for the 'df_accepter' plugin, so we get the feerate
correct.

Changelog-Added: pyln-client: `fundpsbt`/`utxopsbt` now support `min_witness_weight` param
2021-01-05 19:12:00 +01:00
Antoine Poinsot
19afe15968 hsmtool: only write 32 bytes to hsm_secret on generatehsm
Fixes #4302
Changelog-fixed: hsmtool: the `generatehsm` command now generates an appropriately-sized hsm_secret
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot
aa2586a45a pytest: reproduce issue #4302
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 19:02:28 +01:00
Antoine Poinsot
1513a2d07e hsmtool: input encryption password from stdin
This slightly breaks the API, but still accept the input: we just don't
take it into account anymore.

For `dumponchaindescriptors`, we have to still take the old place of the
`network` parameter into account to not entirely break the API.

Changelog-Added: hsmtool: password must now be entered on stdin. Password passed on the command line are discarded.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-04 16:15:40 +01:00
Christian Decker
5b3af00e28 pytest: Really only check DB statements if the env-var is "1" 2021-01-04 11:25:50 +01:00
Christian Decker
94820ad559 pytest: Disable test_funding_cancel_race on slow machines
The CI regularly trips up on this very taxing test (100 nodes) so only
run it if we have the required horsepower.
2021-01-04 11:25:50 +01:00
Christian Decker
b3fd47e5f6 pytest: Stabilize test_routing_gossip
We were getting bad gossip because some nodes discarded the channel
announcement for being in the future. This is because the node was, at
that time, below the confirmation height. It'd then discard the
followup messages because not preceded by an announcement, and getting
upset about that.
2021-01-04 11:25:50 +01:00
Christian Decker
7e867e5ee6 pytest: Add throttler to limit load on the test system
Both my machine and apparently the CI tester machines regularly run
into issues with load on the system, causing timeouts (and
unresponsiveness). The throttler throttles the speed with which new
instances of c-lightning get started to avoid overloading. Since the
plugin used for parallelism when testing spawns multiple processes we
need to lock on the fs. Since we have that file open already, we'll
also write a couple of performance metics to it.
2021-01-04 11:25:50 +01:00
Rusty Russell
52af729641 plugins/offer and plugins/fetchinvoice: send and recv errors.
This also lets us extend our testing to cover error cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
96caf9f4ab fetchinvoice: return the next period for recurring offers.
This is useful for the caller to know when to call again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
19af1d10e6 invoice: hack in merkle of invoice as "payment_secret" (EXPERIMENTAL_FEATURES)
This lets actually pay the invoice that fetchinvoice returns.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
2baa410e2b pytest: finally test fetchinvoice send, and offer plugin receive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
08e110b568 JSON: offers plugin offer command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Karol Hosiawa
e582c0ac0a Added listfunds test 2020-12-22 13:00:02 +01:00
Rusty Russell
723c16072a cleanups: feedback from Christian Decker review.
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
a33e39b7e8 pay, wallet: rename internal bolt11 vars to invstring.
And handle bolt12 strings if EXPERIMENTAL_FEATURES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
d971e3de98 Plugin: support extra args to "start".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `start` command can now take plugin-specific parameters.
2020-12-15 09:28:56 +10:30