Commit Graph

124 Commits

Author SHA1 Message Date
Jan Sarenik
a777d21fb7 pytest: Rename all fund_channel to fundchannel 2020-09-25 21:04:56 +02:00
Rusty Russell
74e960c20b pytest: increase channel sizes so they reach min capacity with option_anchor_outputs
With a feerate of 7500perkw and subtracting 660 sats for anchors, a
20,000 sat channel has capacity about 9800 sat, below our default:

	You gave bad parameters: channel capacity with funding 20000sat, reserves 546sat/546sat, max_htlc_value_in_flight_msat is 18446744073709551615msat, channel capacity is 9818sat, which is below 10000000msat

So bump channel amounts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-01 12:27:39 -05:00
Rusty Russell
fde353ab00 pytest: use get_nodes more widely.
I started replacing all get_node() calls, but got bored, so then just did the
tests which call get_node() 3 times or more.

Ends up not making a measurable speed difference, but it does make some
things neater and more standard.

Times with SLOW_MACHINE=1 (given that's how Travis tests):

Time before (non-valgrind):
	393 sec (had 3 failures?)
Time after (non-valgrind):
	410 sec

Time before (valgrind):
	890 seconds (had 2 failures)
Time after (valgrind):
	892 sec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell
046b402c18 gossipd: return channel_announcement features for listchannels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON API: `listchannels` now shows channel `features`.
2020-05-07 08:44:58 +09:30
Rusty Russell
855debcfe1 gossipd: upgrade v7 gossip_store to v8.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-07 08:44:58 +09:30
Rusty Russell
46793bdaaf pytest: test gossip_store upgrade from version 7 to version 8.
The previous patch changed the gossip_store, but in a trivial way.
The next patch will implement upgrading, so this is the test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-07 08:44:58 +09:30
Rusty Russell
78ffea61e1 channeld: tell gossipd what the features are for our local channels.
This msg is stored in the gossip_store, so it means a version bump.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-07 08:44:58 +09:30
lisa neigut
f2aa770a3e tests: since features are longer now, so bytes read is longer
Since the node announcements now include the 55th bit flag for keysends,
the total amount of bytes read from disk is now +8
2020-04-28 10:39:07 -05:00
lisa neigut
d6d26dd056 features: split expected feature bits into node/peer sets
The new `keysend` plugin modifies the node features that we send to
peers. This commit breaks out the 'expected_features' we use for tests
to encompass this differentiation.
2020-04-28 10:39:07 -05:00
Rusty Russell
a430abf899 connectd: permit multiple descriptors of the same type.
This restriction was removed from the spec as of
86c2ebcc5973a4133d3ce4d80ae1c203061a1646.

We also fix up some strange formatting in that part of the documentation.

Changelog-changed: We now announce multiple addresses of the same type, if given.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
darosior
dc3aa33927 pytest: use pyln.client for functional tests 2020-01-21 22:38:14 +01:00
Christian Decker
cca18a151e pytest: Fix the TOR statictor and torblob tests
How did these ever work?
2020-01-13 08:55:32 +08:00
Rusty Russell
839909d2cf Protocol: make var_onion, payment_secret and basic_mpp non-EXPERIMENTAL.
Thanks to @t-bast, who made this possible by interop testing with Eclair!

Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:16:03 +01:00
Rusty Russell
5d413e0f79 common: offer option_basic_mpp for EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Christian Decker
d6a5b87395 pytest: Parametrize test_query_short_channel_id for elements
Same as the previous commit.
2019-12-11 16:18:34 +01:00
Christian Decker
fc93787db9 pytest: Parametrize test_gossip_query_channel_range for elements
We were hardcoding the chainparams->chain_hash which caused the query to
return an empty result. By parametrizing the test we can make it work on
elements.
2019-12-11 16:18:34 +01:00
Saibato
2c16b410ab Add basic tests
Signed-off-by: Saibato <saibato.naga@pm.me>
2019-12-03 23:35:18 +01:00
Rusty Russell
aab83e729b lightningd: change config-dir from plugin / wallet / hsm POV into <network> subdir
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell
3437f7e25d devtools/gossipwith: change timeout to seconds.
I always get this wrong, then wonder why it's dying!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 11:01:20 +01:00
Rusty Russell
709c98f539 logging: remove spaces from subsystem names.
Spaces just make life a little harder for everyone.

(Plus, fix documentation: it's 'jsonrpc' not 'json' subsystem).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
4fc498f901 lightningd: enable io logging on subdaemons iff we're going to print it.
This simplifies our tests, too, since we don't need a magic option to
enable io logging in subdaemons.

Note that test_bad_onion still takes too long, due to a separate minor
bug, so that's marked and left dev-only for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell
e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
40d34fed9e pytest: clean up test_gossip_notices_close now that gossipwith has more options.
And drive-by fix: document that you can now (since
e40f07803c) use --max-messages=0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-07 03:50:53 +00:00
Rusty Russell
30634aa837 pytest: fix test_gossip_notices_close where we really do inject bad gossip!
It currently works because we inject it so fast that it's still doing the
txout lookup, but that's about to change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-07 03:50:53 +00:00
Rusty Russell
2d8e93687d pytest: prepare test_gossip_timestamp_filter to be spammed.
We're about to change it so we always send our local messages, which
breaks this test.  Add a new node which doesn't have any local
messages, so the test works correctly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-04 17:50:58 +01:00
Rusty Russell
bd55f6d940
common/features: only support a single feature bitset.
This is mainly an internal-only change, especially since we don't
offer any globalfeatures.

However, LND (as of next release) will offer global features, and also
expect option_static_remotekey to be a *global* feature.  So we send
our (merged) feature bitset as both global and local in init, and fold
those bitsets together when we get an init msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-11 02:52:04 +00:00
Rusty Russell
213263ca3f pytest: filter out gossip_timetamp_filter should we receive it.
We completely rework test_node_reannounce: it's assumes we always ask for
all gossip and that assumption will be broken in future patches too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
33c658ecfb gossipd: advertize all our features in node_announcement.
This preempts the acceptance of
https://github.com/lightningnetwork/lightning-rfc/pull/666 but it's
clear that feature bits are going to be distinct, so this is safe to
do anyway.

See https://github.com/lightningnetwork/lightning-rfc/pull/680

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-07 05:51:05 +00:00
Christian Decker
e23d8a4aff pytest: Mark some tests as skippable that are too bitcoin-specific 2019-10-03 04:32:57 +00:00
Rusty Russell
18aabc3596 pytest: use query_gossip in test_gossip_query_channel_range.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell
1386fedfb6 pytest: use query_gossip in test_query_short_channel_id.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell
d534a146d2 pytest: clean up test_gossip_timestamp_filter, use query_gossip.
It relies on the fact that nodes don't do their own gossip queries.
Use devtools instead.

This revealed that the entire logic was broken!  It just happened to work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 07:08:07 +00:00
Rusty Russell
d24c850899 gossipd: restore a flag for fast pruning
I was seeing some accidental pruning under load / Travis, and in
particular we stopped accepting channel_updates because they were 103
seconds old.  But making it too long makes the prune test untenable,
so restore a separate flag that this test can use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-27 00:01:34 +00:00
Rusty Russell
39c9dcbafc ratelimit: adjust based on --dev-fast-gossip, test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell
147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell
a92ead48bf gossipd: ignore redundant channel_update and node_announcement.
If you send a message which simply changes timestamp and signature, we
drop it.  You shouldn't be doing that, and the door to ignoring them
was opened by by option_gossip_query_ex, which would allow clients to
ignore updates with the same checksum.

This is more aggressive at reducing spam messages, but we allow refreshes
(to be conservative, we allow them even when 1/2 of the way through the
refresh period).

I dropped the now-unnecessary sleep from test_gossip_pruning, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell
0bab2580fc gossipd: clean up local channel updates.
Make update_local_channel use a timer if it's too soon to make another
update.

1. Implement cupdate_different() which compares two updates.
2. make update_local_channel() take a single arg for timer usage.
3. Set timestamp of non-disable update back 5 minutes, so we can
   always generate a disable update if we need to.
4. Make update_local_channel() itself do the "unchanged update" suppression.
   gossipd: clean up local channel updates.
5. Keep pointer to the current timer so we override any old updates with
   a new one, to avoid a race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
Rusty Russell
70c4ac6d74 gossipd: suppress our own too-close node_announcement messages.
Never make them less than gossip_min_interval apart.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00
trueptolemy
059a6e0e0d pytest: Test excluding nodes in getroute 2019-09-16 12:22:06 +08:00
Rusty Russell
1c0d435f5e pytest: remove flaky part of test_gossip's test_gossip_no_empty_announcements
This "wait_for" failed on Travis, but it's unnecessary anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell
c99906a9a9 per-peer-daemons: tie in gossip filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Rusty Russell
5292f11818 pytest: test (fail) that we don't repeat gossip back to the node we got it from
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-06 14:35:01 +02:00
Christian Decker
8b8538024d bitcoind: Defer initialization of filteredblock_call->result
During sync it is highly likely that we can coalesce multiple calls and share
results among them. We also report back failures for non-existing blocks early
on, so we don't run into issues with blocks that our bitcoind doesn't have
yet.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-20 00:07:38 +00:00
Christian Decker
187e493ab8 gossip: Stop backfilling the future
This was caused by us not checking against the max_blockheight, but rather the
min_blockheight which can be negative with a newly created node. This is still
safe since we check for duplicates anyway in `wallet_filteredblock_add`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-08-20 00:07:38 +00:00
Rusty Russell
944439853a pytest: two tests for gossip of channels in as-yet-unknown blocks.
Two tests which crash lightningd in different ways.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-20 00:07:38 +00:00
Rusty Russell
bf3b77a947 Travis: skip testing VALGRIND=1 DEVELOPER=0, remove the slowest non-developer tests.
I don't remember ever seeing a bug which only showed up in VALGRIND=1 with developer
mode disabled, so don't test that, and spread out the other test more evenly.

In addition, disable the worst-performing tests in DEVELOPER=0 mode.

Here timings from my build machine: the worst 6 (- DEVELOPER=0 VALGRIND=0)
with the same tests (+ DEVELOPER=1 VALGRIND=1)

-452.42s call     tests/test_pay.py::test_channel_spendable
+87.69s call     tests/test_pay.py::test_channel_spendable
-335.66s call     tests/test_gossip.py::test_gossip_store_compact_on_load
+47.41s call     tests/test_gossip.py::test_gossip_store_compact_on_load
-332.07s call     tests/test_connection.py::test_opening_tiny_channel
+89.71s call     tests/test_connection.py::test_opening_tiny_channel
-331.97s call     tests/test_pay.py::test_channel_spendable_large
+56.23s call     tests/test_pay.py::test_channel_spendable_large
-305.28s call     tests/test_invoices.py::test_invoice_routeboost
+37.57s call     tests/test_invoices.py::test_invoice_routeboost
-284.28s call     tests/test_plugin.py::test_htlc_accepted_hook_forward_restart
+49.12s call     tests/test_plugin.py::test_htlc_accepted_hook_forward_restart

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-14 11:14:38 +00:00
ZmnSCPxj
3e74ca4b86 gossipd/routing.c: Correctly handle a duplicated entry in exclude of getroute. 2019-08-02 16:06:15 +02:00
ZmnSCPxj
a5fb37298c tests/test_gossip.py: Add test to check that duplicated exclusions in getroute have no lasting effect. 2019-08-02 16:06:15 +02:00