Commit Graph

151 Commits

Author SHA1 Message Date
Rusty Russell
bfcef9002e lightningd: fix handling of WIRE_UPDATE_FAIL_MALFORMED_HTLC.
1. We asserted that there wouldn't be a raw failcode.
2. We didn't pass the failure information via JSON in this case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-29 21:15:25 +01:00
Rusty Russell
c11212bb52 pytest: test that we handle WIRE_UPDATE_FAIL_MALFORMED_HTLC correctly.
We could use sendonion to do this, but it actually takes a different path through
pay, and I wanted to test all of it, so I made a new dev flag.

We currently get upset with the response:

	lightningd/pay.c:556: payment_failed: Assertion `!hout->failcode' failed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-29 21:15:25 +01:00
Christian Decker
b0529843ac pytest: Add a plugin for custommsgs and check that they get the msgs
This completes the custommsg epic, finally we are back where we began all that
time ago (about 4 hours really...): in a plugin that implements some custom
logic.
2020-01-28 23:50:52 +01:00
Christian Decker
b18c1ea543 json-rpc: Restrict custommsgs to be odd-typed
This solves a couple of issues with the need to synchronously drop the
connection in case we were required to understand what the peer was talking
about while still allowing users to experiment, just not kill connections.
2020-01-28 23:50:52 +01:00
Christian Decker
e9fc9aef34 channeld: Send messages if instructed to by lightningd 2020-01-28 23:50:52 +01:00
Christian Decker
3c88d5c8c4 openingd: Implement sendcustommsg handling in openingd
Most of the work is done in `lightningd`, here we just need to queue the
message itself.
2020-01-28 23:50:52 +01:00
darosior
dc3aa33927 pytest: use pyln.client for functional tests 2020-01-21 22:38:14 +01:00
ZmnSCPxj
54cc735201 lightningd/peer_control.c: Implement waitblockheight.
This is needed to fully implement handling of blockheight disagreements
between us and payee.
If payee believes the blockheight is higher than ours, then `pay`
should wait for our node to achieve that blockheight.

Changelog-Add: Implement `waitblockheight` to wait for a specific blockheight.
2020-01-21 22:23:21 +01:00
Christian Decker
8cad3ffeac pyln: Work around the socket path length on Linux OSs
Some Linux OSs impose a length limit on the path a Unix socket may have. This
is not an issue in `lightningd` since we `chdir()` into that directory before
opening the socket, however in pyln this became a problem for some tests,
since we use absolute paths in the testing framework. It's also a rather
strange quirk to expose to users.

This patch introduces a `UnixSocket` abstraction that attempts to work around
these limitations by aliasing the directory containing the socket into
`/proc/self/fd` and then connecting using that alias.

It was inspired by Open vSwitch code here https://github.com/openvswitch/ovs/blob/master/python/ovs/socket_util.py

Signed-off-by: Christian Decker <@cdecker>
2020-01-02 16:05:52 +01:00
Rusty Russell
839909d2cf Protocol: make var_onion, payment_secret and basic_mpp non-EXPERIMENTAL.
Thanks to @t-bast, who made this possible by interop testing with Eclair!

Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:16:03 +01:00
Rusty Russell
5d413e0f79 common: offer option_basic_mpp for EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 15:03:53 +01:00
Christian Decker
843846f4d5 pytest: Parametrize test_config_in_subdir for elements
Same as previous
2019-12-11 16:18:34 +01:00
Christian Decker
e1b1f47c53 pytest: Avoid starting the btcproxy twice 2019-12-11 16:18:34 +01:00
Jarret Dyrbye
d712f732d8 pylightning - allow unicode symbols in the description to be passed as-is to the daemon
addresses issue #2753.

Formatting the JSON with the default parameters will escape the unicode
symbols in a way that c-lightning won't allow, leading to an exception.

Changelog-Fixed: `pylightning` now handles unicode characters in JSON-RPC requests and responses correctly.
2019-11-29 21:38:17 +01:00
Christian Decker
dc1f27c0b3 pytest: Add failing test for unicode in JSON-RPC through pylightning 2019-11-29 21:38:17 +01:00
Michael Schmoock
6ed320189e feat: restructure plugin and options in listconfigs
This will change the command `listconfigs` output in several ways:

 - Deprecated the duplicated "plugin" JSON output by replacing it with
 - a "plugins" array with substructures for each plugin with:
 - path, name and their options

Changelog-Changed: JSON-RPC: `listconfigs` now structures plugins and include their options
Changelog-Deprecated: JSON-RPC: `listconfigs` duplicated "plugin" paths
2019-11-25 18:15:02 +01:00
Rusty Russell
8393d21503 common/features: add payment_secret feature if EXPERIMENTAL.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell
14997f6237 lightningd: fix handling of truncated config options.
Do the same thing '--help' does with them; append `...`.

Valgrind noticed that we weren't NUL-terminarting if answer was over
78 characters.

Changelog-Fixed: JSONRPC: listconfigs appends '...' to truncated config options.
2019-11-23 22:42:34 +00:00
Rusty Russell
badeaf1a67 pytest: upgrade tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell
57a32ee044 pytest: test for config parsing errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell
aab83e729b lightningd: change config-dir from plugin / wallet / hsm POV into <network> subdir
Changelog-changed: .lightningd plugins and files moved into <network>/ subdir
Changelog-changed: WARNING: If you don't have a config file, you now may need to specify the network to lightning-cli
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell
dc23c308e4 config: Read both top-level and network-subdir config files.
This lets you have a default, but also a network-specific config.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Options: `config` and <network>/`config` read by default.
2019-11-23 22:42:34 +00:00
Rusty Russell
34c89cb226 config: Add include directive support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: configuration files now support `include`.
2019-11-23 22:42:34 +00:00
Rusty Russell
654faa6174 lightningd: don't start if bitcoind is behind.
This leads to all sorts of problems; in particular it's incredibly
slow (days, weeks!)  if bitcoind is a long way back.  This also changes
the behaviour of a rescan argument referring to a future block: we will
also refuse to start in that case, which I think is the correct behavior.

We already ignore bitcoind if it goes backwards while we're running.

Also cover a false positive memleak.

Changelog-Fixed: If bitcoind goes backwards (e.g. reindex) refuse to start (unless forced with --rescan).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-21 05:18:29 +00:00
Rusty Russell
d4b48a6640 pytest: test that we refuse to follow bitcoind backwards.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-21 05:18:29 +00:00
Rusty Russell
709c98f539 logging: remove spaces from subsystem names.
Spaces just make life a little harder for everyone.

(Plus, fix documentation: it's 'jsonrpc' not 'json' subsystem).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
f91ff993ea pytest: deflake test_htlc_send_timeout
We can definitely get a pong from l1 (should test be slow enough):
it's l3 we are concerned about.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
4fc498f901 lightningd: enable io logging on subdaemons iff we're going to print it.
This simplifies our tests, too, since we don't need a magic option to
enable io logging in subdaemons.

Note that test_bad_onion still takes too long, due to a separate minor
bug, so that's marked and left dev-only for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell
e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
darosior
7dac7a8c52 pytest: deactivate shadow routing for some tests that use 'pay'
So that we can assert payment values
2019-11-08 03:27:58 +00:00
lisa neigut
181e0d4982 test: amend 'behind sync' tests
Check explicitly for both 'fundchannel' case and funding a transaction;
remove unnecessary 'second funding'.
2019-11-04 17:52:48 +01:00
Rusty Russell
ab48ecf07c pylightning: wrap signmessage/checkmessage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-15 17:15:23 +02:00
Rusty Russell
e5d9c7effc lightningd: checkmessage can intuit pubkey in some cases.
*If* we know the key has signed something else (as is the case for
channel_announcement) then we can effectively trust the key derivation.

This matches how LND's VerifyMessage works, though in the next patch
we will document it exactly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
150598e061 tests: add test for signmessage and lightning-checkmessage.
Thanks Twitter helpers @duck1123 and @jochemin for tests!
And @bitconner for the initial test vector.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
ca53c1b699 gossipd: push our own gossip messages harder.
I had a report of a 0.7.2 user whose node hadn't appeared on 1ml.  Their
node_announcement wasn't visible to my node, either.

I suspect this is a consequence of recent version reducing the amount of
gossip they send, as well as large nodes increasingly turning off gossip
altogether from some peers (as we do).  We should ignore timestamp filters
for our own channels: the easiest way to do this is to push them out
directly from gossipd (other messages are sent via the store).

We change channeld to wrap the local channel_announcements: previously
we just handed it to gossipd as for any other gossip message we received
from our peer.  Now gossipd knows to push it out, as it's local.

This interferes with the logic in tests/test_misc.py::test_htlc_send_timeout
which expects the node_announcement message last, so we generalize
that too.

[ Thanks to @trueptolmy for bugfix! ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 15:00:37 -05:00
darosior
43e55fc8e4 pytest: test lightningd startup with a relative config dir 2019-10-08 22:43:30 +00:00
Christian Decker
e23d8a4aff pytest: Mark some tests as skippable that are too bitcoin-specific 2019-10-03 04:32:57 +00:00
Christian Decker
858b3f2b93 pytest: Mark tests using network specific addresses under elements
We may be able to re-enable them later, but this is just painful right now.
2019-10-03 04:32:57 +00:00
Christian Decker
8a69ea7bf5 elements: Parametrize the sample addresses we test against
Since elements addresses look quite different from the bitcoin mainnet
addresses I just added a sample to the chainparams fixture. In addition I
extracted some of the fixed strings to reference chainparams instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker
e4c343a386 elements: Search for the correct address and invoice prefixes
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker
67fc8ffbcf elements: Use chainparams when the test relies on addresses
We were checking against hardcoded hrp and prefixes. Now we parametrize via
the chainparams.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Christian Decker
cd93a855ce elements: Give pytest some information about the test chain
We are checking against chain-dependent constants, so let's make sure we are
using the ones for the correct chain.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Rusty Russell
15612d269a Make option_static_remotekey non-EXPERIMENTAL now it's in spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 01:50:44 +00:00
Rusty Russell
4e8141aa36 features: remove INITIAL_ROUTING_SYNC.
It only had an effect if the peer didn't support option_gossip_queries, but
still, we don't want a gossip blast any more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-25 04:01:56 +00:00
Christian Decker
efc4aa94a3 pytest: Skip some tests that assume we have a sqlite3 db on postgres
These will not work since they touch the DB file itself.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Christian Decker
96a22b4003 pytest: Add db_provider and db instances for configurable backends
We will soon have a postgres backend as well, so we need a way to control the
postgres process and to provision DBs to the nodes. The two interfaces are the
dsn that we pass to the node, and the python query interface needed to query
from tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell
27790832a5 gossipd: gossip_queries_ex is not longer experimental.
The master spec has some typos which make it not parse, so I created
a PR and generated the CSV from that:

https://github.com/lightningnetwork/lightning-rfc/pull/673

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
147eaced2e developer: consolidiate gossip timing options into one --dev-fast-gossip.
It's generally clearer to have simple hardcoded numbers with an
#if DEVELOPER around it, than apparent variables which aren't, really.

Interestingly, our pruning test was always kinda broken: we have to pass
two cycles, since l2 will refresh the channel once to avoid pruning.

Do the more obvious thing, and cut the network in half and check that
l1 and l3 time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-20 06:55:00 +00:00