Commit Graph

781 Commits

Author SHA1 Message Date
Rusty Russell
2bd7df93c6 gossipd: preserve unannounced channels across store compaction.
Otherwise we'd forget them on restart, again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 04:41:43 +00:00
Rusty Russell
3dd47950ad pytest: test that gossipd remembers unannounced local channels across restarts
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 04:41:43 +00:00
Rusty Russell
7ccf3af51d pytest: test the db hook plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell
a498b1fdbf pytest: fix flaky test_setchannel_fee_restart test.
gossipd in l1 might not have registered l2 reconnecting, thus considering
the channel local-disabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-19 14:22:27 +01:00
Michael Schmoock
162a97894d test: adds test_setchannelfee_all
- will also update old setchannelfee tests to new command output
2019-03-19 00:14:09 +00:00
Michael Schmoock
7e1cc2897c test: add tests for setchannelfee command
Goissp related tests are disabled for non-developers.
New setchannelfee testscases

* py.test -v tests/test_pay.py -k test_setchannelfee_usage
* py.test -v tests/test_pay.py -k test_setchannelfee_routing
* py.test -v tests/test_pay.py -k test_setchannelfee_state
* py.test -v tests/test_pay.py -k test_setchannelfee_zero
* py.test -v tests/test_pay.py -k test_setchannelfee_restart
2019-03-15 02:48:18 +00:00
Rusty Russell
1069f48082 connectd: fix binding to a UNIX domain socket.
lightning_connectd(19780): STATUS_FAIL_INTERNAL_ERROR: Failed to bind on 2 socket: Address family not supported by protocol

"Untested code is buggy code"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-14 15:05:56 +01:00
Rusty Russell
740f4314ea pytest: test binding lightningd to a UNIX domain socket.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-14 15:05:56 +01:00
Rusty Russell
cc342fda83 pylightning: make sure UTF-8 gets passed intact.
1. We need to read in as a byte string, then decode into utf8 once we
   have a marker.  Otherwise we seem to mangle it horribly, and we
   might have a bad utf8 string anyway.

2. We need to suppress the JSON \u escapes on output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-05 11:43:22 +01:00
Rusty Russell
5f437de771 pytest: test passing utf8 into and out of plugin JSON.
We should be able to pass UTF-8 strings to and from plugins without
python turning them into JSON-\u escapes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-05 11:43:22 +01:00
Rusty Russell
049f2351d4 newaddr: deprecate 'address' output.
It doesn't make sense with 'all', and it's ok to be explicit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Rusty Russell
3e67c09d5e newaddr: support getting both bech32 and p2sh addresses.
Higher layers consume less addresses this way.

Fixes: #2390
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 15:11:20 +01:00
Rusty Russell
6e63d79159 pylightning: translate msat input to class Millisatoshi
Rather than using LightningJSONDecoder's implicit "field name and
value ends in msat, try converting to Millisatoshi", we do it to
parameters using type annotations.

If you had a parameter which was an array or dict itself, we don't
delve into that, but that's probably OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-25 21:45:21 +00:00
Rusty Russell
c7316d7ba2 pytest: test that we translate to and from Millisatoshi on plugin RPC.
We don't, but we should, like we do for normal RPC.  However, I chose
to use function annotations, rather than names-ending-in-'msat'
because it's more Pythony.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-25 21:45:21 +00:00
Rusty Russell
203ef2ed0b listsendpays: updated version of listpayments.
New name is less confusing, and most people should be transitioning to
listpays rather than this anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-23 05:45:25 +00:00
Rusty Russell
1ec959752b plugin/pay: rename 'description' to 'label', deprecate 'description'.
This is the same deprecation, but one level up.  For the moment, we
still support invoices with a `h` field (where description will be
necessary) but that will be removed once this option is removed.

Note that I just changed pylightning without backwards compatibility,
since the field was unlikely to be used, but we could do something
more complex here?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-23 05:45:25 +00:00
Christian Decker
ed6a455a3c wallet: Display addresses derived from scriptPubKey where available
In particular this matches the case of `their_unilateral/to_us` outputs, which
were missing their addresses so far.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 11:15:24 -08:00
Christian Decker
72f1c78a1e jsonrpc: Arm the minconf=1 parameter and deal with the fallout
We want to disallow using unconfirmed outputs by default, so making the
default 1 confirmation seems a good idea. This also matches `bitcoind`s
minimum confirmation requirement.

Arming however breaks some of our tests, so I used `minconf=0` for the
breaking tests and added a new test specifically for the `minconf` parameter
for `fundchannel`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Christian Decker
7f11b4854e pylightning: Split @method and @async_method decorators
Suggested-by: Rusty Russell <@rustyrussell>
Suggested-by: Conor Scott <@conscott>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
571fb44d20 pylightning: Add a small test for async rpcmethods
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Rusty Russell
6ed54dd34e db: fix dangling peers on db upgrade.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
594f260de7 pytest: test starting with db corrupted by dangling peer.
db was taken from the failed test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
7887e5c00a pytest: add support for starting nodes with a pre-canned db.
With xz, the db is only 9120 bytes, vs 163840.  And lzma is a builtin
in Python 3.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
3b587a1c6d lightningd: fix db error where we can have detached peer.
An uncommitted channel should not keep the peer in the db, since the
uncommitted channel isn't in the db itself.

Fixes: #2367
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
26df586fbf pytest: add test for db constraint failure
wallet_channel_insert: UNIQUE constraint failed: peers.node_id
lightningd: Fatal signal 6 (version v0.6.3rc1-202-g4c8cb98)
0x555625b14261 crashdump
	common/daemon.c:40
0x7f8f7400e0ff ???
	???:0
0x7f8f7400e077 ???
	???:0
0x7f8f73fef534 ???
	???:0
0x555625af34f2 fatal
	lightningd/log.c:624
0x555625b3dd96 db_exec_prepared_
	wallet/db.c:448
0x555625b44fcd wallet_channel_insert
	wallet/wallet.c:1067
0x555625af64f7 wallet_commit_channel
	lightningd/opening_control.c:229
0x555625af6e86 opening_funder_finished
	lightningd/opening_control.c:394
0x555625af7bfa openingd_msg
	lightningd/opening_control.c:714
0x555625b0d421 sd_msg_read
	lightningd/subd.c:474
0x555625b0cd3b read_fds
	lightningd/subd.c:302
0x555625b548d3 next_plan
	ccan/ccan/io/io.c:59
0x555625b553ef do_plan
	ccan/ccan/io/io.c:395
0x555625b5542d io_ready
	ccan/ccan/io/io.c:405
0x555625b5700e io_loop
	ccan/ccan/io/poll.c:310
0x555625af1562 main
	lightningd/lightningd.c:827
0x7f8f73ff109a ???
	???:0
0x555625adaad9 ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20190220020526

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
948ca470ad bitcoin: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
0d30b89043 channeld: use amount_msat for struct htlc amount.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
85b8b25749 bitcoin/chainparams: use amount_sat / amount_msat
Simple changes, but ripples through the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
cd341b34d6 plugins/pay: use struct amount_msat.
This is particularly interesting because we handle overflow during route
calculation now; this could happen in theory once we wumbo.

It fixes a thinko when we print out routehints, too: we want to print
them out literally, not print out the effect they have on fees (which
is in the route, which we also print).

This ABI change doesn't need a CHANGELOG, since paystatus is new since
release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
cc95a56544 pylightning: handle msat fields in JSON more appropriately.
Little point having users handle the postfixes manually, this
translates them, and also allows Millisatoshi to be used wherever an
'int' would be previously.

There are also helpers to create the formatting in a way c-lightning's
JSONRPC will accept.

All standard arithmetic operations with integers work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
83adb94583 lightningd and routing: use struct amount_msat.
We use it in route_hop, and paper over it in the JSON APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
3ba544bfde common/bolt11: use struct amount_msat
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Christian Decker
5d9d5ceed1 pytest: Test the new peer_connected hook with a reject plugin
This plugin just rejects `node_id`s it gets told about.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-20 15:37:59 +01:00
Rusty Russell
7a1d13a77e pytest: make test_pay_direct more robust.
We had occasional failures, because the fuzz could overwhelm the difference
in routes.  Increasing the amount to 2,000,000 millisatoshis makes the
riskfactor 53msat (2000000 * 14 * 10 / 5259600) which is always greater
than the worst-case fuzz of 5% on the fee of 1002msat.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-18 15:32:22 +01:00
Rusty Russell
cb6a97152e pytest: fix race in test_pay_direct.
I got a spurious failure because the final node gave a CLTV error and
so it decided to use a different channel.  It should probably handle
this corner case better, but meanwhile make the test robust.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-18 15:32:22 +01:00
Rusty Russell
41be796e0e pytest: add simple test for --daemon.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-18 12:01:06 +01:00
Christian Decker
59fa47bf64 pytest: Mark the worst gossip offenders as developer-only tests
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-18 00:06:02 +00:00
Christian Decker
b7222531fe pytest: Stabilize the test_pay_direct test
It was waiting for a remote channel, but not for all the interesting
channels we want to check. It can sometimes happen that further away
channels are added before closer ones are added, depending on
propagation path, flush timers and bitcoind poll timers. This now just
checks for all channels, which also reduces the ambiguity of whether
we selected a path solely because we were lacking alternatives.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-08 01:08:44 +00:00
Rusty Russell
53c0a21d2c plugins: get usage from plugins (required unless deprecated_apis == True).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
da355284de jsonrpc: help, even for a single item, should be in an array.
This is what we do for every other can-be-single JSON API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
fbf2168902 pytest: don't time out on test_pay_direct !DEVELOPER
Travis timed out.

Waiting for three fundchannel commands depends on the bitcoind polling
interval (30 seconds), and then waiting for gossip propagation
requires two propagation intervals (120 seconds).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
3ae0c20026 getroute: change definition (and pay default) for riskfactor.
Up until now, riskfactor was useless due to implementation bugs, and
also the default setting is wrong (too low to have an effect on
reasonable payment scenarios).

Let's simplify the definition (by assuming that P(failure) of a node
is 1), to make it a simple percentage.  I examined the current network
fees to see what would work, and under this definition, a default of
10 seems reasonable (equivalent to 1000 under the old definition).

It is *this* change which finally fixes our test case!  The riskfactor
is now 40msat (1500000 * 14 * 10 / 5259600 = 39.9), comparable with
worst-case fuzz is 50msat (1001 * 0.05 = 50).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
706debf3d4 pytest: make test_pay_direct more effective.
The test sometimes passes: our routing logic always chooses between
the shorter of two equal-cost routes (because we compare best with <
not <=).

By adding another hop, we add more noise, and by making the alternate
route fee 0 we provide the worst case.

But to be fair, we make the amount of the payment ~50c (15,000,000
msat), and increase our cltv-delay to 14 and fee-base 1000 to match
mainnet.  The final patch shows the effect of this choice.

Otherwise our risk penalty is completely in the noise on
mainnet which has the vast majority of fees set at 1000msat + 1ppm.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
5d658012d6 plugins/pay: try without routehints first.
This is the direct cause of the failure of the original
test_pay_direct test and it makes sense: invoice routehints may not be
necessary, so try without them *first* rather than last.

We didn't mention the use of routehints in CHANGELOG at all yet, so
do that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
ZmnSCPxj
38a2f6c616 test_pay.py: Add test that we prefer direct route. 2019-02-06 18:39:52 +01:00
Rusty Russell
6bd1e46b25 invoice: don't allow creation of unpayable invoices.
Fixes: #2301
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-31 15:30:12 +01:00
Simon Vrouwe
10057c8335 openingd/json_fund_channel:
- result fundchannel command now depends on successful or failed broadcast of the funding tx
- failure returns error code FUNDING_BROADCAST_FAIL
- don't fail the channel when broadcast failed, but keep in CHANNELD_AWAITING_LOCKIN
- after fixing the initial broadcast failure, the user could manually rebroadcast the tx and
  keep the channel

openingd/opening_funder_finished:
- broadcast_tx callback function now handles both success and failure

jsonrpc: added error code FUNDING_BROADCAST_FAIL
manpage: added error code returned by fundchannel command

This makes the user more aware of broadcast failure, so it hopefully doesn't
try to broadcast new tx's that depend on its change_outputs. Some users have reported (see
issue #2171) a whole sequence of fundings failing, because each funding was using the change
output of the previous one, which would not confirm.
2019-01-29 04:50:01 +00:00
Rusty Russell
2d7c1ed0cf pytest: create proper mock failures.
We actually produce an invalid JSON error at the moment: bitcoin-cli
complains "JSON value is not an integer as expected" rather than returning
the given error.  Make our error a valid JSON RPC error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-29 04:50:01 +00:00
Christian Decker
27b66997da pytest: Temporarily disable test_htlcs_cltv_only_difference with VG
It is suddenly timing out a lot and is breaking master, so we
temporarily disable it until it is fixed.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00
Christian Decker
5a55972f1a pytest: Have bitcoind own its proxies
We were restarting the with the nodes before, which was causing some
port contention. This is more natural since `bitcoind` will take care
of terminating all proxies it returned.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00