Commit Graph

5183 Commits

Author SHA1 Message Date
Rusty Russell
a8d1588154 libplugin: provide usage in getmanifest.
Same method as lightningd: call the handlers up-front to get their param()
usage.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
c58a4b9b42 libplugin: put method name into command struct.
We'll need this for the usage map coming in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Conor Scott
6e4eae372d [pylightning] add args list to getmanifest
[ Shamelessly stolen by Rusty, tweaked parameter name ]
2019-02-07 20:33:50 +00:00
Rusty Russell
0c89fc5d70 pylightning: use different decoration for init msg.
The next patch wants to decorate the methods with a compulsory
'usage' option, which doesn't make sense for init.  So I wanted
to change the init to its own decoration.

Made-to-work-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
5770e0c700 jsonrpc: probe sites for usage information once, at start.
We store it in a strmap.  This means we call the jsonrpc handler earlier,
so all callers need to call param() before they do anything else; only
json_listaddrs and json_help needed fixing.

Plugins still use '[usage]' for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
e5c80f63d7 lightningd: add code to search strmaps for memleak detection.
Didn't put this in common/memleak because only lightningd currently needs it
(as of next patch).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
c506d42679 plugins: don't keep redundant jsonrpc pointer.
We have ld already, just use that in the one place we need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
dabdefefae jsonrpc: populate ld->jsonrpc ourselves, so we can use it.
Next patch will call commands to get usage inside jsonrpc_new(): to do
this it will need access to ld->jsonrpc, so we can't use the current
pattern.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
cc76416447 jsonrpc: use tal destructor to remove json commands when required.
This fixes a bug with a plugin duplicating an existing name
where we'd crash, too.

This doesn't work for builtins, which aren't tal objects, so
create a separate path for them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
da355284de jsonrpc: help, even for a single item, should be in an array.
This is what we do for every other can-be-single JSON API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
a1ebfc9e5e pylightning: help() accepts an argument.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 20:33:50 +00:00
Rusty Russell
0361f43ec3 ccan: update
Brings in fixes for closing stderr in parent for pipecmd (oops!)
and configurator fix (which we don't need yet)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-07 13:05:59 +01:00
fanquake
132d8460fc doc: skip building bitcoin-qt on macOS
Skip installing GUI dependancies and building bitcoin-qt.
2019-02-07 03:23:03 +00:00
Christian Decker
35e4227015 docker: Do not use the copy of a clone
This was failing the docker hub builds, since the git-config retains
an absolute path to the worktree location when cloning. Copying it
over from the host system means that this path now points to a
non-existent location, which then interfered with the submodule
initialization.

This fixes it by not using the copy directly, but rather it creates a
clean clone from the copied location, including a submodule init.

Signed-off-by: Christian Decker <@cdecker>
2019-02-07 03:07:18 +00:00
fanquake
01c9cdec97 travis: fix cache directories syntax 2019-02-06 18:40:38 +01:00
Rusty Russell
fbf2168902 pytest: don't time out on test_pay_direct !DEVELOPER
Travis timed out.

Waiting for three fundchannel commands depends on the bitcoind polling
interval (30 seconds), and then waiting for gossip propagation
requires two propagation intervals (120 seconds).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
3ae0c20026 getroute: change definition (and pay default) for riskfactor.
Up until now, riskfactor was useless due to implementation bugs, and
also the default setting is wrong (too low to have an effect on
reasonable payment scenarios).

Let's simplify the definition (by assuming that P(failure) of a node
is 1), to make it a simple percentage.  I examined the current network
fees to see what would work, and under this definition, a default of
10 seems reasonable (equivalent to 1000 under the old definition).

It is *this* change which finally fixes our test case!  The riskfactor
is now 40msat (1500000 * 14 * 10 / 5259600 = 39.9), comparable with
worst-case fuzz is 50msat (1001 * 0.05 = 50).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
05f95b59c1 gossipd: take into account risk in final route comparison.
We were only comparing by total msatoshis.

Note, this *still* isn't sufficient to fix our indirect problem, as
our risk values are all 1 (the minimum):

	lightning_gossipd(25480): 2 hop solution: 1501990 + 2
	lightning_gossipd(25480): 3 hop solution: 1501971 + 3
	...
	lightning_gossipd(25480): => chose 3 hop solution

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
662bb0c565 gossipd: fix riskfactor passing.
We used a u16, and a 1000 multiplier, which meant we wrapped at
riskfactor 66.  We also never undid the multiplier, so we ended up
applying 1000x the riskfactor they specified.

This changes us to pass the riskfactor with a 1M multiplier.  The next
patch changes the definition of riskfactor to be more useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
706debf3d4 pytest: make test_pay_direct more effective.
The test sometimes passes: our routing logic always chooses between
the shorter of two equal-cost routes (because we compare best with <
not <=).

By adding another hop, we add more noise, and by making the alternate
route fee 0 we provide the worst case.

But to be fair, we make the amount of the payment ~50c (15,000,000
msat), and increase our cltv-delay to 14 and fee-base 1000 to match
mainnet.  The final patch shows the effect of this choice.

Otherwise our risk penalty is completely in the noise on
mainnet which has the vast majority of fees set at 1000msat + 1ppm.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
5d658012d6 plugins/pay: try without routehints first.
This is the direct cause of the failure of the original
test_pay_direct test and it makes sense: invoice routehints may not be
necessary, so try without them *first* rather than last.

We didn't mention the use of routehints in CHANGELOG at all yet, so
do that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
Rusty Russell
6a26b0c18d gossipd: increase randomness in route selection.
We have a seed, which is for (future!) unit testing consistency.  This
makes it change every time, so our pay_direct_test is more useful.

I tried restarting the noed around the loop, but it tended to fail
rebinding to the same port for some reason?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-06 18:39:52 +01:00
ZmnSCPxj
38a2f6c616 test_pay.py: Add test that we prefer direct route. 2019-02-06 18:39:52 +01:00
Christian Decker
7ac0f53da3 travis: Attempt to setup travis without docker and without sudo
This should speed up testing since it no longer requires virtualization.
2019-02-04 17:10:24 +01:00
Simon Vrouwe
35545f705f lightningd/json_withdraw:
Add the change output to owned_txfilter so its entry in db will
get a confirmation_height when detected in a block by filter_block_txs

before this commit, after a 'withdraw' command, 'listfunds' would
not show our change outputs as confirmed

Modified the log message in wallet_extract_owned_outputs to
append 'CONFIRMED' when it is called with a blockheight arg.
To make distinction between (1st call) when adding owned output to the
db and (2th call) when confirmed in block.
2019-02-04 12:52:57 +01:00
Christian Decker
7eaf5b55ff make: Add an option to compile with address sanitizer
Currently only works with `gcc` due to google/sanitizers#1028, so
configure makes sure we warn if clang with ASAN is attempted.

According to [my benchmarks][benchmarks] the performance degradation
is small enough to have it active always.

[benchmarks]: https://github.com/ElementsProject/lightning/issues/2277#issuecomment-455897417

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-03 23:50:18 +00:00
Rusty Russell
0da4054045 Makefile: fix make install to depend on plugins.
Otherwise a straight "make install" gives:
	install: cannot stat 'plugins/pay': No such file or directory
	make: *** [Makefile:482: install-program] Error 1

Fixes: #2288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-31 15:30:56 +01:00
Rusty Russell
6bd1e46b25 invoice: don't allow creation of unpayable invoices.
Fixes: #2301
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-31 15:30:12 +01:00
Rusty Russell
f1a837e091 CHANGELOG.md: document deadlock fix.
Useful if others hit it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-29 11:45:17 +01:00
Rusty Russell
c9a907cd71 common: handle peer input before gossipd input (for closingd, openingd)
Similar to the previous "handle peer input before gossip input", this
fixes similar potential deadlock for closingd and openingd which use
peer_or_gossip_sync_read.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-29 11:45:17 +01:00
Simon Vrouwe
6e4d9acac3 channeld: prioritize read from peer over (read from gossipd and) write to peer
This solves (or at least reduces probability of) a deadlock in channeld
when there is lot of gossip traffic, see issue #2286. That issue is
almost identical to #1943 (deadlock in openingd) and so is the fix.
2019-01-29 11:45:17 +01:00
Simon Vrouwe
10057c8335 openingd/json_fund_channel:
- result fundchannel command now depends on successful or failed broadcast of the funding tx
- failure returns error code FUNDING_BROADCAST_FAIL
- don't fail the channel when broadcast failed, but keep in CHANNELD_AWAITING_LOCKIN
- after fixing the initial broadcast failure, the user could manually rebroadcast the tx and
  keep the channel

openingd/opening_funder_finished:
- broadcast_tx callback function now handles both success and failure

jsonrpc: added error code FUNDING_BROADCAST_FAIL
manpage: added error code returned by fundchannel command

This makes the user more aware of broadcast failure, so it hopefully doesn't
try to broadcast new tx's that depend on its change_outputs. Some users have reported (see
issue #2171) a whole sequence of fundings failing, because each funding was using the change
output of the previous one, which would not confirm.
2019-01-29 04:50:01 +00:00
Rusty Russell
2d7c1ed0cf pytest: create proper mock failures.
We actually produce an invalid JSON error at the moment: bitcoin-cli
complains "JSON value is not an integer as expected" rather than returning
the given error.  Make our error a valid JSON RPC error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-29 04:50:01 +00:00
Christian Decker
27b66997da pytest: Temporarily disable test_htlcs_cltv_only_difference with VG
It is suddenly timing out a lot and is breaking master, so we
temporarily disable it until it is fixed.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00
Christian Decker
0266c33476 cleanup: Fix a typo in the sendpay manpage
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00
Christian Decker
5a55972f1a pytest: Have bitcoind own its proxies
We were restarting the with the nodes before, which was causing some
port contention. This is more natural since `bitcoind` will take care
of terminating all proxies it returned.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00
Christian Decker
f687262658 pytest: Update test dependencies to latest version
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00
Christian Decker
1c8b980985 pylightning: Handle empty log lines correctly
Logging an empty line (without newline character) would raise an
Exception due to out of bounds check.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00
Christian Decker
5d05694920 json-rpc: Remove double-quoting on errors in JSON-RPC
The use of `json_tok_full_len` and `json_tok_full` in addition to
single quotes will result in double quoting, which is really weird. I
opted to single quoting using `'` instead which does not need to be
escaped.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-29 00:50:09 +00:00
arowser
c029f2fb0f remove unused local var 2019-01-25 23:01:56 +01:00
Mark Beckwith
287af7b660 docs: fix getroute manpage rendering
My manpage viewer did not know what to do with curly braces,
so I switched them to quotes and it works fine.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2019-01-23 13:43:41 -08:00
Mark Beckwith
17b969383b Changed testnet faucet
The previous ones were no longer working.
This one currently does.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2019-01-23 13:36:06 -08:00
Rusty Russell
390117c9bb docs: document changes to waitsendpay command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell
f8ecd08721 pay: don't list dummy channel if error is from final hop.
List the final one instead; if there's an error from the node it
may actually make sense to blame that channel (ie. previous node
did something wrong).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell
4e6b8e13a4 lightningd/pay: simplify code significantly.
We no longer need a 'sendpay_result' structure, we can pass
appropriate parameter directly now they're simple calls.

Every waitsendpay command ends in tell_waiters_failed or
tell_waiters_success, which call sendpay_success or sendpay_fail on
all matching waiters.  These all return 'struct command_result *'.

In cases where the result is immediately known, we call
sendpay_success/sendpay_fail directly for the command.

This also adds a helpful 'failcodename' field to the JSON output.

[ This was four separate cleanup patches, but that contained much
redundancy and was even worse to review ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell
1c58351551 lightningd: hardcode callbacks again.
With only one caller, we don't need a callback pointer any more; we can simply
call the function.

This required some code shuffling, and I changed the callback function
arguments to be in a more natural order, now they're not used as
callbacks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell
a45a62aff6 lightningd: move pay internals back into pay.c
Now we don't have a second caller for these routines, we can move
them back into pay.c and make the functions static.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell
afab1f7b3c gossipd: handle onion errors internally.
As a general rule, lightningd shouldn't parse user packets.  We move the
parsing into gossipd, and have it respond only to permanent failures.

Note that we should *not* unconditionally remove a channel on
WIRE_INVALID_ONION_HMAC, as this can be triggered (and we do!) by
feeding sendpay a route with an incorrect pubkey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell
4eddf57fd9 gossipd: don't mark channels unroutable.
For transient failures, the pay plugin should simply exclude those
from route considerations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-23 22:08:08 +01:00
Rusty Russell
018a3f1d58 short_channel_id: make mk_short_channel_id return a failure.
We had a bug 0ba547ee10 caused by
short_channel_id overflow.  If we'd caught this, we'd have terminated
the peer instead of crashing, so add appropriate checks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-21 12:31:06 +01:00