Commit Graph

4815 Commits

Author SHA1 Message Date
Rusty Russell
ad2519a6f4 spelling: Check LockTime Verify.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-23 16:55:35 +02:00
Rusty Russell
e1b1abd328 doc: fix argument documentation for lightning-sendpay.
The parameter is 'payment_hash' not 'hash', and the 'description' parameter
wasn't documented at all.

Reported-by: @darosior
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-23 16:47:32 +02:00
Rusty Russell
f3329d6013 pylightning: add msatoshi optional sendpay param.
And there's a difference between no description and "" as a description:
for no description, listpayments doesn't show the field at all.  So fix
that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-23 16:47:32 +02:00
Rusty Russell
7eec2253e9 lightningd: fix up compile errors from bad merge.
The new listforwards was old-style, and the new API was merged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-20 03:34:58 +00:00
Rusty Russell
0dcd66880c Rename struct json_result to struct json_stream (RENAMEONLY)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
66dcba099d gossipd: hand raw pubkeys in getnodes and getchannels entries.
We spend quite a bit of time in libsecp256k1 moving them to and from
DER encoding.  With a bit of care, we can transfer the raw bytes from
gossipd and manually decode them so a malformed one can't make us
abort().

Before:
	real	0m0.629000-0.695000(0.64985+/-0.019)s

After:
	real	0m0.359000-0.433000(0.37645+/-0.023)s

At this point, the main issues are 11% of time spent in ccan/io's
backend_wake (I tried using a hash table there, but that actually makes
the small-number-of-fds case slower), and 65% of gossipd's time is
in marshalling the response (all those tal_resize add up!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
c403415caa lightningd: format JSON directly into json connection membuf.
My test case is a mainnet gossip store with 22107 channels, and
time to do `lightning-cli listchannels`:

Before: `lightning-cli listchannels` DEVELOPER=0
	real	0m1.303000-1.324000(1.3114+/-0.0091)s

After:
	real	0m0.629000-0.695000(0.64985+/-0.019)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
03e7be5bce json_connection: keep pointer to the conn.
This is required for our next hack.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
f4a2c4f8bb jsonrpc: helper to move an existing io_write.
It's a very ugly one-liner; really ccan/io should have an io_replan
for this, but it would have to be written carefully as it makes
assumptions currently about plans not changing.  In this case, we know
it's in io_write, and we're just moving a pointer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
240abf6c46 common/io_lock: add helper to query if lock is taken.
Not just for debugging; we actually need to know if a write is active for
the coming hack.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
e46ce0fc84 jsonrpc: declare up front whether a response is success or fail.
Such an API is required for when we stream it directly.  Almost all our
handlers fit this pattern already, or nearly do.

We remove new_json_result() in favor of explicit json_stream_success()
and json_stream_fail(), but still allowing command_fail() if you just
want a simple all-in-one fail wrapper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
12adcda370 test/run-param: don't reach into json_result to get the string.
It's about to change, so refactor this first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
39f0dfb664 jsonrpc: use ccan/membuf instead of a string for our output buffer.
This isn't a big change, since we basically dump the entire JSON
resuly string into the membuf then write it out, but it's prep for the
next changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
e9fcd120f8 jsonrpc: fix reading of multiple commands.
We occasionaly had a travis hang in test_multirpc, and it's due to a
thinko in the prior patch: if a command completes immediately, it will
do the wake before we go to sleep.  That means we don't digest the
rest of the buffer until the next write.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
ce0bd7abd3 jsonrpc: only allow a single command at a time.
There's a DoS if we keep reading commands and don't insist the client
read the responses.

My initial implementation simply removed the io_duplex, but that
doesn't work if we want to inject notifications in the stream (as we
will eventually want to do), so we operate it as duplex but have each
side wake the other when it's done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
8238fe6acf common/json: only maintain array of nested types if DEVELOPER mode.
My test case is a mainnet gossip store with 22107 channels, and
time to do `lightning-cli listchannels`:

Before: `lightning-cli listchannels` DEVELOPER=0
	real	0m1.396000-1.409000(1.4022+/-0.005)s

After:
	real	0m1.307000-1.320000(1.3128+/-0.005)s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
20bed28759 jsonrpc: use empty flag instead of examining the tail of the JSON.
This is in preparation for streaming directly to the JSON RPC buffer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Rusty Russell
305795b01e common/json: move JSON creation routines into lightningd/
It's the only user of them, and it's going to get optimized.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

gossip.pydiff --git a/common/test/run-json.c b/common/test/run-json.c
index 956fdda35..db52d6b01 100644
2018-10-19 22:02:11 +00:00
Rusty Russell
71a40faae7 withdraw: fix incorrect error when we have an empty wallet.
This also highlights the danger of searching the logs: that error
appeared previously in the logs, so we didn't notice that the actual
withdraw call gave a different error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 22:02:11 +00:00
Christian Decker
f583e6bc58 pytest: Silence some more warnings about raw_strings in regexps 2018-10-19 22:01:22 +00:00
Christian Decker
f3b4dcb1f6 channel: Set the billboard to reconnecting when we restarted
Some people were alarmed that the state was set to "Loaded from
database" indefinitely. Saying that we are trying to reconnect may be
more informative.
2018-10-19 22:01:22 +00:00
Rusty Russell
31d8a5a9a4 pytest: cleanup test_forward_stats.
Use dev_ignore_htlcs to make sure HTLC gets stuck.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 21:58:27 +00:00
Rusty Russell
604638712b jsonrpc: Only show total fees collected in getinfo.
And use wallet_forward_status_in_db() everywhere in db code.
And clean up extra CHANGELOG.md entry (looks like rebase error?)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 21:58:27 +00:00
Christian Decker
5471e3c0fc json-rpc: Mention listforwards and stats in CHANGELOG 2018-10-19 21:58:27 +00:00
Christian Decker
6b7546b94d json-rpc: Rename getroutestats and move stats to getinfo
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Christian Decker
6d333f16cc wallet: Correctly handle forwards when channels or htlcs are deleted
The left join should make sure we still get the results but
referencing the fields and/or attempting to write them to the JSON-RPC
result will cause unforeseen problems. So just omit if we forgot
something.
2018-10-19 21:58:27 +00:00
Christian Decker
facd7d16aa json-rpc: Add listforwardings command 2018-10-19 21:58:27 +00:00
Christian Decker
6efdd5275c pytest: Test that we record forwardings correctly
Adapts the `test_forward_stats` test to include checks for the
`forwarded_payments` table. Will add checks for the `listforwardings`
RPC call next.

Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Christian Decker
4b4c549c9d htlcs: Wire up the forward statistics on HTLC updates
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Christian Decker
5b924a7eb7 wallet: Add primitives to add forwarded payments to the DB
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Christian Decker
07ebc525e9 wallet: Add new forwards table to keep track of forwarded payments
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Amin Shah Gilani
6d2e5bace3 Copy-edit README.md
Fix a few typos and grammatical errors in the README file.
2018-10-19 14:52:19 +02:00
Rusty Russell
09b33015c4 connectd: give user a hint when wrong key is used.
When the wrong key is used, the remote end simply hangs up.

We used to get a random errno, which tends to be "Operation now in progress."
Now it's defined to be 0, detect and provide a better error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 00:53:00 +00:00
Rusty Russell
cd8446f081 ccan: update to get new ccan/io which sets errno to 0 on EOF.
We get structeq and htable updates we don't need for free.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 00:53:00 +00:00
Rusty Russell
acc01e6436 ccan: update.
This was from a different series, so I just cherry-picked it.

It adds ccan/membuf as a depenency of ccan/rbuf, though we don't use
it directly yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-19 00:53:00 +00:00
Rusty Russell
6af8f29392 bolt11: better message when you try to 'lightning-cli pay' an onchain addr.
Give a clear error at the beginning if it's not bolt11 payment,
rather than falling foul of other checks.

This will work at least until some altcoin adapts the 'ln' prefix :)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-17 22:59:20 +00:00
Rusty Russell
e2f4359c08 CHANGELOG: Fix up Added order, add note about htlc_maximum_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-17 22:57:06 +00:00
Rusty Russell
81823e5fed CHANGELOG: add entry for help updates.
This covers both cbde3e20f7 which added
the parameter names, and d23a0e8adc which
added fallback for missing man page entries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-17 22:57:06 +00:00
Rusty Russell
3e3543a12c CHANGELOG: note autotor port fix.
This was fixed in d5c3626fa73967ff59d60b36c8ec21394f357f9d; we're still
getting use to CHANGELOG.md.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-17 22:57:06 +00:00
Rusty Russell
6ff7beab1f CHANGELOG: note gossip fix 0855422110
This was the one causing us to spam old gossip onto the network!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-17 22:57:06 +00:00
Rusty Russell
bbc36a7bec gossipd: update node announcement even if we change within a second.
Usually Travis triggers corner cases because it's so slow, but this
time the moons aligned, and it managed to fail test_node_reannounce
because it generated the updated node_announcement with the same
timestamp as the old one.

This is because we only updated "last_announce_timestamp" when
we generated the announcement, not when we got it off the wire or
loaded it from the gossip store.

The fix is to ask the routing code what the latest timestamp is;
we could still generate a clashing timestamp if (1) the gossip store
is lost, and (2) we restart within one second.  Hard to care.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-16 04:24:03 +00:00
lisa neigut
66ca2a333f channeld: use u64 fo htlc_minimum_msat
As per spec.
2018-10-16 03:32:27 +00:00
lisa neigut
0ae1d03513 BOLT7: broadcast htlc_maximum_msat in `channel_update s
Have c-lightning nodes send out the largest value for
`htlc_maximum_msat` that makes sense, ie the lesser of
the peer's max_inflight_htlc value or the total channel
capacity minus the total channel reserve.
2018-10-16 03:32:27 +00:00
Christian Decker
e904fcc00c docker: Update docker images and docs to non-CVE-2018-17144 bitcoind
While not strictly necessary it's certainly a good idea to test
against the latest one and not encourage users to use old versions.

Reported-by: Jonas Nick <@jonasnick>
Signed-off-by: Christian Decker <@cdecker>
2018-10-15 23:05:25 +00:00
Rusty Russell
afac01380d gossipd: don't initialize broadcast interval, make field name explicit.
We initialize it to 30 seconds, but it's *always* overridden by the
gossip_init message (and usually to 60 seconds, so it's doubly
misleading).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-15 23:04:17 +00:00
Rusty Russell
3991425111 gossipd: don't accept forwarding short_channel_ids we don't own.
Gossipd provided a generic "get endpoints of this scid" and we only
use it in one place: to look up htlc forwards.  But lightningd just
assumed that one would be us.

Instead, provide a simpler API which only returns the peer node
if any, and now we handle it much more gracefully.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-15 23:04:17 +00:00
Rusty Russell
030fe1ce53 gossipd: don't expose private channels for routeboost.
We don't create unannouncable channels, but other implementations can.
Not only is it rude to expose these via invoices, it's probably not
useable anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-15 23:04:17 +00:00
Christian Decker
d0de7b49b2 libwally: Update to latest master to fix some compilation issues
Signed-off-by: Christian Decker <@cdecker>
2018-10-15 00:39:33 +00:00
Christian Decker
a44491fff0 benchmark: Add two benchmarks for forwarded payments
We weren't benchmarking them, so do that now.
2018-10-13 10:58:53 +00:00
Christian Decker
81f5b89fe3 wallet: Actually load wallet stats when asked to
The call to `sqlite3_step` is actually needed, otherwise we'll always
get the default values for all types.
2018-10-13 10:58:53 +00:00