Commit Graph

5235 Commits

Author SHA1 Message Date
Conor Scott
0535cbbc96 [rpc] Add funding allocation to listpeers command 2019-01-16 11:50:52 -08:00
Conor Scott
3d270fcca6 Skip README.md when reading in plugins dir 2019-01-15 19:40:12 +00:00
nicolas.dorier
a565915d08 bitcoind: allow "getblock" to fail for txout lookup.
Apparently on pruned nodes it sometimes gives exit status 1?
2019-01-15 19:39:15 +00:00
Rusty Russell
0ba547ee10 gossipd: handle overflowing query properly (avoid slow 100% CPU reports)
Don't do this:
  (gdb) bt
  #0  0x00007f37ae667c40 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #1  0x00007f37ae668b38 in ?? () from /lib/x86_64-linux-gnu/libz.so.1
  #2  0x00007f37ae669907 in deflate () from /lib/x86_64-linux-gnu/libz.so.1
  #3  0x00007f37ae674c65 in compress2 () from /lib/x86_64-linux-gnu/libz.so.1
  #4  0x000000000040cfe3 in zencode_scids (ctx=0xc1f118, scids=0x2599bc49 "\a\325{", len=176320) at gossipd/gossipd.c:218
  #5  0x000000000040d0b3 in encode_short_channel_ids_end (encoded=0x7fff8f98d9f0, max_bytes=65490) at gossipd/gossipd.c:236
  #6  0x000000000040dd28 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=8) at gossipd/gossipd.c:576
  #7  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290511, number_of_blocks=16) at gossipd/gossipd.c:595
  #8  0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=32) at gossipd/gossipd.c:596
  #9  0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290495, number_of_blocks=64) at gossipd/gossipd.c:595
  #10 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=128) at gossipd/gossipd.c:596
  #11 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=256) at gossipd/gossipd.c:595
  #12 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=512) at gossipd/gossipd.c:595
  #13 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17290431, number_of_blocks=1024) at gossipd/gossipd.c:595
  #14 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2047) at gossipd/gossipd.c:596
  #15 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4095) at gossipd/gossipd.c:595
  #16 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8191) at gossipd/gossipd.c:595
  #17 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16382) at gossipd/gossipd.c:595
  #18 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=32764) at gossipd/gossipd.c:595
  #19 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=65528) at gossipd/gossipd.c:595
  #20 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=131056) at gossipd/gossipd.c:595
  #21 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=262112) at gossipd/gossipd.c:595
  #22 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=524225) at gossipd/gossipd.c:595
  #23 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=1048450) at gossipd/gossipd.c:595
  #24 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=2096900) at gossipd/gossipd.c:595
  #25 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=4193801) at gossipd/gossipd.c:595
  #26 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=8387603) at gossipd/gossipd.c:595
  #27 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=17289408, number_of_blocks=16775207) at gossipd/gossipd.c:595
  #28 0x000000000040ddee in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=33550414) at gossipd/gossipd.c:596
  #29 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=67100829) at gossipd/gossipd.c:595
  #30 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=134201659) at gossipd/gossipd.c:595
  #31 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=268403318) at gossipd/gossipd.c:595
  #32 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=536806636) at gossipd/gossipd.c:595
  #33 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=1073613273) at gossipd/gossipd.c:595
  #34 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=2147226547) at gossipd/gossipd.c:595
  #35 0x000000000040ddc6 in queue_channel_ranges (peer=0x3868fc8, first_blocknum=514201, number_of_blocks=4294453094) at gossipd/gossipd.c:595
  #36 0x000000000040df26 in handle_query_channel_range (peer=0x3868fc8, msg=0x37e0678 "\001\ao\342\214\n\266\361\263r\301\246\242F\256c\367O\223\036\203e\341Z\b\234h\326\031") at gossipd/gossipd.c:625

The cause was that converting a block number to an scid truncates it
at 24 bits.  When we look through the index from (truncated number) to
(real end number) we get every channel, which is too large to encode,
so we iterate again.

This fixes both that problem, and also the issue that we'd end up
dividing into many empty sections until we get to the highest block
number.  Instead, we just tack the empty blocks on to then end of the
final query.

(My initial version requested 0xFFFFFFFE blocks, but the dev code
which records what blocks were returned can't make a bitmap that big
on 32 bit).

Reported-by: George Vaccaro
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell
ba41d6e3df pytest: failing test for overflow in query_channel_range
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell
52750f2dcc pytest: tighten the query_channel_range test.
Make the two channels adjacent, and specify exactly the number of
divide-and-conquer steps there are.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell
4c57bf002e pytest: neaten fund_channel scid extraction.
We used to have a bug where decoderawtransaction would fail, fixed in
fedcfd661 (pytest: hand 'True' to decoderawtransaction so it doesn't
get confused.).

So we can remove the fallback decode, and might as well extract the
ugliness into a helper function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 11:34:45 -08:00
Rusty Russell
a00c357854 JSON: remove redundant word "channel" from direction fields.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
9f1f79587e short_channel_id_dir: new primitive for one direction of short_channel_id
Currently only used by gossipd for channel elimination.

Also print them in canonical form (/[01]), so tests need to be
changed.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
80753bfbd5 Feedback from @niftynei.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
dc2ee9639b listchannels: allow source arg to list channels by their source node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
de682f5806 json_getroute: don't return generic error.
We use the PAY error code here, but it's appropriate (otherwise the
pay command simply has to substitute it, which seems silly).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
5b3abd80b1 pytest: don't skip valgrinding plugins.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
358b7fda91 getroute: allow caller to specify maximum hops.
This is required for routeboost.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
3d016e7249 getroute: allow array of channels to exclude.
The pay plugin will use this, rather than the current "suppress for 90 second" hacks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
599ec5efbe gossipd: allow an array of excluded channels for getroute_request.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
8738940a8f listpeers: show channel direction for each outgoing channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
be64dd84ca waitsendpay: indicate which channel direction the error was.
You can figure this yourself by knowing the route, but it's better to report
it directly here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
c0cfddfa95 test/run-bench-find_route: fix so it runs properly.
We didn't populate the channels properly so it always failed.

Additionally, somewhere along the line we kept using the single scid
so we only created one channel.

Also, the next patch will start comparing the pubkeys, so make valid
ones: use an array so we don't affect the benchmark too much.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
2a8a8c64e9 invoice: add DEVELOPER-only optional dev-routes param.
This lets us test explicit routes which are more complex than the ones
we currently generate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
0bcad1289e bolt11: fix encoding of routes of length > 1.
We don't do this yet, so it went unnoticed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
1567238dd9 invoice: option to expose/not-expose private channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
fe4a600bc7 routeboost: don't use channels to dead-end nodes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
547d6ab878 routeboost: expose private channel in invoice iff we have no public ones.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
f321b1d35f getroute: remove seed arg, document fromid, make default fuzzpercent match docs.
seed isn't very useful at this level: I've left it in routing.c
because it might be useful for detailed testing.  Pretty sure it's unused,
so I simply removed it.

The fuzzpercent is documented to default at 5%, but actually was 75%.
Fix that too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
e65b680807 json: move bitcoin/lightning specific helpers into common/json_helpers.
We don't need them in common/json, since lightning-cli doesn't need these,
but plugins want them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
ea7e13b5a7 lightningd: fix leak when next peer is unknown.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
c75f9f4318 devtools/bolt11-cli: print min_final_cltv_expiry.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
26dda57cc0 utils: make tal_arr_expand safer.
Christian and I both unwittingly used it in form:

	*tal_arr_expand(&x) = tal(x, ...)

Since '=' isn't a sequence point, the compiler can (and does!) cache
the value of x, handing it to tal *after* tal_arr_expand() moves it
due to tal_resize().

The new version is somewhat less convenient to use, but doesn't have
this problem, since the assignment is always evaluated after the
resize.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Christian Decker
a5ed98a2ea misc: Add a bolt quote to the short_channel_id
[ Formatting fixed for make check-source to find, punctutation fixed --RR ]
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 03:50:27 +00:00
Christian Decker
659a26ea5a misc: Update short_channel_id representation to use 'x' separators
Reported-by: Alex Bosworth <@alexbosworth>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-01-15 03:50:27 +00:00
Rusty Russell
d3ac2503d6 CHANGELOG.md: plugins FTW
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 02:52:13 +00:00
Rusty Russell
61420bf79b configure: (mostly) revert b7a56f0531
Plugins are a first-class citizen again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 02:52:13 +00:00
Christian Decker
94eb2620dc bolt: Updated the BOLT specification to the latest version
This is mainly just copying over the copy-editing from the
lightning-rfc repository.

[ Split to just perform changes after the UNKNOWN_PAYMENT_HASH change --RR ]

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 02:19:56 +00:00
Rusty Russell
c3e96e058e bolt: Updated the BOLT specification to unify UNKNOWN_PAYMENT_HASH & INCORRECT_PAYMENT_AMOUNT
This is based on Christian's change, but removes all trace of the old codes.

I've proposed another spec change which removes this code altogether:
	https://github.com/lightningnetwork/lightning-rfc/pull/544

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 02:19:56 +00:00
Christian Decker
65054ae72e bolt: Updated the BOLT specification to a07dc3df3b4611989e3359f28f96c574f7822850
This is mainly just copying over the copy-editing from the
lightning-rfc repository.

[ Split to just perform changes prior to the UNKNOWN_PAYMENT_HASH change --RR ]

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 02:19:56 +00:00
Rusty Russell
0d5f0d79da build: allow building from source zip file.
Changes both how we construct it and how we deal with not having git.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 02:18:30 +00:00
Rusty Russell
75df22e9e2 doc/MAKING-RELEASES: clarifications based on previous release.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 02:18:30 +00:00
Conor Scott
a2413f47e5 [doc] rename plugins.md -> PLUGINS.md... because the rest were uppercase 2019-01-14 18:45:08 +01:00
Rene Pickhardt
6b5db38774 pylightning: Made rpc a public member of Plugin and sorted imports
After this code change people can use `plugin.rpc` from anywhere in
their plugin code this is much nicer than going this way:

```
@plugin.method("init")
def init(options, configuration, plugin):
    global rpc
    basedir = plugin.lightning_dir
    rpc_filename = plugin.rpc_filename
    path = os.path.join(basedir, rpc_filename)
    rpc = LightningRpc(path)
```

or similarly that way:
```
@plugin.method("init")
def init(options, configuration, plugin):
    global rpc
    basedir = configuration['lightning-dir']
    rpc_filename = configuration['rpc-file']
    path = os.path.join(basedir, rpc_filename)
    rpc = LightningRpc(path)
```

Also the imports have been sorted alphabetically

Co-authored-by: Rene Pickhardt <rene@rene-pickhardt.de>
Co-authored-by: Christian Decker <decker.christian@gmail.com>
2019-01-12 22:12:01 +01:00
Rusty Russell
14db874395 CHANGELOG.md: mention manpage fix.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-11 21:59:47 +00:00
Rusty Russell
ba31af5a75 CHANGELOG: Fix references, add new Unreleased section.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-11 21:59:47 +00:00
Rusty Russell
9e4d405ab6 v0.6.3: Release.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-11 15:31:34 +10:30
William Casarin
41468fb607 cli: source help from local docs when not installed
Teach `lightning-cli help` to look in the doc directory relative to
lightning-cli if it can't find the manpage in any man paths.

This makes `lightning-cli help` work even if clightning hasn't been installed
yet.

Signed-off-by: William Casarin <jb55@jb55.com>
2019-01-11 04:57:58 +00:00
Christian Decker
b7a56f0531 plugin: Gate the plugin subsystem with the --enable-plugins configure flag
Since we are planning to release a bug fix release, and the plugin
subsystem is not yet complete, it is better to make plugin support
opt-in while we continue testing.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-10 17:15:10 -08:00
William Casarin
a01342a66a make: fail build if git isn't present
git is needed to generate version information

Add a sanity check so that the build don't continue with an empty VERSION. This
is useful for sandboxed build where we might have forgot to include git.

Signed-off-by: William Casarin <jb55@jb55.com>
2019-01-10 14:33:22 -08:00
Rusty Russell
b5423775e2 Release of 0.6.3-rc1: hastened by 0-HTLC crash reports.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-09 05:05:16 +00:00
Conor Scott
2ee1061c16 [docs] Add missing 'level' description for listpeers manpage. 2019-01-08 15:35:17 -08:00
Michael Hawkins
22921f1ab1 Manpage for lightning-listchannels created 2019-01-08 23:02:17 +00:00
Rusty Russell
55e8634a1f channeld: don't fail channel on unknown code in update_fail_malformed_htlc.
We will probably not add another BADONION code, but this is safer.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-08 19:20:28 +01:00