Commit Graph

1328 Commits

Author SHA1 Message Date
Christian Decker
ee25547576 Re-enable builds with DEVELOPER=0
Two changes:
 - Fixed the function signature of noleak_ to match in both
   configurations
 - Added memleak.o to linker for tests

Generating the stubs for the unit tests doesn't really work since the
stubs are checked in an differ between the two configurations, so
adding memleak to the linker fixes that, by not requiring stubs to be
generated in the first place.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 12:43:10 +01:00
Rusty Russell
8c4eecc089 sphinx: fix leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
795a03e8c4 fromwire_bitcoin_tx: quieten leak reporting.
We create a temporary tx which is a child of the real tx, for simplicity of
marshalling.  That's OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
20fcb5d6ef memleak: allow marking of entire tal trees to be noleak.
In particular, the pay command attaches all kinds of stuff to the
struct command; they're not really a leak, since commands expire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
7a8532830f memleak: track reallocs and frees in notleak()
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
70d01b22e1 memleak: add second exclude arg to exclude current commands' jcon.
This is not a child of cmd, since they have independent lifetimes, but
we don't want to noleak them all, since it's only the one currently in
progress (and its children) that we want to exclude.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
19b1b35d31 memleak: explicitly exclude ccan/io allocation.
It's a single, full-lifetime allocation; make sure we only exclude that
one, though this is fragile: tests will break if it moves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
dfc132b2fe memleak: add backtrace to allocations.
We use the tal notifiers to attach a `backtrace` object on every
allocation.

This also means moving backtrace_state from log.c into lightningd.c, so
we can hand it to memleak_init().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
c956d9f5eb lightningd: tal memleak detection, dev-memleak command.
This is a primitive mark-and-sweep-style garbage detector.  The core is
in common/ for later use by subdaemons, but for now it's just lightningd.
We initialize it before most other allocations.

We walk the tal tree to get all the pointers, then search the `ld`
object for those pointers, recursing down.  Some specific helpers are
required for hashtables (which stash bits in the unused pointer bits,
so won't be found).

There's `notleak()` for annotating things that aren't leaks: things
like globals and timers, and other semi-transients.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Rusty Russell
0cab9d1dd1 daemon_conn: helper to release daemon_conn.
We'll want this for the next change, where gossipd migrates remote peers
back to local ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
afea2520ba wireaddr: marshal empty address properly.
On unmarshal, we stop unmarshaling on a 0 (ADDR_TYPE_PADDING) type.  So
we should also stop marshaling in that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
b00525b8c8 options: don't leak memory when people override options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-13 22:46:10 +01:00
ZmnSCPxj
5fd74f9933 Change withdraw_tx to accept scriptpubkey rather than plain address. 2017-12-13 03:10:04 +00:00
Rusty Russell
0610f66c34 bolt11: handle r value fee spec change.
We don't use it yet, but now we'll decode correctly.

See: https://github.com/lightningnetwork/lightning-rfc/pull/317
lightning-rfc commit: ef053c09431442697ab46e83f9d3f86e3510a18e

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-12 11:45:44 +01:00
practicalswift
61c47c09d0 Fix typos 2017-12-08 13:07:20 +01:00
Christian Decker
9ba99d2b2d hsm: Cleanup after merging control and client libraries
Change all calls to use the correct serialization and deserialization
functions, include the correct headers and remove the control
messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Rusty Russell
02411b04e4 channel: remove awaiting_revoke_and_ack flag.
We can tell this more generically because the count of revocations
received != count of commitments sent.  This is the correct condition
which allows us to restore the test we had to eliminate in
c3cb7f1c85.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:52:18 +01:00
Rusty Russell
ed60e9ee57 close_tx: use version 2.
As per the pending spec change: the other implementations use version 2,
and the current best practice advice (from maaku) is to use the latest
defined tx version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-01 18:30:57 +01:00
Rusty Russell
af7e6158af Makefile: clean needs to do more, distclean should remove everything.
I checked this with git status --ignored after a full build and 'make distclean'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:29:03 +01:00
Christian Decker
9bcb01e710 fixup! common/test: add spec tests for bolt11. 2017-11-24 13:22:18 +01:00
Rusty Russell
ae6faf7f9c common/test: add spec tests for bolt11.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell
60c30214c1 bolt11: fix encoding of x and c fields.
Fixes: #374
Reported-by: Nadav Ivgi
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell
45f5bb7fac bolt11: move to common/ and sign via callback.
JSON stuff is moved to lightningd/invoice.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
Rusty Russell
78d0a5e840 bech32: move to common/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:22:18 +01:00
ueno
4312deddfa typo 2017-11-23 23:10:08 +01:00
Rusty Russell
b836b452dc feerate: keep feerates separately for each side.
When we support changing them, they can be different during the transition.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
f1e4cad9d4 feerate: use u32 everywhere.
The wire protocol uses this, in the assumption that we'll never see feerates
in excess of 4294967 satoshi per kiloweight.

So let's use that consistently internally as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-23 12:40:32 +01:00
Rusty Russell
0a131c6389 channel: use flag to indicate we're awaiting revoke_and_ack.
We currently scan through HTLCs: this isn't enough if we've only got a
feechange in the commitment, so use a flag (but keep both for now for
debugging).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell
c328a76438 channeld: use flags to track whether changes are pending.
This is required when we have non-HTLC changes (ie. fees).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell
7ece90038b derive_basepoints: actually derive a separate HTLC key.
This means we'll find anywhere still using the payment key,
even though we still expose the private payment key to channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell
4db460903a htlc_tx: wire up the htlc points.
All the callers need to pass it in: currently channeld and openingd just
fake it by copying the payment point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell
fe5614a489 basepoints/secrets: add htlc entry
Currently derive_basepoints just sets it to match the payment point/secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell
5502a19d1e json: reject incoming JSON which has any unusual characters in tokens.
ie. non-printable, quotes or escapes.  We allow these outside tokens
(we expect tabs and \n for example).

This is a big hammer, but if someone really wants we can add support
later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell
112ae0d0f5 common/test/run-json: test JSON escaping.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell
ab634dfcdd common/test_sphinx: change to standard run- format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell
8f17effeeb json: just blatt weird characters in string.
Don't try to escape them.  It's whack-a-mole and they shouldn't do it anyway.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
a9002eac52 Update to latest BOLT.
And nail "make check-source" to that specific version (which is a commit id,
not a branch name, so needs a different syntax for git).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
02c1d10c9f json: escape strings we output in JSON.
We're going to output description strings, which are untrusted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
753f15f503 common/hash_u5: routines to hash 5 bit values.
Needed for creating/verifying signatures on bolt11 invoices.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
a7d6326bef type_to_string: format wireaddr.
Good for printing, and removes some code from peer_control.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
78cd25d620 ipaddr: rename to wireaddr.
In future it will have TOR support, so the name will be awkward.

We collect the to/fromwire functions in common/wireaddr.c, and the
parsing functions in lightningd/netaddress.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
bd1cac34ce netaddr: remove.
We use ipaddr everywhere now, so we can remove this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
ac92138603 common: remove unused assert() headers.
Auditing for assert/abort in common/ code used by lightningd, this is all
that showed up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell
81db5896e1 common/json: remove asserts() which may trigger from user input.
They don't currently, since callers check, but be safe.  In addition,
handle NULL returns from these in the bitcoind code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell
3c6eec87e3 Add DEVELOPER flag, set by default.
This is a bit messier than I'd like, but we want to clearly remove all
dev code (not just have it uncalled), so we remove fields and functions
altogether rather than stub them out.  This means we put #ifdefs in callers
in some places, but at least it's explicit.

We still run tests, but only a subset, and we run with NO_VALGRIND under
Travis to avoid increasing test times too much.

See-also: #176
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:53:09 +02:00
Rusty Russell
4e14185961 cryptomsg: add helpers to determine if we're partway through msg read/write.
For message read, we do it as header then body, so we can have
io_plan_in_started(conn) false, but we're between header and body.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 18:34:35 +02:00
Rusty Russell
74e684cc0d is_all_channels: rename to channel_id_is_all
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-22 16:24:10 +02:00
Rusty Russell
2273ce783e dev_disconnect: support multiple disconnects in the same daemon.
We currently assume the daemon gives up; gossipd won't, and we want to
test it there too.

This reveals a bug (returning io_close() is bad if the call is to
duplex()), and breaks a test which now continues after dropping a
packet..

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
6ceec17943 dev_disconnect: make commit suppression a "-nocommit" modifier.
Useful if we want to drop & suppress, for example.  We change '=' to mean
do nothing to the packet.

We use this to clean up the test_reconnect_sender_add test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
8430e33f3b common/status: add status_tracev() for making status wrappers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
9b589fb5ba common/wire_error: helpers to create/parse WIRE_ERROR messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
2394c9a2e7 crypto_state: move to its own file.
In particular, the main daemon needs to pass it about (marshal/unmarshal)
but it won't need to actually use it after the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
0969626918 close_tx: make version 1, not version 2.
Fixes: #311
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-18 13:00:29 +02:00
Rusty Russell
f32e0b35ad Revert "Send/receive cltv_expiry_delta in open/accept channel."
This reverts commit 18e3f9820f.
2017-10-11 11:54:50 +02:00
Rusty Russell
18e3f9820f Send/receive cltv_expiry_delta in open/accept channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Rusty Russell
e137e2527f Update BOLT references with typo fixes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Christian Decker
3dda72c885 db: Create table for HTLCs
Also added a small warning to one of the used enums not to reorder or
insert values. They'd break the update path.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-10-09 11:51:13 +10:30
Rusty Russell
3404c393b7 common/io_debug: replacement for ccan/io's poll which does sanity checks.
For now we just check for outstanding take() or tal_tmpctx().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
e587ec3bd3 tal_tmpctx: keep information around so we can find leaks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
7200002773 tests: detect and close leaks in unit tests.
Fixes: #288
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
ce160d9b17 lightnind: _ dev-disconnect argument to suppress commit timer.
Required for catching daemon in exact state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
6ef64cd52b script: make "sig_and_empty" more generic, make htlc tx witness fns clearer.
For non-delayed HTLC success spends, we have a similar pattern ("<sig>
<preimage> <wscript>") so a we want to use the same function.

The other routines don't say "witness" in them, and should.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
9123d6d45e type_to_string: support preimage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
86313383a7 type_to_string: support ripemd160.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
a21e31b2b2 subdaemons: catch backtrace on fatal signals.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell
ef28b6112c status: use common status codes for all the failures.
This change is really to allow us to have a --dev-fail-on-subdaemon-fail option
so we can handle failures from subdaemons generically.

It also neatens handling so we can have an explicit callback for "peer
did something wrong" (which matters if we want to close the channel in
that case).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell
4f6ae3209b common/test/test_sphinx: build fixes.
It isn't currently included in make check, so fix it to build and do that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-11 14:29:20 +02:00
Christian Decker
a9f02a8985 sphinx: Remove annoying warning when MACs do not match
This was printing to stderr during the tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-08 17:02:04 +09:30
Christian Decker
a44f893838 sphinx: Switch to version 0x00 like the spec says
We were using the wrong version, oops.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-08 17:02:04 +09:30
Rusty Russell
5889ad5fc4 dev-disconnect: add blackhole option.
To reproduce the next bug, I had to ensure that one node keeps thinking it's
disconnected, then the other node reconnects, then the first node realizes
it's disconnected.

This code does that, adding a '0' dev-disconnect modifier.  That means
we fork off a process which (due to pipebuf) will accept a little
data, but when the dev_disconnect file is truncated (a hacky, but
effective, signalling mechanism) will exit, as if the socket finally
realized it's not connected any more.

The python tests hang waiting for the daemon to terminate if you leave
the blackhole around; to give a clue as to what's happening in this
case I moved the log dump to before killing the daemon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-06 19:11:21 +02:00
Rusty Russell
bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
739b163f8b Makefiles: simplify dependencies.
Gather all binaries and objects and make the depend on external
requirements and common headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
f42f34b82d external: new subdirectory for all external libraries and submodules.
You will want to 'make distclean' after this.

I also removed libsecp; we use the one in in libwally anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
8c22bd9ee1 headers: fix up header idempotent lines.
For future reference, done via:
	for f in `find wire/ bitcoin/ common/ lightningd -name '*.h' ! -name 'gen*'`; do ID=`echo -n LIGHTNING/$f | tr 'a-z' 'A-Z' | tr -cs 'A-Z0-9' _`; sed 's/^#\(ifndef\|define\) .*_H$/#\1 '$ID/ < $f | sed 's,#endif /..*_H ./$,#endif /* '$ID' */,' | bagto $f; done

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
a37c165cb9 common: move some files out of lightningd/
Basically all files shared by different daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
8375857116 common: absorb remaining files from daemon/
Also, we split the more sophisticated json_add helpers to avoid pulling in
everything into lightning-cli, and unify the routines to print struct
short_channel_id (it's ':',  not '/' too).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
a3c51168db routing: use status() instead of log-to-nowhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
85ff95e829 common: new directory for any shared objects.
To avoid everything pulling in HTLCs stuff to the opening daemon, we
split the channel and commit_tx routines into initial_channel and
initial_commit_tx (no HTLC support) and move full HTLC supporting versions
into channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00