Commit Graph

707 Commits

Author SHA1 Message Date
Christian Decker
285b8b4698 sphinx: Use libsecp256k1 to generate shared secrets
So far we did it on our own, but since the spec specifies that we use
the libsecp256k1 version anyway, we can remove our own implementation.
2017-01-16 11:08:36 +10:30
Christian Decker
679dec3e6a sphinx: Actually use the full pubkey in ECDH key generation
The spec says that we use the libsecp256k1 style ECDH, which uses the
full compressed pubkey from the scalar multiplication which is then
hashed. This is in contrast to the btcsuite implementation which was
only using the hashed X-coordinate.
2017-01-16 11:08:36 +10:30
Rusty Russell
6bf3c30a10 lightningd/Makefile: fix check-source
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-13 10:51:57 +10:30
Rusty Russell
73d07ce441 libsodium: use our local submodule.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-11 10:04:26 +10:30
Rusty Russell
e39d09d5a5 daemon/test/scripts: maek slightly more general for lightningd/lightningd
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell
015eb072b8 Makefile: split CORE_SRC into CORE_SRC, CORE_TX_SRC and CORE_PROTOBUF_SRC
With the lightningd daemon split, we don't need them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell
a0ac5c276e status: API for status reporting.
The API formalizes how daemons should report their statuses back to
the main lightningd.  It's a simple write API, which includes tracing
support (currently it always sends traces, later it could send iff
there's a failure, for example).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:24:20 +10:30
Rusty Russell
42f474af18 wire: make lengths of variable fields implied by tal_count()
This is a much nicer interface, and works better in practice too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:19:25 +10:30
Rusty Russell
7b1a4fc765 utils: add tal_hex() helper.
This is a shortcut when the data being dumped is a tal array.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:19:25 +10:30
Rusty Russell
4a233090ae log: rename struct log_record to struct log_book.
I think "log entry" when I see "log record", so this name is better.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:18:26 +10:30
Rusty Russell
350d8edb7e log: don't include dstate any more.
Before we had a global secp256k1_ctx we needed to hold this to print
out pubkeys, now it's completely orthogonal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:17:51 +10:30
Christian Decker
af43cc5e2c makefile: Be more verbose in check-daemon-headers
So far it was failing silently, now it diffs the Makefile state
against the directory listing. This also fixes a bug when the locale
was not set the sort order would not match.
2017-01-10 09:26:06 +10:30
Rusty Russell
b59fe5a2b6 daemon/Makefile: objects depend on wire headers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-06 13:23:50 +10:30
Christian Decker
96af89139e base58: Unittests need in-tree libbase58 2017-01-05 12:12:30 +10:30
Lucas Betschart
3e6bb958eb Add libbase58 as git submodule
Pointing to the latest release (0.1.4)
2017-01-05 12:11:18 +10:30
Rusty Russell
bdc4972df6 wire/gen_peer_wire_csv: update to latest spec #2
828eda61df5a7be27051c605f7808e4f690739e4, in particular, it has the
new address format for node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
d3bdb073b5 wire/gen_peer_wire_csv: update to latest spec #1
In particular, add features bitfields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
bdaa22e247 generate-wire.py: allow NULL len parameter from fromwire_*
It implies tal_count() gives the length. Great for almost all callers which
don't care if there are extra bytes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
e076d56709 generate-wire.py: include type bytes in towire/fromwire routines.
This removes some redundancy in creating messages, but also allows
a lazy form or parsing without explicitly checking the type.

A helper fromwire_peektype() is added to look up the type and handle
the too-short-for-type problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:21 +10:30
Rusty Russell
c864b28068 generate-wire.py: don't generate structures, hand in all values.
This is a bit more awkward for large structures, but avoids
indirection for the simpler ones (I copied the structures for the test
code, however).  We also remove explicit padding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
a08a2105ea generate-wire.py: generalize, move to tools.
We're going to want to use this for inter-daemon comms, so generalize it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
b7789bf065 Makefile: generalize whitespace check.
Spread to individual Makefiles, and include headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
0861ec33a5 check-source-bolt: generalize.
This way sub-Makefiles can add their own files to check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
c2cc164d6d daemon: disable old BOLT checks.
This is useful for the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
05feefbb8a test: move mockup script outside daemon/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
d30b67a3fe test: generalize update-mocks
So we can use it in other directories.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:20 +10:30
Rusty Russell
dec3b9d030 peer: don't ever set up listener on dynamic port.
Simplifies the logic somewhat.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:09:15 +10:30
Rusty Russell
12b30ab4fc jsonrpc: register base on linkage.
Other than being neater (no more global list to edit!), this lets the
new daemon and old daemon have their own separate routines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:08:15 +10:30
Rusty Russell
c7b69abdaa type_to_string: move formatting to appropriate files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:07:15 +10:30
Rusty Russell
f8eb454139 type_to_string: move pretty printing of types from log
It's not just useful for logging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:05:15 +10:30
Rusty Russell
756b2a0530 options: --help and --version are early args.
If they use these, we definitely don't want to create the config dir...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:04:15 +10:30
Rusty Russell
93471e06c1 options: return true if we created new directory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 14:03:15 +10:30
Rusty Russell
038ef0250a options: move option and config code into its own file.
For the moment, the new lightningd will share all this anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 13:22:29 +10:30
Rusty Russell
ca4d03aa2a lightningd_state: move invoices into separate structure.
It's all maintained by invoice.c, so make it private to that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-04 13:18:47 +10:30
Christian Decker
570003235e gossip: Fixing minor issues
Thanks @rustyrussell for the review ^^
2017-01-03 15:08:05 +10:30
Christian Decker
95bc8f6987 gossip: Removing origin from staggered broadcast
Keeping a pointer to the peer that initially sent us a message
could (actually will!) result in dangling pointers. Removing this
results in some additional messages, which will be discarded by the
recipient, so that should not be a problem.
2017-01-03 15:08:05 +10:30
Christian Decker
0ed23c46a9 gossip: Implemented routing table sync with newly connected peers 2017-01-03 15:08:05 +10:30
Christian Decker
5ce1f7c2f3 gossip: Set announce timeout to 5 hours 2017-01-03 15:08:05 +10:30
Christian Decker
71a18fdd2e gossip: Trigger announce on channel establishment
Since we ultimately want to have a very long re-announcement timeout
we better trigger an announcement upon channel state change to normal.
2017-01-03 15:08:05 +10:30
Christian Decker
e54c0adced gossip: Do not consider half-open connections for routes
Connections are in a half-open state after receiving the
`channel_announcement` and before the `channel_update` makes them
usable, so we need to ignore channels that are not yet fully open.
2017-01-03 15:08:05 +10:30
Christian Decker
611f4833c5 gossip: Implemented the staggered broadcast 2017-01-03 15:08:05 +10:30
Christian Decker
fca9bf1a16 gossip: Partially sign channel_announcements
We can already add our own signatures, will have to defer remote
signatures until we implement the channel setup messages from the spec.
2017-01-03 15:08:05 +10:30
Christian Decker
d93ce12afa gossip: Broadcasting and handling gossip messages 2017-01-03 15:08:05 +10:30
Christian Decker
e8d68757f6 gossip: Tracking tx index in topology
The txlocator was reporting the index in the purged array of
transactions instead of the index in the original block.
2017-01-03 15:08:05 +10:30
Christian Decker
57d5ae96a4 routing: Added channel_id and directions to connections
The gossip protocol spec refers to channels by their `channel_id` and
a direction. Furthermore, inbetween the `channel_announcement` and the
`channel_update` for either direction, the channel direction is in an
undefined state and cannot be used, so added the `half_add_connection`
function and an `active` flag to differentiate usable connections from
unusable ones.
2017-01-03 15:08:05 +10:30
Christian Decker
db481d881a proto: Added handling for nested packets 2017-01-03 15:08:05 +10:30
Rusty Russell
a44e4fb631 daemon/json, test/test_protocol: avoid gcc -O warnings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:49:51 +10:30
Rusty Russell
e109977df7 peer: don't print uninitialized commit_num.
Reported-by: Lucas Betschart <lucasbetschart@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:44:48 +10:30
Rusty Russell
2a68a984c3 peer: fix unassigned error packet on receiving non-ASCII error.
Reported-by: Lucas Betschart <lucasbetschart@gmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:42:24 +10:30
Rusty Russell
1b170c85bb Revert "Fix -Wsometimes-uninitialized and -Wuninitialized"
This reverts commit ff9cdda4fa.

These are real bugs, so I prefer to fix them explicitly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-15 11:41:31 +10:30
Lucas Betschart
900ab78d46 Fix -Wtautological-constant-out-of-range-compare 2016-12-14 08:47:21 +01:00
Lucas Betschart
ff9cdda4fa Fix -Wsometimes-uninitialized and -Wuninitialized 2016-12-14 08:45:40 +01:00
Lucas Betschart
c163fe04ac Fix -Wenum-conversion 2016-12-14 08:43:48 +01:00
Lucas Betschart
33d6cbe6bc Fix -Wgnu-variable-sized-type-not-at-end 2016-12-14 08:41:30 +01:00
Rusty Russell
45ec071e58 lightningd: remove whitespace failing make check-source.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-05 14:03:06 +10:30
Rusty Russell
36c8fc7ef8 lightningd: remove secpctx
Use the global in the few remaining places.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:13:27 +10:30
Rusty Russell
a4fdaab5b3 Use global secp256k1_ctx instead of passing it around.
If I'd known how large this patch would be (though trivial), I'd
have done it in parts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:12:58 +10:30
Rusty Russell
c938ebb5c0 utils: add a global secp, fix wire to use it.
This repairs make check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:11:06 +10:30
Rusty Russell
1f447688bc test: make generic driver for unit tests.
Not just in daemon.  This fixes 'make check' when it tries to
run 'make wire-tests'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-12-02 18:10:13 +10:30
Christian Decker
e40509c6c3 wire: Linking against wire objs 2016-12-01 13:50:02 +10:30
Rusty Russell
0ba93cb948 lightningd: don't listen at all if no port number set.
When we support the Milan protocol, we'll use a default port.  But
for now, don't listen at all unless a port is specified.

Fixes: #54
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 17:13:29 +10:30
Rusty Russell
fe14b3f4d9 Merge remote-tracking branch 'origin/pr/92'
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 17:05:42 +10:30
Rusty Russell
97a155bad5 Merge remote-tracking branch 'origin/pr/97'
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-18 14:28:22 +10:30
Christian Decker
42a33df42e irc: Checking for param count before accessing 2016-11-16 22:55:29 +01:00
Christian Decker
188b3c3e19
sphinx: Fixed a buffer overflow in hmac generation
Our HMACs are truncated to 20 byte, but sodium still generates 32 byte
HMACs and we were handing in a buffer that was too small, so we
overflowing the buffer by 12 bytes. This manifested itself only in the
32 bit variant because of different alignment in the 64bit version.

Fixes #94.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2016-11-16 14:41:19 +01:00
Christian Decker
b2a3e8010f Merge pull request #96 from ElementsProject/minor-sphinx-cleanups
Minor sphinx cleanups
2016-11-16 14:35:19 +01:00
Christian Decker
bf81f7af15 Merge pull request #93 from ElementsProject/fix-87-open-too-small-for-fee
json_connect: don't try to double-fail if amount insufficient.
2016-11-16 13:04:26 +01:00
Rusty Russell
41299b679c sphinx: process_onionpacket doesn't modify the packet.
Make it const.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:22:14 +10:30
Rusty Russell
423572190b sphinx: use assignment instead of memcpy where possible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:22:08 +10:30
Rusty Russell
6f9dedbe7f sphinx: remove redundant initialization
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:04:50 +10:30
Rusty Russell
8599d63256 sphinx: add brackets around constant definition
Always do this, otherwise "MACRO * foo" can have unexpected results.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 14:02:56 +10:30
Rusty Russell
5465b61784 json: fix 32-bit build.
Assume we have strtoull everywhere (it's C99 and was common before).

Fixes: #91
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-16 13:34:56 +10:30
Christian Decker
686ff6edef irc: Switched to LFNet.org IRC server
This is a temporary fix until we finish #88.
2016-11-14 23:19:32 +01:00
Rusty Russell
94adbd241d json_connect: don't try to double-fail if amount insufficient.
Closes: #87
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-14 11:46:09 +10:30
Christian Decker
34b4134cb0 bugfix: Removed spurious printf in peer.c
I must've left that on in. Fixes #89
2016-11-14 00:29:01 +01:00
Christian Decker
e5b44ff232 Merge pull request #85 from ElementsProject/whitespace
Whitespace
2016-11-11 16:40:23 +01:00
Christian Decker
9848b4ac3e travis-ci: Added travis-ci config
Added .travis.yml to get travis-ci to build and run tests for us.

In addition this fixes a flaky test due to the fact that when lightning2
connects to lightning3 and we tell lightning3 to restart, then
lightning2 will back-off its reconnection attempts, potentially causing
a timeout to trigger during tests. This was triggered by travis-ci
relatively consistently since the restart would take quite some
time. Now simply restarting them in reverse order and a small timeout
seems to fix this consistently.
2016-11-11 14:25:41 +01:00
Rusty Russell
f5c00deec7 Remove trailing whitespace from source.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-11 09:32:04 +10:30
Rusty Russell
716da21f85 chaintopology: reduce how far back we start on testnet.
When initially reading the blockchain, we start 100 back from the
current block, or at the first block with a funding transaction,
whichever is earlier.

This slows testing slightly, so use whatever the "forever" value is
(10 on testnet, still 100 on mainnet).

make check -j12 times:
	Before:	7m52.005s
	After:  6m31.896s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:53:48 +10:30
Rusty Russell
ce96812ef0 test.sh: split into multiple scripts.
This significantly reduces re-testing of the same paths, and simplifies
the addition of new tests.

make check -j12 times:
	Before: 9m24.973s
	After:	7m52.005s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:44:49 +10:30
Rusty Russell
fae7f68731 helpers.sh: make check_tx_spend able to check for a specific transaction.
It currently takes an unused "who to blame" argument, which doesn't
always make sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:34:35 +10:30
Rusty Russell
c6a187a5d7 helpers.sh: set up funding in start_lightningd()
Every test wants this anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:33:35 +10:30
Rusty Russell
9e777ae922 test.sh: move core routines to scripts/helpers.sh
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:32:35 +10:30
Rusty Russell
973ef3c932 test.sh: move valgrind PREFIX setting to vars.sh
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:31:35 +10:30
Rusty Russell
700c536a27 Makefile: remove all the old Elements Alpha support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:30:35 +10:30
Rusty Russell
bff81c0ecf lightningd: ignore SIGPIPE.
It can happen when a peer hangs up, but also a JSON connection.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:29:35 +10:30
Rusty Russell
e7e7345596 peer: fix steal (penalty tx) generation code.
In particular, we got a segv because we were measuring the wrong
wscript, then we miswired the inputs.  It only worked because our
current steal tests don't have a to_us_idx output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 23:28:35 +10:30
Rusty Russell
3372645d8e peer_disconnect: simply free if in STATE_INIT.
db_forget_peer() was harmless, but we haven't been entered into the
database yet anyway, and it asserted that we should have been STATE_CLOSED.

Closes: #67
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-10 11:21:16 +10:30
Rusty Russell
c2d71497ff opt, utils: new arg for wrappers due to low-level tal change.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:56:38 +10:30
Rusty Russell
9708c7a019 timers: use time_mono, as that's what ccan/timers now wants.
Fixes: #58
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:56:15 +10:30
Rusty Russell
4bed6c8c67 controlled_time: remove
We don't need it for testing at the moment, and if we do it'll have
to change to relative anyway now we're going to use time_mono().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:54:15 +10:30
Rusty Russell
c5de5d4c39 timeout: remove absolute timers.
We don't actually use them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:53:15 +10:30
Rusty Russell
3aca5c87e3 init: rebroadcast anchors on restart if we haven't seen them.
It's possible that we won't have sent the anchor, but state is
committed in db.  And our current philosophy is that we retransmit all
the txs dumbly, all the time.

Our --restart --timeout-anchor test trigger this case, too, so
re-enable that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 18:52:15 +10:30
Rusty Russell
0f09605e17 test: restore anchor timeout tests.
Importantly, they're now entirely block driven.  We don't use
dev-setmocktime at all any more.

This also fixes a bug if we run the test twice against the same
bitcoind; we need to extract the time from the block header rather
than assuming bitcoind is on the current time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 17:50:54 +10:30
Rusty Russell
6f360422d4 chaintopology: restore anchor timeout.
Instead of using wall-clock time, we use blocks.  This is simpler and
better for database restores.  And both sides will time out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 17:14:22 +10:30
Rusty Russell
1fe83f47a9 dev-broadcast: control whether we send out transactions.
Good for testing; we also flush broadcast when it's re-enabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 17:14:21 +10:30
Rusty Russell
f715752e71 chaintopology: keep txs to send in a struct.
This allows us to add a new field for a callback at the end, but
more subtle, ensures broadcast in order (which simplifies testing).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 17:14:10 +10:30
Rusty Russell
56b0f03c5a peer: fix retransmission before open packet.
Re-enabling the next test revealed bugs: if we need to retransmit the
initial open_commit_sig packet, we currently tried to send it as an
UPDATE_COMMIT, which isn't allowed.  Fixing that revealed that if
we have to retransmit the initial open, we didn't do that either.

Thus the initial open should count towards the ack count, and we should
special case transmissions of 0 (pkt_open) and 1
(pkt_open_commit_sig).

We also save those early state changes to the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 08:04:28 +10:30
Rusty Russell
41f3f8e067 db: store anchor input.
If we die before we've established connection, we should remember how
we were going to fund it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 08:04:28 +10:30
Rusty Russell
adae62e261 peer: determine fees earlier.
Sure, information might be out of date, but at least we can report an
error earlier.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2016-11-09 08:04:28 +10:30