Commit Graph

4049 Commits

Author SHA1 Message Date
Christian Decker
0b427b4c3c opts: Add the max_fee_multiplier to specify acceptable fee ranges
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-14 00:59:42 +00:00
Christian Decker
4dca6daf34 opts: Bump locktime_max so we don't disagree as much with lnd
I still believe that 2 weeks is way too much, but we were promised that these
defaults would be slowly reduced to saner values as the stability increases.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-14 00:59:42 +00:00
Rusty Russell
8183590cdb pytest: test_onchaind_replay needs to wait for tx broadcast.
Waiting for 'Disabling channel' is not enough, since it's async to the
actual tx broadcast: I caught a case where the unilateral close wasn't
in the block, and so failed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-13 16:22:23 +02:00
Rusty Russell
965c20caae tests: reenable developer tests.
72d103d6bb deprecated DEVELOPER env var
in favor of config.vars, but didn't update test_closing.py or test_gossip.py.

5d0a54b7f0 then removed the explicit
DEVELOPER= setting from Travis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-13 16:22:23 +02:00
Rusty Russell
02b55e35b8 daemon: print version in crash backtrace.
Sometimes people only send us the crash, not the logs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-12 23:18:53 +02:00
Rusty Russell
56046d470b configure: take PYTEST from env var or cmdline.
We should only autodetect if it's not set manually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-09 13:39:27 +02:00
Christian Decker
3cf1581d6d configure: Use the ./configure step to find pytest
Fixes #1544

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Christian Decker
135bd962f7 pytest: Mark test_closing_different_fees as flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Christian Decker
37327d31de topo: Remove obsolete FIXME marker
This was addressed in bdb87aa994

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Christian Decker
985af483cf gossip: Wrap insert_broadcast and gossip_store_add in persistent_broadcast
They should sync up nicely otherwise we may be overestimating the stale rate.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
6632f44133 gossip: Disable gossip_store temporarily while replaying messages
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
2b5e1ee65f gossip: Enable the consistency check only when really pedantic
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
8a5bebed59 gossip: Disable future compactions if we fail a compaction
A failed compaction shouldn't be deadly, but we should also not attempt to do
one on every gossip message after the first one fails.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
74a1cbd877 gossip: Implement gossip_store compaction
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
b9a2400a5f gossip: Simplify message handling in gossip_store
`gossip_store_add` is the entry point for messages from the network, so it
should do the bookkeeping and disable on failures. `gossip_store_append` is the
shared function that wraps messages and writes it to the given file. This is
shared between the from network path and the compaction path, so we don't
directly use the `gossip_store` instance, but `fd`s.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
60efa314fe gossip: Separate writing to gossip_store fd from append
We write both when coming from outside, as well as when compacting, so we
extract the write functionality to use it in both cases.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
e6ab594904 gossip: Have gossip_store annotate gossip messages
This makes the exposed interface much smaller, cleaner and will allow us to just
replay gossip messages from the broadcast.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
0546ca446d gossip: Pass routing_state to the gossip_store
We'll need it later to annotate the raw gossip messages, e.g., the capacity of a
channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
eaba5a249a gossip: Introduce bookkeeping into gossip_store for rewrite
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
552ddb8dfd gossip: Pass broadcast_state to gossip_store
We'll be sourcing messages from this `broadcast_state` when rewriting the
`gossip_store`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
37dc458b4d gossip: Have the broadcast_state track its message count
This is far more precise than bolting on the stale tracking in the
`gossip_store`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Jan Sarenik
eab2a62894 external/libsodium: Update to 1.0.16
Fixes lots of extra warnings (#1540)
2018-06-09 06:42:02 +00:00
Nadav Ivgi
d9bd25cff8 README: Add ./configure to installation instructions
Now required, after changes in c49765553f
2018-06-08 21:04:55 +02:00
Christian Decker
4e7fc99ae1 gossip: Duplicate removes can result in null pointers in broadcast
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-08 20:00:27 +02:00
arowser
4429c6e7cd udpate broken link 2018-06-08 17:56:48 +02:00
Rusty Russell
109b429311 libbacktrace: pull latest version, which has config.guess fix applied.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
5aa1f37f07 devtools/Makefile: add devtools/onion.c to DEVTOOLS_TOOL_SRC
This way the object file correctly depends on external headers.  Currently
a parallel build on a clean tree can give:

```
In file included from ./common/sphinx.h:6:0,
                 from devtools/onion.c:5:
./bitcoin/pubkey.h:8:10: fatal error: secp256k1.h: No such file or directory
 #include <secp256k1.h>
          ^~~~~~~~~~~~~
compilation terminated.
<builtin>: recipe for target 'devtools/onion.o' failed
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
5d0a54b7f0 travis: use VALGRIND rather than NO_VALGRIND.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
c49765553f Makefile: not running configure first is so yesterday:
Make should only run configure if config.var already exists:

    $ make
    ./configure --reconfigure
    ./configure: 65: .: config.vars: not found
    ./configure --reconfigure
    ./configure: 65: .: config.vars: not found
    Makefile:179: recipe for target 'ccan/config.h' failed
    make: *** [ccan/config.h] Error 2
2018-06-08 17:56:01 +02:00
Rusty Russell
4bd27ec722 Makefile: clean should not delete configure results, distclean only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
72d103d6bb Makefile: import config.vars.
We leave VALGRIND env var as an override for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
ca8671de0a configure: add --enable-valgrind/disable-valgrind.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
f6d22ed0cf configure: new script.
You can use environment variables or the commandline to set defaults.

It looks very autoconf, but you don't need to learn m4.

Doesn't cover all the obscure flags, but it's easy to extend.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
d395b58ffa ccan: update to have new configurator.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
76dd1577bd external: fix submodule handling for parallel builds, submodule URL changes.
If we change an upstream URL, all submodules break.  Users would need
to run 'git submodule sync'.  Note that the libbacktrace fix was merged
upstream so this is no longer necessary, but it's good for future changes.

Also, stress-testing reveals that git submodule fails locking
'.git/config' when run in paralell.  It also segfaults and other
problems.

This is my final attempt to fix submodules; I've wasted far too many
days on obscure problems it creates: I've already lost one copy of my
repo to apparently unfixable submodule preoblems.  The next "fix" will
be to simply import the source code so it works properly.

Reported-by: @jsarenik
Fixes: #1543
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:56:01 +02:00
Rusty Russell
5d6a9f3fb0 gossipd: check consistency.
This is a hack to check that our gossip state is consistent on every
insert and delete.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell
da55d3c0ff gossipd: handle node_announcement when channel_announcement removed.
Two cases:
1. Node no longer has any public channels: remove node_announcement.
2. Node's node_announcement now preceeds all the channel_announcements:
   move node_announcement to the end.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell
def18a7bc1 gossipd: implement broadcast_del to delete a specific index.
Required if we want to reorder node_announcement broadcasts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell
a38c619486 gossipd: keep index of node and channel announcements.
This lets detect if a node announce preceeds a channel announce once we
delete the node announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Christian Decker
294c76e749 gitignore: Ignore .pytest_cache
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:32:27 +00:00
Christian Decker
de3b359782 pytest: Use pytest fixtures for the test directory and clean it up
The modern, pytest based, tests now clean up after themselves by removing
directories of successful tests and the base directory if there was no failure.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:32:27 +00:00
Christian Decker
38b7a0e2d2 pytest: Remove directories of successful tests
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:32:27 +00:00
Christian Decker
d43403257d wallet: Cleanup db files if the tests succeed
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:32:27 +00:00
Christian Decker
6298ce3b03 gossip: Don't ask bitcoind for outpoints we should know
Compares the `blocknum` in the `short_channel_id` with the range of blocks we
store in the database and abort if we should have known about it. Avoids
bombarding `bitcoind` with requests for channels that have already been spent or
were invalid in the first place.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Christian Decker
2415f48723 topo: Tell chain_topology about the min and max block height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Christian Decker
024dca0fff wallet: Return both min and max block heights
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Christian Decker
0d4b7eaa2c topo: Have chain_topology track both min and max block heights
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Rusty Russell
1bb7713274 gossipd: minor cleanups.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
e4457e59ed pytest: test_gossip_jsonrpc can always test aliases.
The whole test is under DEVELOPER anyway, but even if it weren't it
would work fine.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
035d6067e4 Rename consider_own_node_announce to maybe_send_own_node_announce.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00