Commit Graph

3775 Commits

Author SHA1 Message Date
Rusty Russell
7604f27fb8 lightningd: make sure openingd and uncommitted_channel free each other.
Without this, we can get errors on shutdown:

Valgrind error file: valgrind-errors.27444
==27444== Invalid read of size 8
==27444==    at 0x1950E2: secp256k1_pubkey_load (secp256k1.c:127)
==27444==    by 0x19CF87: secp256k1_ec_pubkey_serialize (secp256k1.c:189)
==27444==    by 0x14FED9: towire_pubkey (towire.c:59)
==27444==    by 0x15AAFB: towire_gossipctl_peer_disconnected (gen_gossip_wire.c:969)
==27444==    by 0x1253EF: opening_channel_errmsg (opening_control.c:526)
==27444==    by 0x1386A3: destroy_subd (subd.c:589)
==27444==    by 0x18222C: notify (tal.c:240)
==27444==    by 0x1826E1: del_tree (tal.c:400)
==27444==    by 0x182733: del_tree (tal.c:410)
==27444==    by 0x182733: del_tree (tal.c:410)
==27444==    by 0x182B1F: tal_free (tal.c:511)
==27444==    by 0x11FC53: main (lightningd.c:410)
==27444==  Address 0x6c3af98 is 72 bytes inside a block of size 216 free'd
==27444==    at 0x4C30D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27444==    by 0x1827BC: del_tree (tal.c:421)
==27444==    by 0x182B1F: tal_free (tal.c:511)
==27444==    by 0x11F3C7: shutdown_subdaemons (lightningd.c:211)
==27444==    by 0x11FC27: main (lightningd.c:406)
==27444==  Block was alloc'd at
==27444==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27444==    by 0x182296: allocate (tal.c:250)
==27444==    by 0x182863: tal_alloc_ (tal.c:448)
==27444==    by 0x12F2DF: new_peer (peer_control.c:74)
==27444==    by 0x125600: new_uncommitted_channel (opening_control.c:576)
==27444==    by 0x125870: peer_accept_channel (opening_control.c:668)
==27444==    by 0x13032A: peer_sent_nongossip (peer_control.c:427)
==27444==    by 0x116B9E: peer_nongossip (gossip_control.c:60)
==27444==    by 0x116F2B: gossip_msg (gossip_control.c:172)
==27444==    by 0x138323: sd_msg_read (subd.c:503)
==27444==    by 0x137C02: read_fds (subd.c:330)
==27444==    by 0x175550: next_plan (io.c:59)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Rusty Russell
d0bfd8407a test_lightningd.py: catch unexpected reconnections.
I had a weird failure which was caused by an unexpected disconnect and
reconnecct.  Since we are prersistend and recover from these, they can
slip through our tests; most tests don't involve reconnection, so we
need to catch this explicitly.

For the connect() helper, we always suppress reconnection; tests which
want it all want other options so don't use this helper anyway.  (Actually,
after I said that, test_closing_while_disconnected was added when I
rebased, which did require it, so I had to open-code that one).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Rusty Russell
05ba976a41 lightningd: --dev-no-reconnect needs to always suppress reconnection.
It didn't in the restore-from-db case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ
d6bf7930b8 Loosen close timeout in test_closing_different_fees 2018-04-23 05:24:46 +00:00
ZmnSCPxj
0b331a2b60 test_lightningd.py: Clean up some uses of 'close' RPC. 2018-04-23 05:24:46 +00:00
Rusty Russell
68758a5d42 json_close: test that it works while disconnected.
It should, indeed, close once they reconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 05:24:46 +00:00
ZmnSCPxj
2cee1ab20f peer_control: Make close wait for complete closure, with timeout.
Also report tx and txid, and whether we closed unilaterally or
bilaterally, if we could close the channel.

Also make a manpage.

Fixes: #1207
Fixes: #714
Fixes: #622
2018-04-23 05:24:46 +00:00
BT
4673ba6a0a Add in sudo apt-get update into ubuntu
Add in sudo apt-get update into ubuntu instructions because sometimes the process fails if not updated
2018-04-22 21:18:16 +02:00
ZmnSCPxj
78a074af5a test_lightningd: Also test receive of P2WPKH is correct.
For completeness.
Existing test will check P2SH-P2WPKH address only.
P2WPKH seems OK but test it anyway in case of regression.
2018-04-22 12:51:03 +02:00
ZmnSCPxj
990ef729b0 walletrpc: Refactor out encoding of pubkey to address. 2018-04-22 12:51:03 +02:00
ZmnSCPxj
3bf626a750 walletrpc: Correct generation of P2SH-P2WPKH addresses for listfunds. 2018-04-22 12:51:03 +02:00
ZmnSCPxj
c2e85cd051 test_lightningd: Check listfunds address matches newaddr paid to. 2018-04-22 12:51:03 +02:00
Christian Decker
140123b4c8 pytest: Add local_add tests to test_gossip_persistence and cleanup
Mixing the test into the existing test allows us to reduce the execution, but
adds a bit of complexity, so I added two small helpers.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-22 12:50:34 +02:00
Christian Decker
64fbea1528 gossip_store: Save local_add_channel messages and replay them
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-22 12:50:34 +02:00
Christian Decker
7497f972f1 moveonly: Move handle_local_add_channel to routing.h
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-22 12:50:34 +02:00
Christian Decker
ddbf016152 gossip: Pass rstate to handle_local_add_channel directly
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-22 12:50:34 +02:00
conanoc
7170521895 change spaces to tabs, align function parameters 2018-04-21 15:55:00 +02:00
conanoc
0733770559 Adjust indents 2018-04-21 15:55:00 +02:00
ZmnSCPxj
0a586acf6e lightning-autocleaninvoice.7: Properly generate instead of using an empty file.
Strange behavior where `a2x` sometimes does not work at all.
2018-04-18 12:37:15 +02:00
ZmnSCPxj
774af5f817 payalgo: Describe maxdelay argument of pay. 2018-04-17 17:29:36 +02:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ
11ca729d85 wallet, payalgo: Save detail of payment failures for later reporting. (#1345)
Pointless for remote failures as those are never sent by
the erring node, but for local failures we can give more
detail.
2018-04-16 15:29:40 +02:00
practicalswift
0eff28c80f Assert our parse_wireaddr assumptions in run-ip_port_parsing.c
We assume that the parse_wireaddr(...) calls return true.
Otherwise addr.port will be uninitialized.
2018-04-15 17:45:47 +02:00
conanoc
b2f7e9af4a Support debugging with lldb
Running with lldb cause SIGINT, which makes waitpid() returns
error with errno as EINTR. This patch retry waitpid() to ignore
EINTR errors.
2018-04-15 17:42:24 +02:00
Rusty Russell
a6c76f9079 closingd: even with old clients, still require funding offer < commitment fee.
For older clients we could do more exhaustive checks, but effort is better
spent on removing this altogether post 0.6 as clients upgrade.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-15 15:32:14 +02:00
Rusty Russell
1faec99c59 closingd: make sure we restrict feerange with initial offer.
@ZmnSCPxj points out that this is allowed, though invalid:

1. commitment_fee = 1000
2. funder: 800
3. fundee: 200
4. funder: 900

We need to adjust the feerange using the initial funder offer.
2018-04-15 15:32:14 +02:00
Rusty Russell
7ca4422d7d closing_control: always prefer lower fee, not closest to ideal.
We had an intermittant test failure, where the fee we negotiated was
further from our ideal than the final commitment transaction.  It worked
fine if the other side sent the mutual close first, but not if we sent
our unilateral close first.

ERROR: test_closing_different_fees (__main__.LightningDTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test_lightningd.py", line 1319, in test_closing_different_fees
    wait_for(lambda: p.rpc.listpeers(l1.info['id'])['peers'][0]['channels'][0]['status'][1] == 'ONCHAIN:Tracking mutual close transaction')
  File "tests/test_lightningd.py", line 74, in wait_for
    raise ValueError("Error waiting for {}", success)
ValueError: ('Error waiting for {}', <function LightningDTests.test_closing_different_fees.<locals>.<lambda> at 0x7f4b43e31a60>)

Really, if we're prepared to negotiate it, we should be prepared to
accept it ourselves.  Simply take the cheapest tx which is above our
minimum.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-15 15:32:14 +02:00
Rusty Russell
90364a8cc6 closingd: Correct whose turn it is when moving from initial negotiation to continuous negotiation.
[minor neatening by Rusty]

Fixes: #1361
Reported-by: @nayuta-ueno
Implemented-by: @ZmnSCPxj
2018-04-15 15:32:14 +02:00
Rusty Russell
6424b2decd closing: initialize feerange->allow_mistakes.
This exists for compatiblity with older versions, but it's currently
uninitialized.

Fixed-by: @ZmnSCPxj
2018-04-15 15:32:14 +02:00
Christian Decker
f27cd3e43f topo: Remove in-memory txs from the block struct
The only use for these was to compute their txids so we could notify depth
in case of reorgs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
23984ecde4 chaintopology: Use the DB to locate transactions and rebroadcast txs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
86b6402e5c chaintopology: Refactor get_tx_depth to use the DB backed tx store
We are slowly hollowing out the in-memory blockchain representation to make
restarts easier.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
aa696370af txwatch: Switch to passing only txid into the depth callbacks
All of the callback functions were only using the tx to generate the txid again,
so we just pass that in directly and save passing the tx itself.

This is a simplification to move to the DB backed depth callbacks. It'd be
rather wasteful to read the rawtx and deserialize just to serialize right away
again to find the txid, when we already searched the DB for exactly that txid.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
85fbab2fab wallet: Add function to retrieve a watched transaction's blockheight
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
50600ae241 wallet: Store transactions we are watching, broadcast or own
This will later allow us to determine the transaction confirmation count, and
recover transactions for rebroadcasts.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
633ffeec6f wallet: Add methods to add transactions to the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
0f89653ce8 db: Add table for transactions we are interested in
Currently these are either transactions we sent ourselves or transactions that
we are watching because they are part of a channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Rusty Russell
b0c2e3cd5c gossipd: use a separate CSV file for the gossip_store types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
57b38cac71 gossip_store: empty, don't truncate, on error.
Christian points out that we don't get spend notifications for old
channels if we truncate the store.  We'd need more work to do this,
either validating the channels are still unspent, or replaying old
blocks from the truncation point.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
d5767fb3bb gossipd: print stats even if we truncate store.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
2b8293c9f6 gossipd: don't use pwrite, better error messaging on init.
Since we open with O_APPEND, any write() will append as we want it to.

But we want to distinguish a new store creation from a truncation due
to bad version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
7d0a76c533 goossipd: make store load truncate on errors.
We don't need pread, we just need read, and we can loop internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
3e1b584e73 gossipd: always add message internally before store.
If something goes (fatally) wrong, we won't add it to the store.

This reveals a latent bug in routing_add_channel_announcement() and
friend which did a take() on msg, which it doesn't own.  TAKES means
that it will take ownership IF the caller requests, not an unconditional
ownership transfer (which is an antipattern).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
abbbfac8e2 gossipd: return bool from message announce routines.
Now we can tell if they fail, so we can respond appropriately if we're
loading from the store.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
e8a052eb6d routing: add more debugging to announcement replaced fail.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
30c1ab424f gossipd: reorder handle_node_announcement
I found the logic a bit confusing, so this reworks to bunch the
"no node" cases together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
4aca909acb routing: don't store node_announce unannounced nodes.
We enter nodes in the map when we create channels, but those channels
could be local and unannounced.  This triggered a failure in
test_gossip_persistence since the store truncated when it saw the
first thing was a node_announce.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
ZmnSCPxj
957513666c closing_control: Fix loop limit in better_closing_fee. 2018-04-10 20:45:16 +00:00
ZmnSCPxj
86290b54d4 routing: Use 64-bit msatoshi for messages to and from routing.
Internally both payment and routing use 64-bit, but the interface
between them used 32-bit.
Since both components already support 64-bit we should use that.
2018-04-09 20:45:26 +02:00
Christian Decker
0ba687732f bitcoind: Do not copy the newline character when asking for a block
In the short_channel_id check we were copying the entire result into the next
bitcoin-cli call, including the newline character.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-By: @gdassori
2018-04-09 00:21:20 +00:00
Christian Decker
ba3ceb2abf wallet: Lowerbound the rescan by going at most back to LNs origin
Repeated crashes could result in the `last_processed_block` variable being
pushed further and further into the past (in some cases going as far back as
scanning blocks from 2012...). This is a stop-gap solution that just lower
bounds the value to what is the first possible block we might be interested in
LN, until we have the 0-rescan fix I'm working on.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00