Commit Graph

8023 Commits

Author SHA1 Message Date
Rusty Russell
13356b75bf wire: move towire/fromwire_amount from wire/ into common/amount.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
4dbfce6057 wire: move towire/fromwire_preimage out to bitcoin/preimage.c.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
27220646c3 common/wire: move bitcoin-specific marshalling functions into bitcoin files.
We did this originally because these types are referred to in the bolts, and we
had no way of injecting the correct include lines into those.  Now we do, so
there's less excuse for this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
f77d70d546 common/json: move rest of bitcoin/lightning-specific json functions to json_helpers.
This dramatically reduces the linking requirements of lightning-cli.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
fda5f0b427 common/channel_id: move channel_id into its own file.
The definition was in wire/wire.h, and helper functions in fromwire.c!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
f02c464c2e wire/Makefile: list wire/tlvstream.c in WIRE_SRC.
Otherwise, it doesn't depend on anything (it's compiled becasue
plugins/keysend explicitly lists wire/tlvstream.o as a dependency).

This made me miss a compile break.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
6635fe12e4 common/test/run-features.c: fix cppcheck 1.90 false positive.
We only use sizeof(f1->bits).

```
common/test/run-features.c:84:36: error: Uninitialized variable: f1 [uninitvar]
 for (size_t i = 0; i < ARRAY_SIZE(f1->bits); i++) {
                                   ^
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
3c625b367d doc: update cli documentation.
1. help now uses a boutique format, not -H.
2. document the -F option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 15:25:44 +02:00
Rusty Russell
abb16b4226 cli: implement new 'flattened JSON' mode.
Much nicer for grepping, since `{ "foo": { "bar": [7] } }` is turned into
`foo.bar[0]=7`.

Changelog-Added: cli: New `--flat` mode for easy grepping.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 15:25:44 +02:00
Rusty Russell
679d3494b4 cli: tweak -H output to remove format-hint fields.
-H removes the top-level if there's only one, and 'format-hint'
breaks this heuristic, so we end up with:

```
help=command=autocleaninvoice [cycle_seconds] [expired_by]
category=plugin
description=Set up autoclean of expired invoices.
verbose=Perform cleanup every {cycle_seconds} (default 3600), or disable autoclean if 0. Clean up expired invoices that have expired for {expired_by} seconds (default 86400).
command=check command_to_check
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 15:25:44 +02:00
mb300sd
b6285ffa5d Fix bash completion.
At some point lightning-cli help defaulted to human readable format, the additional -H broke the bash completion.

Changelog-Fixed: bash completion on lightning-cli now works again
2020-05-15 15:25:33 +02:00
Rusty Russell
97e3d61748 pyln: fix incorrect python syntax.
contrib/pyln-proto/pyln/proto/bech32.py:120
  /home/rusty/devel/cvs/lightning/contrib/pyln-proto/pyln/proto/bech32.py:120: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    assert decode(hrp, ret) is not (None, None)

I think this warning is correct (though I don't see the warning once I installed coincurve:
are we suppressing warnings?)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 15:21:50 +02:00
Rusty Russell
4bcac3032c bitcoin: remove unused struct bitcoin_tx_input.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 09:55:59 +09:30
Rusty Russell
88402bc234 Makefile: fix clean and distclean for external libs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 09:55:59 +09:30
William Casarin
d6ede8db1b build: cleanup compiler output
Hide CFLAGS and LDFLAGS line noise each time an object file is compiled
or linked.

Also add a `make show-flags` command for displaying CC, LD, CFLAGS and
LDFLAG information. This is shown at the start of each build.

Use `V=1 make` to restore original output

make

  CC: gcc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning" -Wall [..]
  LD: gcc   -Og    -Lexternal -lwallycore -lsecp256k1 -ljsmn  [..]
  ...
  cc wallet/test/run-db.c
  cc lightningd/test/run-jsonrpc.c
  cc lightningd/test/run-invoice-select-inchan.c
  cc lightningd/test/run-log-pruning.c
  cc lightningd/test/run-find_my_abspath.c
  cc cli/test/run-large-input.c
  cc cli/test/run-remove-hint.c
  ld lightningd/lightning_hsmd
  ld lightningd/lightning_gossipd
  ld lightningd/lightning_openingd
  ld lightningd/lightning_channeld
  ld lightningd/lightning_closingd
  ...

Signed-off-by: William Casarin <jb55@jb55.com>
Changelog-Changed: build: default compile output is prettier and much less verbose
2020-05-14 17:44:01 +02:00
Rusty Russell
75a93ec32f lightningd: EXPERIMENTAL_FEATURES: fix crash caused by test_sendonionmessage
Commit b0c9059602 broke the case where
next_node_id is NULL:

[libsecp256k1] illegal argument: pubkey != NULL
lightning_channeld: FATAL SIGNAL 6 (version 13d9c27)
0x55b4cd261b64 send_backtrace
	common/daemon.c:39
0x55b4cd261c0e crashdump
	common/daemon.c:52
0x7fc60307746f ???
	???:0
0x7fc6030773eb ???
	???:0
0x7fc603056898 ???
	???:0
0x55b4cd2c7cee ???
	???:0
0x55b4cd2d74d4 ???
	???:0
0x55b4cd26ac62 node_id_from_pubkey
	common/node_id.c:12
0x55b4cd24e194 handle_onion_message
	channeld/channeld.c:1890
0x55b4cd24e697 peer_in
	channeld/channeld.c:2001
0x55b4cd2521f4 main
	channeld/channeld.c:3419
0x7fc6030581e2 ???
	???:0
0x55b4cd24881d ???
	???:0
0xffffffffffffffff ???
	???:0

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-14 12:55:42 +02:00
Candle
fbfe0e6c0b Comment optimisations
I quickly ran through the comments in lightning.py and saw a few small inconsistencies:

- upper/lower case for the "B" in "Bitcoin" unified (see https://github.com/lnbook/lnbook/pull/98)
- added missing "." after a complete sentence
- removed unnecessary double spaces
2020-05-12 22:19:34 +02:00
Rusty Russell
1fb505d118 pytest: extend timeout for test_payment_duplicate_uncommitted
We've been seeing some Travis timeouts under VALGRIND, with the
10 second timeout here: use TIMEOUT as per standard.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-12 21:57:43 +02:00
Sebastian Falbesoner
0bf93137f6 devtools/credit: check if tag exists first
Changelog-None
2020-05-12 19:48:05 +02:00
lisa neigut
b881f7337e make the experimental features branch work ok 2020-05-12 15:46:58 +09:30
lisa neigut
d35149e5ca try to fix travis test flakes 2020-05-12 15:46:58 +09:30
lisa neigut
487771aee5 tests: skip longer htlc tests on valgring+slow machines
These guys take a while to run, so let's just skip them on the
valgrind/slow-machine combos :/
2020-05-12 15:46:58 +09:30
lisa neigut
6e4b4c50f0 coin tests: try to make a bit more travis robust 2020-05-12 15:46:58 +09:30
lisa neigut
8acbbca05d coins: use the chain's BIP173 name instead of a 'unit of account'
Updates the unit of account to be the chain_id, which is the BIP173 name
of the chain that the coins moved on.

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
de065580f6 coins: update API surface for creating coin movements
Canonicalize the signature for the 'tag-type' of coin moves by unique
constructor/method calls.

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
aab9893661 coins: have we_fulfilled be fully 'ternary'
note that 'null' 'we_fulfilled's are going to be legacy from this
release forward.
2020-05-12 15:46:58 +09:30
lisa neigut
8537e77ac7 coins: re-write API interface for htlc notices
Wrap up more logic internally to the method call for htlcs. Also, don't
touch part id if we're not the 'origin'

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
4bfbb58c56 coins: fix feerate for withdraw so test work? 2020-05-12 15:46:58 +09:30
lisa neigut
087ab166b3 coins, fix: don't crash if the to_us amount is greater than our_msat
It's possible for our peer to publish a commitment tx that has already
updated our balance for an htlc before we've completed removing it from
our commitment tx (aka before we've updated our balance). This used to
crash, now we just update our balance (and the channel balance logs!)
and keep going.

If they've removed anything from our balance, we'll end up counting it
as chain_fees below. Not ideal but fine... probably.
2020-05-12 15:46:58 +09:30
lisa neigut
ffd9467f14 coin moves: persist the coin movement index counter to disk
Should make it easier to track when coin moves in the plugin are
disjoint from what c-lightning says it's broadcast already.
2020-05-12 15:46:58 +09:30
lisa neigut
44f747b29a coin moves: update PLUGINS.md with details about coin_movements
Changelog-Added: Plugins: new notification type, `coin_movement`, which tracks all fund movements for a node
2020-05-12 15:46:58 +09:30
lisa neigut
de86e29e16 coin moves: log all withdrawals when confirmed in a block
This moves the notification for our coin spends from when it's
successfully submited to the mempool to when they're confirmed in a
block.

We also add an 'informational' notice tagged as `spend_track` which
can be used to track which transaction a wallet output was spent in.
2020-05-12 15:46:58 +09:30
lisa neigut
e9d26a46e0 coin moves: actually record the blockheight for all chain moves
Previously we were annotating every movement with the blockheight of
lightningd at notification time. Which is lossy in terms of info, and
won't be helpful for reorg reconciliation. Here we switch over to
logging chain moves iff they've been confirmed.

Next PR will fix this up for withdrawals, which are currently tagged
with a blockheight of zero, since we log on successful send.
2020-05-12 15:46:58 +09:30
lisa neigut
9caf20f636 coin moves: don't log coin moves in onchaind if we're replaying
On node start we replay onchaind's transactions from the database/from
our loaded htlc table.  To keep things tidy, we shouldn't notify the
ledger about these, so we wrap pretty much everything in a flag that
tells us whether or not this is a replay.

There's a very small corner case where dust transactions will get missed
if the node crashes after the htlc has been added to the database but
before we've successfully notified onchaind about it.

Notably, most of the obtrusive updates to onchaind wrappings are due to
the fact that we record dust (ignored outputs) before we receive
confirmation of its confirmation.
2020-05-12 15:46:58 +09:30
lisa neigut
1b5221cbf5 coin moves tests: push_msat and the wallet withdrawal
Check that we account for push_msat and wallet withdrawal/deposits
correctly
2020-05-12 15:46:58 +09:30
lisa neigut
8c986d67db coin moves: test all of the onchaind variants
Mostly we update existing tests to account for channel balances. In a
few places, new tests were needed as there wasn't an existing pathway
that tested the chain-fees for a few penalty cases
2020-05-12 15:46:58 +09:30
lisa neigut
41d3471c7f coin moves tests: save updates to disk
If we don't save to disk, if the node restarts we'll lose them all
and the resulting balance check at the end will be incorrect.
2020-05-12 15:46:58 +09:30
lisa neigut
5d58f125c5 coin moves: record withdrawals
For every withdrawal transaction emitted, we record each of the outputs
plus the fees paid for this transaction.
2020-05-12 15:46:58 +09:30
lisa neigut
fc54bfc488 coin moves: record wallet deposits 2020-05-12 15:46:58 +09:30
lisa neigut
e623143c9f fix: the tx types for element fee updates were annotated wrong
Fixes the tx type annotation in a few places.
2020-05-12 15:46:58 +09:30
lisa neigut
9b429cdf4f coin moves: some additional htlc handling
We record htlcs when they're fulfilled as 'withdrawals' that are
onchain. This should make use of the payment_hash that we stashed.

Additionally, if an htlc spend comes through that's not ours, it's
probably them resolving our attempted cheat; we should allow it to
proceed without bombing, and just do our accounting as necessary. It'll
all come out in the wash.
2020-05-12 15:46:58 +09:30
lisa neigut
6ee6cdc280 coin moves: record their attempted cheat (and our handling thereof)
For cheats, we do a little bit of weird accounting. First we 'update'
our on-ledger balance to be the entirety of the channel's balance. Then,
as outputs get resolved, we record the fees and outputs as withdrawals
from this amount.

It's possible that they might successfully 'cheat', in which case we
record those as 'penalty' but debits (not credits).
2020-05-12 15:46:58 +09:30
lisa neigut
007a62a369 coin moves: handle ignored outputs
Ignored outputs don't end up in the same 'resolved' pathway as other
tracked outputs do, so we mark them as moved when proposed/broadcast
instead of when resolved (since they'll never flow through as resolved)
2020-05-12 15:46:58 +09:30
lisa neigut
19268afba5 coin moves: record moves for unilateral closes 2020-05-12 15:46:58 +09:30
lisa neigut
c215a00c45 coin moves: record cheats / loss due to 'unknown' txs
Whenever we detect an 'unknown' tx is published, we should count that as
a loss, as needed.
2020-05-12 15:46:58 +09:30
lisa neigut
034b2c7ee4 coin moves: add handling for mutual closure case 2020-05-12 15:46:58 +09:30
lisa neigut
1f0cfa71b0 coin moves: pass our currently known channel balance to onchaind
We'll need it to do reconciliation for unexpected/penalty closures and
to compute fees paid / outputs trimmed.
2020-05-12 15:46:58 +09:30
lisa neigut
15e4e922c9 wire: serialize the amounts for a bitcoin tx over the wire
we'll need this for calculating fees etc in onchaind
2020-05-12 15:46:58 +09:30
lisa neigut
e8d10edbe5 coin moves: record onchain movements after they've been resolved
We pass them back to lightningd who sends out a notification for them
2020-05-12 15:46:58 +09:30
lisa neigut
175515b914 coin moves: stash the payment hash for onchaind in tracked_output
We'll use it when we record 'onchain' htlc movements that are fulfilled
2020-05-12 15:46:58 +09:30