Commit Graph

10755 Commits

Author SHA1 Message Date
Rusty Russell
8759641b06 ccan: update to get new helpers in ccan/tal
tal_dup_talarr() is simply stolen from common/utils, but tal_dup_or_null is new.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 14:36:55 +10:30
Rusty Russell
5aa8663c98 common/socket_close.c: remove obsolete comment.
Put more inline comments, and also preserve errno properly from read.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
4ef2367c04 connectd: rename struct peer in peer_exchange_initmsg to early_peer.
We want to have a real (persistent) struct peer eventually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
d287d7d229 gossipd: make request handlers return void.
They all returned the next io_plan, but it was always the same.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
0aad222c2d gossipd: rewrite update generation, remove local_chan.
local_chan was mainly around so we could "soft" disable channels (and
really disable them once we used the channel_update in an error
message).

Instead we introduce the idea of a "deferred_update": it's either
deferred indefinitely (a peer goes offline, if we need to send it in
an error we'll apply it immediatly), or simply delayed to avoid
spamming everyone.

The resulting rewrite is much clearer, IMHO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
8e37eb0028 gossipd: local_direction helper to generalize is_local_channel.
Increasingly we want to know is it local, and get the direction: it's
more efficient to do both at once.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
70ed47d77a channeld: add dev-disable-commit-after instead of dev-disconnect -nocommit
It was always a hack, but an impossible one once connectd will be
interpreting dev-disconnect!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
888745be16 dev_disconnect: remove @ marker.
Once connectd is doing this, we can't close as soon as we send,
and in fact we can't do 'fail write' either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
560fa06f42 pytest: disable dev-disconnect blackhole tests.
These would have to be done by connectd, not the local daemon, once it's
intermediating.  Otherwise the remote peer won't see any change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
2fb58772a4 common/socket_close: don't wait forever.
The "read until closed" trick doesn't work if the other end doesn't
close (as found in the next patch, where we use DEV_DISCONNECT_DISABLE_AFTER).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
35e3c1866e common: generalize extract_channel_id().
connectd is going to end up using this do demux; make it fast and complete.

Fixing this reveals a problem in openingd: it now extracts the channel_id
from funding_signed (which is where we transition off the temporary), and
gets upset.  So fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
5111f39d2a connectd: clean up lightningd connection handling.
They all returned the same thing anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
6ce7e7ea60 common: is_msg_gossip_broadcast() for putting things in filter.
We don't need the other "gossip" messages in our echo-suppression filter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
90b669857e lightningd: handle channel cleanups more explicitly.
1. Freeing an unconfirmed channel already releases the subd, so don't
   do that explicitly.
2. Use channel->owner to transfer ownership where possible, using
   channel_set_owner() which handles all the cases.

This simplifies the code and makes it more readable, IMHO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
18526a3a5b lightningd: close one more fd for subdaemons.
Noticed by stracing for an unrelated problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Vincenzo Palazzo
9f02b6eb59 external: update lnprototest dependeces
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-12-28 09:49:09 +10:30
niftynei
70a73928cb balance-snaps: add a balance_snapshot event; fires after first catchup
Fire off a snapshot of current account balances (node wallet + every
'active' channel) after we've caught up to the chain tip for the *first*
time (in other words, on start).
2021-12-28 04:42:42 +10:30
niftynei
f169597a02 test plugins/coin_moves: clean up/tidy coin moves plugin logic
removes unused in-memory stash of coin-moves; don't write the entire set
of coinmoves out on every update.
2021-12-28 04:42:42 +10:30
niftynei
ec991e3af7 balance snap: first pass 2021-12-28 04:42:42 +10:30
niftynei
5570c6da08 coin_moves: cleanup tags list, update docs, remove unused method
We don't use journal_entry any more; remove it and update all of the
remaining tags (plus docs)
2021-12-28 04:42:42 +10:30
niftynei
8225a9decf coin_mvt: log events for pushes/lease_fees for leased channels
We need to stash/save the amount of the lease fees on a leased channel,
we do this by re-using the 'push' amount field on channel (which is
technically correct, since we're essentially pushing the fee amount to
the peer).

Also updates a bit of how the pushes are accounted for (pushed to now
has an event; their channel will open at zero but then they'll
immediately register a push event).

Leases fees are treated exactly the same as pushes, except labeled
differently.

Required adding a 'lease_fee' field to the inflights so we keep track of
the fee for the lease until the open happens.
2021-12-28 04:42:42 +10:30
niftynei
bddd3694fa coin_mvt: record fees for an outbound htlc
If we initialized the payment, the fees are the entire fee-chain
(final hop amount - starting hop amount)

If it's a payment we routed, the fees are the diff between the
inbound htlc and the outbound (net gain by this routing)

Added to database so data persists nicely.
2021-12-28 04:42:42 +10:30
niftynei
29c6718297 coin_mvt: record new 'fees' field on htlc channel moves
We record the amount of fees collected for a routed payment. For
simplicity's sake on the data agg side, we record the fee payment on
*BOTH* the incoming htlc and the outgoing htlc. Note that this results
in double counting if you add up the fees from both an in-routed and
out-routed payment.
2021-12-28 04:42:42 +10:30
niftynei
b6463174d6 coin moves: turn 'tag' into 'tags' array, add OPENER tag
Channels that the node has hopened will now be tagged with 'opener' in a
list of tags.
2021-12-28 04:42:42 +10:30
niftynei
d3d6c09758 coin_moves: update some info about 'blockheight' 2021-12-28 04:42:42 +10:30
niftynei
4506f639fa coin_moves: remove 'index' for moves; bump version
Get rid of the 'movement_idx', since we replay events now.

Since we're removing a field from the 'coin_movement' event emission, we
bump the version type.

Changelog-Updated: `coin_movements` events have been revamped and are now on version 2.
2021-12-28 04:42:42 +10:30
niftynei
8098a4cd21 onchaind: remove 'is_replay' logic
we used this originally to suppress duplicate issuance of coin-move
events; we're assuming that any plugin expects duplicate events though
(and knows how to de-dupe them), so we no longer need this logic.
2021-12-28 04:42:42 +10:30
niftynei
27ff87c045 coin_moves: de-dupe moves as they come in
test_onchain_dust_out restarts a node, which produces duplicate events.
this is expected, but we need to de-duplicate the events stream to get
accurate results
2021-12-28 04:42:42 +10:30
niftynei
d2c4d4aec2 coin_mvts: rewrite how onchain events are recorded, update tests
The old model of coin movements attempted to compute fees etc and log
amounts, not utxos. This is not as robust, as multi-party opens and dual
funded channels make it hard to account for fees etc correctly.

Instead, we move towards a 'utxo' view of the onchain events. Every
event is either the creation or 'destruction' of a utxo. For cases where
the value of the utxo is not (fully) debited/credited to our account, we
also record the output_value. E.g. channel closings spend a utxo who's
entire value we may not own.

Since we're now tracking UTXOs onchain, we can now do more complex
assertions about the onchain footprint of them. The integration tests
have been updated to now use more 'chain aware' assertions about the
ending state.
2021-12-28 04:42:42 +10:30
niftynei
07039fc2b4 onchaind:move some stuff around so we can call it 2021-12-28 04:42:42 +10:30
niftynei
ade2242d14 coin_mvts: don't overwrite account names we add in onchaind
we assume that every event coming from onchaind is for that channel's
account, but now that we sometimes track external + wallet events from
onchaind, we should only add the channel account name if there's nothing
set there already
2021-12-28 04:42:42 +10:30
niftynei
3b3b8fb3ec coin_mvt: add "output value" to coin movement
When we log "external" wallet events, it's helpful to know what the
value of that output is, so let's log it
2021-12-28 04:42:42 +10:30
niftynei
b7ca45514c coin_moves: fix account name serialization
Switch over to a more robust method of serializing strings
2021-12-28 04:42:42 +10:30
niftynei
5e1d8d6ad6 amount: helper for msat == 0
Shorthand for checking if value is zero
2021-12-28 04:42:42 +10:30
niftynei
43ade318d9 coin-move: the txid is now optional but outpoints are not
we're pivoting from a txid based world to a outpoint based world. every
coin movement (onchain) will correspond with a outpoint; only the spend
of an outpoint will have a tx_txid
2021-12-28 04:42:42 +10:30
niftynei
511c0df63a coin-mvts: on close, record an 'end' of our channel balance
For a mutual close, this is the only record for this that we need.
We remove all of the other tracking around mutual closes.
2021-12-28 04:42:42 +10:30
niftynei
737772f1ca coin-mvt: remove all the chain fees tracking
this will be impllicit going forward.
2021-12-28 04:42:42 +10:30
niftynei
e791efe23b coin-mvt: remove all the spend-tracking
We're not going to do 'spend tracks' any more; instead we'll emit an
event whenever an output is included in a broadcast tx

(even if the broadcast fails!!)
2021-12-28 04:42:42 +10:30
niftynei
9b1a0f9b7b coin-moves: remove penalty + chain fee types
FIXME: still has coin tracking stuff!
2021-12-28 04:42:42 +10:30
Dustin Dettmer
6cee603f71 Add pip requirements install to Mac instructions 2021-12-24 15:49:01 +01:00
Anand Suresh
0c487a61ba Fix comment to reflect the code
The comment mentions a 100MB log book, but the code was modified 15 months ago to allocate a 10MB log book to preserve memory.
2021-12-21 10:59:25 +10:30
Vincenzo Palazzo
5bbe3feee7 doc: document the errors code
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-12-15 12:24:54 +10:30
Vincenzo Palazzo
43ff949ea7 lightningd: support hsm error code
Suggested-by: Rusty Russell
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Changed: Support hsm specific error error code in lightning-cli
2021-12-15 12:24:54 +10:30
Rusty Russell
9a85b02c6f wire: add missing spec patch.
You have to manually run "make extract-bolt-csv" to recreate the
spec files, so we didn't notice that I forgot to check this in :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-15 06:53:39 +10:30
Rusty Russell
064e239ae1 ccan: update, add graphql module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-15 06:53:23 +10:30
Christian Decker
a98ccac777 pay: Disable channels that are not in normal state
Since we have the exact state of the channels from the `listpeers`
response we can filter out the ones that are not yet normal or not
anymore. This is more of a safety net, given that the `gossip_store`
should contain local disables, but better not be racy :-)

Changelog-None
2021-12-14 15:07:45 +10:30
Rusty Russell
1ac5a1b7e2 pytest: restore gossip advertizement tests for default port.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-14 13:41:31 +10:30
Vincenzo Palazzo
82124b7733 doc: force mistune==0.8.4 during pip package resolution.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-None: doc: force mistune==0.8.4 during pip package resolution.
2021-12-14 11:27:46 +10:30
lightning-developer
bf71d250a7 Removed FIXME to match bolts PR 918
If PR 918 is merged to the bolts this requirement will be dropped from the spec and we do not need to have a FIXME in the code base.
2021-12-14 11:26:47 +10:30
Ken Sedgwick
335ef3fb69 hsmd: Add hsmd_ready_channel 2021-12-14 11:24:52 +10:30