Commit Graph

436 Commits

Author SHA1 Message Date
Rusty Russell
57328fe59e tests: use common_setup/common_shutdown to avoid leaks.
It also does more checks (like taken() checks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
c503232cde common: use bitcoin_outpoint.
I started pulling this thread, and the entire codebase got unravelled.

Oh well, it's done now!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
8a85bf6880 lightningd: populate min/max commit fields in db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Christian Decker
5f260840ab jsonrpc: Add groupid to waitsendpay 2021-10-13 13:41:18 +10:30
Christian Decker
817b1b361e db: Add groupid to HTLCs 2021-10-13 13:41:18 +10:30
Rusty Russell
24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
789b330338 lightningd/closing_control.c: move json_close here from peer_control.c
They share some code, but not much: command_find_channel is made
non-static.  Rest is move-only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-15 18:23:30 +02:00
Rusty Russell
183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
6ee8c40b29 closing: add option to set closing range.
This affects the range we offer even without quick-close, but it's
more critical for quick-close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSONRPC: `close` now takes a `feerange` parameter to set min/max fee rates for mutual close.
2021-09-09 12:04:48 +09:30
Sebastian Falbesoner
48473eb3e9 wallet-test: fix segfault due to uninitialized block
The variable `block` (instace of `struct block`) is
allocated on the stack without being initialized, i.e. its
member `prev` points to nowhere. This causes a segmentation
fault on my machine on the binding of "prev_hash" on running
`wallet_block_add`, as the following core-dump analysis
shows:

    $ egdb ./wallet/test/run-wallet ./run-wallet.core
    [...]
    Core was generated by `run-wallet'.
    Program terminated with signal SIGSEGV, Segmentation fault.
    ---Type <return> to continue, or q <return> to quit---
    #0  0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97
    97                      TLOOP1(*dst++ = *src++);
    (gdb) bt
    #0  0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97
    #1  0x000008f73e838f60 in sqlite3VdbeMemSetStr () from /usr/local/lib/libsqlite3.so.37.12
    #2  0x000008f73e83cb11 in bindText () from /usr/local/lib/libsqlite3.so.37.12
    #3  0x000008f44bc91345 in db_sqlite3_query (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:77
    #4  0x000008f44bc91122 in db_sqlite3_exec (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:110
    #5  0x000008f44bcbb3b2 in db_exec_prepared_v2 (stmt=0x8f6845bf028) at ./wallet/db.c:2055
    #6  0x000008f44bcc6890 in wallet_block_add (w=0x8f688b5bba8, b=0x7f7ffffca788) at ./wallet/wallet.c:3556
    #7  0x000008f44bce2607 in test_wallet_outputs (ld=0x8f6a35a7828, ctx=0x8f6a35c0268) at wallet/test/run-wallet.c:1104
    #8  0x000008f44bcddec0 in main (argc=1, argv=0x7f7ffffcaaf8) at wallet/test/run-wallet.c:1930

Fix by explicitely setting the whole structure to zero.

[ Rebuilt generated files, too --RR ]
2021-08-23 19:46:04 +09:30
niftynei
4514d2a180 wallet-test: print better error message on failure 2021-08-17 13:20:00 +09:30
niftynei
773aa26321 utxos: more comprehensive fix for csv check
we only 'need' to check the CSV for 'option_anchor_outputs' (and even
then, really only ones with csv's greater than 1 but...)

Reported-In: #4697
2021-08-17 13:20:00 +09:30
niftynei
67b8a22aa7 channel: wire blockheight updates from channeld in to database 2021-07-20 13:28:38 -04:00
niftynei
265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
niftynei
a396c341cf lease_rates: persist channel's lease info 2021-07-20 13:28:38 -04:00
Christian Decker
edf4b416c2 htlc: Wire extra TLV types into the HTLC onion decoding 2021-06-26 10:55:13 +09:30
Rusty Russell
9929d6383a channeld: add a message to read if we only want to reestablish.
This supports reestablish on a closed channel: we tell channeld to
respond to the reestablish message appropriately, then close the
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 13:23:28 +09:30
Rusty Russell
9f8a6e2314 lightningd: always pass closing connections through channeld.
It handles all the cases of retransmission, and in the normal case
retransmits shutdown and immediately returns for us to run closingd.

This is actually far simpler and reduces code duplication.

[ Includes fixup to stop warn_unused_result from Christian ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We could get stuck on signature exchange if we needed to retransmit the final revoke_and_ack.
2021-06-25 13:23:28 +09:30
Rusty Russell
064ad486e3 close: check that destination is going to be accepted.
Prior to this, sending a v1 address (or, in fact, any random crap!)
would cause the unsupporting node to unilaterally close.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-11 13:54:20 +09:30
Rusty Russell
220b6129bf wallet: save thresholds for option_static_remotekey.
Since we will soon be able to activate it on existing channels,
we need to mark the threshold.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell
bf0320a53e openingd/dualopend: remove send_msg functionality.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-03 16:16:34 +09:30
Rusty Russell
e2f225e4cd lightningd: use connectd to send the final error message, not openingd/dualopend
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-03 16:16:34 +09:30
Antoine Poinsot
fe8074c8c3 Refuse to parse v2 onion addresses without deprecated_apis
Tor v2 hidden services have been deprecated for a while:
https://blog.torproject.org/v2-deprecation-timeline .

This prevents user from being able to set them in the configuration
and to connect to them while still letting us be able to parse them
for gossip.

Changelog-Deprecated: lightningd: v2 Tor addresses.  Use v3.  See https://blog.torproject.org/v2-deprecation-timeline.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-05-24 20:22:45 +09:30
niftynei
062bc12813 rbf: update the channel's funding_txid to match what's mined
If the peer is offline when we see the funding txid, we don't actually
update the channel's info. Here, we move it up to where the scid is set,
so that we always update the channel's funding_txid to the correct
(mined) information.
2021-05-24 12:17:48 +09:30
Rusty Russell
25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
niftynei
71a4a2e31c df: rework closing logic
Trying to put all the disconnect logic into the same path was a dumb
idea. If you asked to reconnect but passed in an 'unsaved' channel, we
would not call the 'reconnect' code.

Instead, we make a differentiation between "unsaved" channels
(ones that we haven't received commitment tx for) and handle the
disconnect for these separate from where we want to do a reconnect.
2021-05-12 11:25:41 +09:30
Rusty Russell
006300ab96 lightningd: set "direction" correctly for connect which is already connected.
This means remembering the connection direction.  We also use the address to try
to reconnect, which we shouldn't bother with if they connect to us.

For peers from the database, we currently always save the addr: we shouldn't really
do this if they connected to us, since it's not useful for reconnecting (we don't
show the addr in JSON reply to listpeers unless we're connected, so it's only an
internal issue).  This is left for future work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-26 13:22:33 +10:30
Rusty Russell
b0d6996ed6 lightningd: get connection direction from connectd.
This matters: if we connected, the address is probably usable for future connections.
But if they connected, the port is probably not (but the IP address may be).

Changelog-Added: JSON-RPC: `connect` returns "direction" ("in": they iniatated, or "out": we initiated)
Changelog-Added: plugins: `peer_connected` hook and `connect` notifications have "direction" field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-26 13:22:33 +10:30
Rusty Russell
6c9d9ee9a2 connect: return address we actually connected to.
Otherwise, we might find an address other than the one given and
the user might think that address worked.

Fixes: #4185
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `connect` returns `address` it actually connected to
2021-03-17 08:38:08 +10:30
Rusty Russell
b62706aa01 close: accept wrong_funding outpoint arg if we negotiated the feature.
Changelog-Added: lightningd: experimental-shutdown-wrong-funding to allow remote nodes to close incorrectly opened channels.
Changelog-Added: JSON-RPC: close has a new `wrong_funding` option to try to close out unused channels where we messed up the funding tx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:08:40 +10:30
Rusty Russell
820fbcd65a channeld: code to send wrong_funding if lightningd says to.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:08:40 +10:30
Rusty Russell
80c2f28373 channeld: accept the 'wrong_funding' shutdown TLV.
If it passes checks, lightningd puts it in the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:08:40 +10:30
niftynei
bec96a6c5b df: add openchannel_abort command
Allows us to clean up an in-progress open that we won't be completing

Changelog-Added: EXPERIMENTAL JSON-RPC: Permit user-initiated aborting of in-progress opens. Only valid for not-yet-committed opens and RBF-attempts
2021-03-15 14:08:44 +10:30
Christian Decker
039751b875 wallet: Load local basepoints from the database alongside the hsmd 2021-03-09 14:58:29 +10:30
Christian Decker
4887a5a18f db: Add migration for the local basepoints and the funding pubkey 2021-03-09 14:58:29 +10:30
niftynei
07153bff6a df: cleanup error handling on lightningd side
Make existing methods understand how unsaved channels work, re-work
errors so that we handle everything appropriately
2021-03-06 15:03:56 +10:30
niftynei
a23277af57 listpeers: include info on channels that are in-process
Channels that we're in negotiation for, but don't have a commitment
transaction saved for yet.
2021-03-06 15:03:56 +10:30
niftynei
ff069ff924 rbf: consolidate failure paths, use "warnings"
We move over to the new "warning" paradigm, instead of using
an "rbf_fail" message.

Every failure is either a warning or an error; on warnings we
hang up and reconnect later, effectively resetting the state.
2021-03-06 15:03:56 +10:30
Rusty Russell
4dcd4ca155 listpeers: add latest feerate and actual last fee amount.
Users have no idea what they would pay for unilateral closes.
At least this gives them a clue!

Reported-by: @az0re on IRC.

Changelog-Added: JSON-RPC: `listpeers` now shows latest feerate and unilaral close fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-05 13:12:41 +10:30
niftynei
da81d4bced channel: skip unsaved channels
Now that "peer->channels" contains `unsaved` channels, skip overthem
where appropriate
2021-03-03 16:19:04 -06:00
niftynei
538fd670ce df: add helper for creating an "unsaved" channel
Channel that's not saved to database yet
2021-03-03 16:19:04 -06:00
niftynei
4baa10ef68 channel: remove psbt and remote_tx_sigs
We just added this, but we've migrated them to the "inflight" struct
instead, as they actually pertain to an single inflight open attempt
2021-03-03 12:28:22 +10:30
niftynei
36f3b13279 inflight: add a 'channel-inflight' concept, in-progress channel stuffs
"inflights" are all potential channel funding transactions that we
currently have commitment transactions for.
2021-03-03 12:28:22 +10:30
Christian Decker
ebb1b19c65 plugin: Fix the custommsg hook not to include the internal prefix
We were always prefixing the `message` field with the internal type
prefix 0x0407, followed by the length prefix. Neither is needed since
the type being constant is of no interest to the plugin and the length
being implicit due to the JSON-encoding.

Reported-by: Ilya Evdokimov
Changelog-Fixed: plugin: The `custommsg` hook no longer includes the internal type prefix and length prefix in its `payload`
Changelog-Deprecated: plugin: The `message` field on the `custommsg` hook is deprecated in favor of the `payload` field, which skips the internal prefix.
2021-03-02 14:41:16 +10:30
Rusty Russell
6b11cc8b8c common: disallow NULL channel_id to peer_failed_err.
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
2021-02-04 12:02:52 +10:30
niftynei
9fdf1ea32a df: reinitialize dual-funding
Prior to this, all reconnect logic lived in channeld. If you
disconnected before we finished building a funding transaction, that was
no big deal. Now, however, we're waiting for the funding to lock in in
dualopend, instead of handing straight to channeld to wait.

So we need a way to restart dualopend.
2021-01-10 13:44:04 +01:00
niftynei
90d936bb60 df: handle shutdown (before lockedin) in dualopend
A channel can be closed before it locks in. This lets dualopend handle
shutdown messages from the peer before the channel is locked in.
2021-01-10 13:44:04 +01:00
niftynei
e0e929ed14 peer_control: remove PSBT from peer_start_channeld
Since we're moving all of the PSBT signing to dualopend, we no longer
need a PSBT to be passed to channeld
2021-01-10 13:44:04 +01:00
Christian Decker
8e908ff652 Revert "wallet: Remove in-memory utxoset filter"
This reverts commit c239a7161b.

The goal of c239a716 was to reduce the memory footprint of our
internal UTXO set tracking, and testing against the sqlite3 backend
showed no performance impact. We have since found that the added
roundtrips to the DB server with postgres was having a considerable
performance impact, and backups would also bloat due to the increased
number of queries.

Undoing this change skips the noop updates that were causing this
regression.

Changelog-Fixed: db: Fixed a performance regression during block sync, resulting in many more queries against the DB than necessary.
2021-01-08 12:08:32 +10:30
Rusty Russell
963f6b5d67 invoice: add an optional local_offer_id.
This allows us to mark an offer used when an invoice derived from it
is paid, and importantly, avoid any other invoices for the offer being
paid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
32c7c133f4 common/sphinx: make onionpacket.routinginfo a dynamic member.
Still asserts that it's the standard size, but makes it a dynamic
member.  For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell
1a3c0a0c0d invoice: fix potential race where invoice is paid/expired while we're calling hook.
There's actually a (very unlikely) race here: we would previously have
crashed with an assertion in invoices_resolve.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 16:50:12 +10:30
Rusty Russell
ca2bd98082 unittest: use common_setup / common_shutdown almost everywhere.
Avoids much cut & paste.  Some tests don't need any of it, but most
want at least some of this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 12:55:09 +10:30
Michael Schmoock
6cc96c07dc feat: adds state_changes to listpeers output
Changelog-Added: RCP: Added 'state_changes' history to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock
68ce25c92d feat: adds timestamp to state_change notification 2020-11-06 14:47:04 -06:00
Michael Schmoock
b7c18517df rpc: adds opener and closer to listpeers channels
Changelog-Added: RPC: Added 'opener' and 'closer' to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock
8a8dabaa58 feat: adds state change cause and message
This adds a `state_change` 'cause' to a channel.
A 'cause' is some initial 'reason' a channel was created or closed by:

  /* Anything other than the reasons below. Should not happen. */
  REASON_UNKNOWN,
  /* Unconscious internal reasons, e.g. dev fail of a channel. */
  REASON_LOCAL,
  /* The operator or a plugin opened or closed a channel by intention. */
  REASON_USER,
  /* The remote closed or funded a channel with us by intention. */
  REASON_REMOTE,
  /* E.g. We need to close a channel because of bad signatures and such. */
  REASON_PROTOCOL,
  /* A channel was closed onchain, while we were offline. */
  /* Note: This is very likely a conscious remote decision. */
  REASON_ONCHAIN

If a 'cause' is known and a subsequent state change is made with
`REASON_UNKNOWN` the preceding cause will be used as reason, since a lot
(all `REASON_UNKNOWN`) state changes are a subsequent consequences of a prior
cause: local, user, remote, protocol or onchain.

Changelog-Added: Plugins: Channel closure resaon/cause to channel_state_changed notification
2020-11-06 14:47:04 -06:00
Rusty Russell
1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
niftynei
b2ec5a9f45 peer_channeld: pass over PSBT, remove second message
We need the PSBT to create the finalized tx from once the peer's
tx_signatures are received. Since we're passing the PSBT, we no longer
need the secondary message to be passed, as it was derived from the
PSBT.

Also removes now unused witness serialization code
2020-10-20 12:50:31 +10:30
Rusty Russell
58f6e316cc lightningd: separate fee_states out of struct channel_info.
It was always kind of weird in there anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
Michael Schmoock
d86855d1f7 feat: adds channel_state_changed notification
This notification will be raised whenever a channel state changes.
The payload includes the channel and peer identifiers and the
old and the new state.

Example payload:

```
{
    "channel_state_changed": {
        "peer_id": "03bc9337c7a28bb784d67742ebedd30a93bacdf7e4ca16436ef3798000242b2251",
        "channel_id": "a2d0851832f0e30a0cf778a826d72f077ca86b69f72677e0267f23f63a0599b4",
        "short_channel_id" : "561820x1020x1",
        "old_state": "CHANNELD_NORMAL",
        "new_state": "AWAITING_UNILATERAL"
    }
}
```

Changelog-Added: Plugins: channel_state_changed notification
2020-09-10 10:24:06 +09:30
niftynei
c50f377a85 psbt: pull out changeset logic into common, update API
Greatly simplify the changeset API. Instead of 'diff' we simply generate
the changes.

Also pulls up the 'next message' method, as at some point the
interactive tx protocol will be used for other things as well
(splices/closes etc)

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei
9c89184c1f dualfund: add feature flag for dual-funding
turn off until we're ready to test both sides
2020-09-09 19:54:20 +09:30
niftynei
d6558deaa4 opening: pass two messages to channeld to send to peer
v2 of channel establishment, in the accpeter case, now sends 2 messages
to our peer after saving the information to disk (our commitment
signatures and our funding transaction signatures)
2020-09-09 19:54:20 +09:30
niftynei
0643945967 peer-control: send set of messages to start, not just one
The accepter has to send 2 messages over to channeld to send at start --
their commitment_signatures and tx_signatures
2020-09-09 19:54:20 +09:30
niftynei
864f2f3e21 channel_id: save to database, dont derive from funding_txid
v2 channel open uses a different method to derive the channel_id, so now
we save it to the database so that we dont have to remember how to
derive it for each.

includes a migration for existing channels
2020-09-09 19:54:20 +09:30
Rusty Russell
2be1f3fe1b lightningd: extract routehint selection code.
We're going to want this for bolt13 formation as well.

As a result of reworking the logic into "candidate selection" then
"route hint selection", we need to change the way round-robin works.
We use a simple incrementing index now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 19:47:11 +09:30
Rusty Russell
e3219d3aa0 utxo: expose is_reserved, make enum constants upper case.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell
83298c030a wallet: switch over to withdraw in module, remove lots of unused code.
This removes the reservation cleanup at startup, too, now they're all
using 'reserved_til'.

This changes test_withdraw, since it asserted that outputs were marked
spent as soon as we broadcast a transaction: now they're reserved until
it's mined.  Similarly, test_addfunds_from_block assumed we'd see funds
as soon as we broadcast the tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` now randomizes input and output order, not BIP69.
2020-09-08 10:14:42 +09:30
Rusty Russell
1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell
8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
ZmnSCPxj jxPCSnmZ
a9a11265dd lightningd/lightningd.h: Maintain a round-robin list of channels. 2020-08-26 09:29:46 +09:30
Rusty Russell
398b4806b9 connectd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
fe8564555f closingd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
dffbf8de85 gossipd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
8ae6740bb1 Makefile: create generic wiregen rules, use for channeld.
This means some files get renamed, and I took the opportunity to clarify
our naming (the *d* is important!)

1. channeld/channel_wire.csv -> channeld/channeld_wire.csv
2. channeld/gen_channel_wire.h -> channeld/channeld_wiregen.h
3. enum channel_wire_type -> enum channeld_wire
4. WIRE_CHANNEL_FUNDING_DEPTH -> WIRE_CHANNELD_FUNDING_DEPTH.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
754765c139 utxo: keep flag to recognize to-remote option_anchor_outputs closes.
We need to remember this in the db (it's a P2WSH for option_anchor_outputs),
and we need to set nSequence to 1 to spend it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
d8d1014ca4 channeld: implement htlc sig checking changes for option_anchor_outputs.
This is best done by passing `struct bitcoin_signature` around instead
of raw signatures.  We still save raw sigs to the db, and of course the
wire protocol uses them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
85e3b43176 channeld, openingd: take into account option_anchor_outputs for fees.
HTLC fees increase (larger weight), and the fee paid by the opener
has to include the anchor outputs (i.e. 660 sats).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
niftynei
d87f31f9a8 utxo: clean up NULL handling of scriptpubkey
Now that we're *guaranteed* to have a scriptpubkey entry in the
database, we remove the NULL handling for it.
2020-07-29 13:13:46 +02:00
niftynei
90b393ca1a hsmd/db: backfill pubkey information so that psbts signing works
the way we use PSBTs to sign things requires that we have the
scriptpubkey available on the utxo so we can populate the witness-utxo
field with it.

this causes problems if we don't already have the scriptpubkey cached in
the database, as in *some* cases we require a round trip to the HSM to
populate them

to get over this hump, we backfill any and all missing scriptpubkey
information for the utxo's that we hold in our wallet.

this will allow us to clean up the NULL handling of missing
scriptpubkeys.
2020-07-29 13:13:46 +02:00
niftynei
65c2bac2f3 hsmd/wallet: pass the bip32_key down into migrations
we're about to add a migration that requires access to the bip32_key
in order to calculate missing scriptpubkeys.

prior to this patch, we don't have access to the bip32 key in the db
migration, as it's set on the wallet but after the db migrations are
run.

here we patch it through so that every migration can access it
2020-07-29 13:13:46 +02:00
Rusty Russell
a9427f1a8d bitcoin/feerate: new exposure for feerate parsing outside lightningd.
This exposes the numeric part of param_feerate() as param_feerate_val().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-08 21:07:20 +02:00
niftynei
bb589e0eaf psbt: database migration for converting last_tx to a psbt
We update the `last_tx` in `channels` to be psbt format, instead
of a linearized transaction.

We need the amount of the input populated, which we have since
this is the 'funding' amount. Ideally we'd also populate the funding
scriptPubkey, but to do that we'd need to access the HSM module to fetch
our local funding pubkey, which isn't initialized at the time that the
database migrations are run.

Since the only field the HSM uses currently when signing these is the
amount field, it's ok to just leave it out.

needs a test!
2020-06-11 13:13:13 +02:00
niftynei
a04f0fe250 psbt: remove input_amounts from bitcoin tx
Instead we will stash them into the PSBT as a utxo/witness record (which
includes the amount)
2020-06-11 13:13:13 +02:00
Antoine Poinsot
f598caa60d config: don't ignore the --commit-fee option.
We did not take the value of --commit-fee into account : this removes
the unused option from lightningd and instead registers it in bcli,
where we set the actual feerate of commitment transactions. This also
corrects the documentation.

Changelog-Fixed: config: we now take the --commit-fee parameter into account.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-20 06:09:24 +09:30
niftynei
fbe50e087a setup: create a common setup which will handle the wally-context
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core

Changelog-None
2020-05-19 13:35:42 +09:30
Rusty Russell
cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
fda5f0b427 common/channel_id: move channel_id into its own file.
The definition was in wire/wire.h, and helper functions in fromwire.c!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
lisa neigut
8acbbca05d coins: use the chain's BIP173 name instead of a 'unit of account'
Updates the unit of account to be the chain_id, which is the BIP173 name
of the chain that the coins moved on.

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
de065580f6 coins: update API surface for creating coin movements
Canonicalize the signature for the 'tag-type' of coin moves by unique
constructor/method calls.

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
8537e77ac7 coins: re-write API interface for htlc notices
Wrap up more logic internally to the method call for htlcs. Also, don't
touch part id if we're not the 'origin'

Suggested-By: @rustyrussell
2020-05-12 15:46:58 +09:30
lisa neigut
e9d26a46e0 coin moves: actually record the blockheight for all chain moves
Previously we were annotating every movement with the blockheight of
lightningd at notification time. Which is lossy in terms of info, and
won't be helpful for reorg reconciliation. Here we switch over to
logging chain moves iff they've been confirmed.

Next PR will fix this up for withdrawals, which are currently tagged
with a blockheight of zero, since we log on successful send.
2020-05-12 15:46:58 +09:30
lisa neigut
9caf20f636 coin moves: don't log coin moves in onchaind if we're replaying
On node start we replay onchaind's transactions from the database/from
our loaded htlc table.  To keep things tidy, we shouldn't notify the
ledger about these, so we wrap pretty much everything in a flag that
tells us whether or not this is a replay.

There's a very small corner case where dust transactions will get missed
if the node crashes after the htlc has been added to the database but
before we've successfully notified onchaind about it.

Notably, most of the obtrusive updates to onchaind wrappings are due to
the fact that we record dust (ignored outputs) before we receive
confirmation of its confirmation.
2020-05-12 15:46:58 +09:30
lisa neigut
fc54bfc488 coin moves: record wallet deposits 2020-05-12 15:46:58 +09:30
lisa neigut
ce8bdfcc45 coin_mvt: wire up notifications for in-channel htlcs
HTLCs trigger a coin movement only when their final form (state) is
reached. This prevents us from needing to concern ourselves with
retries, as well as being the absolutely most correct in terms of
answering the question 'when has the money irrevocably changed hands'.

All coin movements should pass this bar, for ultimate accounting
correctness
2020-05-12 15:46:58 +09:30
lisa neigut
434cad0c3b wallet-htlc: add 'we-filled' flag to saved htlc state
The current plan for coin movements involves tagging
origination/destination htlc's with a separate tag from 'routed' htlcs
(which pass through our node). In order to do this, we need a persistent flag on
incoming htlcs as to whether or not we are the final destination.
2020-05-12 15:46:58 +09:30
Christian Decker
d1f8509060 watchtower: Call the commitment_revoked hook every time we update
Changelog-Added: plugin: Added a new `commitment_revocation` hook that provides the plugin with penalty transactions for all revoked transactions.
2020-05-07 15:05:39 +09:30