We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS. Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need to remember this in the db (it's a P2WSH for option_anchor_outputs),
and we need to set nSequence to 1 to spend it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Includes:
psbt: Use renamed functions for new wally version
psbt: Set the transaction directly to avoid script workarounds
psbt: Use low-S grinding when computing signatures
tx: Use wally_tx_clone from libwally now that its exported
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
the way we use PSBTs to sign things requires that we have the
scriptpubkey available on the utxo so we can populate the witness-utxo
field with it.
this causes problems if we don't already have the scriptpubkey cached in
the database, as in *some* cases we require a round trip to the HSM to
populate them
to get over this hump, we backfill any and all missing scriptpubkey
information for the utxo's that we hold in our wallet.
this will allow us to clean up the NULL handling of missing
scriptpubkeys.
We're not using the change_outnum for withdraw tx's (and the way
we were calculating it was broken as of the addition of 'multiple
outputs'). This removes the change output knowhow from withdraw_tx
entirely, and pushes the responsibility up to the caller to
include the change output in the output set if desired.
Consequently, we also remove the change output knowhow from hsmd.
now that witness script data is saved into the tx/psbt which is
serialized across the wire, there's no reason to use witscript to do
this. good bye witscript!
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core
Changelog-None
ChangeLog-Added: New `getsharedsecret` command, which lets you compute a shared secret with this node knowing only a public point. This implements the BOLT standard of hashing the ECDH point, and is incompatible with ECIES.
Instead of making it ourselves, lightningd does it. Now we only have
two cases of failed htlcs: completely malformed (BADONION), and with
an already-wrapped onion reply to send.
This makes channeld's job much simpler.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This sets the nLockTime to the tip (and accordingly each input's nSequence to
0xfffffffe) for withdrawal transactions.
Even if the anti fee-sniping argument might not be valid until some time yet,
this makes our regular wallet transactions far less distinguishable from
bitcoind's ones since it now defaults to using native Segwit transactions
(like us). Moreover other wallets are likely to implement this (if they
haven't already).
Changelog-Added: wallet: withdrawal transactions now sets nlocktime to the current tip.
I was wondering why TAGS was missing some functions, and finally
tracked it down: PRINTF_FMT() confuses etags if it's at the start
of a function, and it ignores the rest of the file.
So we put PRINTF_FMT at the end, but that doesn't work for
*definitions*, only *declarations*. So we remove it from definitions
and add gratuitous declarations in the few static places.1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This splits maybe_create_hsm_secret() in two parts (either encrypted
or in clear) for clarity, and adds an encryption detection in load_hsm().
There are actually three cases if an encryption key is passed:
- There is no hsm_secret => just create it and store the encrypted seed
- There is an encrypted hsm_secret => the provided key should be able to
decrypt the seed, if the wrong key is passed libsodium will nicely error
and hsmd will exit() to not throw a backtrace (using status_failed() as for
other errors) at the face of an user who mistyped its password.
- There is a non-encrypted hsm_secret => load the seed, delete the
hsm_secret, create the hsm_secret, store the encrypted seed.
We now have a pointer to chainparams, that fails valgrind if we do anything
chain-specific before setting it.
Suggested-by: Rusty Russell <@rustyrussell>
If we are handling an elements transaction the value is not stored in the
satoshi field, rather it is stored in the `value` field which is prefixed with
a version (0x01) and is counted in `asset` units.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
531c8d7d9b
In this one, we always send my_current_per_commitment_point, though it's
ignored. And we have our official feature numbers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The largest change is inside hsmd: it hands a null per-commitment key
to the wallet to tell it to spend the to_remote output.
It can also now resolve unknown commitments, even if it doesn't have a
possible_remote_per_commitment_point from the peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is the other origin, besides `bitcoin_tx`, where we create `bitcoin_tx`
instances, so add the context as soon as possible. Sadly I can't weave the
chainparams into the deserialization code since that'd need to change all the
generated wire code as well.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
The way we build transactions, serialize them, and compute fees depends on the
chain we are working on, so let's add some context to the transactions.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
If we ever do this, we'd end up with an unspendable commitment tx anyway.
It might be able to happen if we have htlcs added from the non-fee-paying
party while the fees are increased, though. But better to close the
channel and get a report about it if that happens.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As a side-effect, we now only add txfilters for addresses we actually
expose, rather than always filtering for both p2sh and native segwit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This fixes block parsing on testnet; specifically, non-standard tx versions.
We hit a type bug in libwally (wallt_get_secp_context()) which I had to
work around for the moment, and the updated libsecp adds an optional hash
function arg to the ECDH function.
Fixes: #2563
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.
We didn't get much space saving in gossipd, even though we should save
24 bytes per node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Pubkeys are not not actually DER encoding, but Pieter Wuille corrected
me: it's SEC 1 documented encoding.
Results from 5 runs, min-max(mean +/- stddev):
store_load_msec,vsz_kb,store_rewrite_sec,listnodes_sec,listchannels_sec,routing_sec,peer_write_all_sec
38922-39297(39180.6+/-1.3e+02),2880728,41.040000-41.160000(41.106+/-0.05),2.270000-2.530000(2.338+/-0.097),44.570000-53.980000(49.696+/-3),32.840000-33.080000(32.95+/-0.095),43.060000-44.950000(43.696+/-0.72)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The `wally_tx_input`s do not keep track of their input value, which means we
need to track them ourselves if we try to sign these transactions at a later
point in time.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We set the version BIP32_VER_TEST_PRIVATE for testnet/regtest
BIP32 privkey generation with libwally-core, and set
BIP32_VER_MAIN_PRIVATE for mainnet.
For litecoin, we also set it like bitcoin else.
Basically we tell it that every field ending in '_msat' is a struct
amount_msat, and 'satoshis' is an amount_sat. The exceptions are
channel_update's fee_base_msat which is a u32, and
final_incorrect_htlc_amount's incoming_htlc_amt which is also a
'struct amount_msat'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As a side-effect of using amount_msat in gossipd/routing.c, we explicitly
handle overflows and don't need to pre-prune ridiculous-fee channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They're generally used pass-by-copy (unusual for C structs, but
convenient they're basically u64) and all possibly problematic
operations return WARN_UNUSED_RESULT bool to make you handle the
over/underflow cases.
The new #include in json.h means we bolt11.c sees the amount.h definition
of MSAT_PER_BTC, so delete its local version.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is mainly just copying over the copy-editing from the
lightning-rfc repository.
[ Split to just perform changes after the UNKNOWN_PAYMENT_HASH change --RR ]
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
It's more natural than using a zero-secret when something goes wrong.
Also note that the HSM will actually kill the connection if the ECDH
fails, which is fortunately statistically unlikely.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is prep work for when we sign htlc txs with
SIGHASH_SINGLE|SIGHASH_ANYONECANPAY.
We still deal with raw signatures for the htlc txs at the moment, since
we send them like that across the wire, and changing that was simply too
painful (for the moment?).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It means an extra allocation at startup, but it means we can hide the definition,
and use standard patterns (new_daemon_conn and typesafe callbacks).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Have c-lightning nodes send out the largest value for
`htlc_maximum_msat` that makes sense, ie the lesser of
the peer's max_inflight_htlc value or the total channel
capacity minus the total channel reserve.
BOLT 7's been updated to split the flags field in `channel_update`
into two: `channel_flags` and `message_flags`. This changeset does the
minimal necessary to get to building with the new flags.
We used to use it to complain about bad requests, but we use the status conn
now, so it's unused except for tests and asserts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It offers them a DoS vector, if they don't read the replies. We really want
to use raw ccan/io so we can avoid buffering for this.
It makes the handing of fds for new clients a bit more complex
(callback based), but it's not too bad.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Thanks greatly to the four people who I *know* have read this:
@wythe, @ZmnSCPxj, @SimonVrouwe, and @cdecker
Your feedback will help future developers seeking enlightenment!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
That matches the other CSV names (HSM was the first, so it was written
before the pattern emerged).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The current code sends hsmstatus_client_bad_request via the req fd;
this won't work, since lightningd uses that synchronously and only
expects a reply to its commands. So send it via status_conn.
We also enhance hsmstatus_client_bad_request to include details, and
create convenience functions for it. Our previous handling was ad-hoc;
we sometimes just closed on the client without telling lightningd,
and sometimes we didn't tell lightningd *which* client was broken.
Also make every handler the exact same prototype, so they now use the
exact same patterns (hsmd *only* handles requests, makes replies).
I tested this manually by corrupting a request to hsmd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
@renepickhardt: why is it actually lightningd.c with a d but hsm.c without d ?
And delete unused gossipd/gossip.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we make sure that we only have one fd per dbid, but more importantly
it enables leak detection, since we can iterate the clients we have.
If we get a second hsmfd request for the same dbid, we free the old
one: sometimes we get the new request before we notice the old daemon
died. We also have to handle the three 0-value dbid daemons a bit
specially.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. The handle pointer is always set to handle_client: just call direclty.
2. Call the root 'client' variable master.
3. We never exit the io_loop: we exit via master_gone instead, so cleanup there.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently it works for any secret (we don't know the current secret),
but importantly it doesn't leak timing information when checking.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I managed to crash the HSM by asking for point -1 (shachain_index has an
assert). Fail in this case, instead.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.
We shim tal_bytelen() for now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This will be used by onchaind for now, but also for openingd and channeld
in future, so it returns the old revocation secret as well.
Of course, the HSM should refuse to sign a commitment transaction if it
has handed out the revocation secret previously!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I crashed the HSMD, and it gave no output at all. That's because we
were only reading the status fd when we were waiting for a reply.
Fix this by using a separate request fd and status fd, which also means
that hsm_sync_read() is no longer required.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We need this later, to generate its seed. When we switch to lnd's key system,
we'll only need this, and not peerid.
Note also that the peerid is not just for messages any more, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Originally we were supposed to tell the HSM we had just created the directory,
otherwise it wouldn't create a new seed. But we modified it to check if
there was a seed file anyway: just move that logic into a branch of hsmd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In particular, the main daemon and subdaemons share the backtrace code,
with hooks for logging.
The daemon hook inserts the io_poll override, which means we no longer
need io_debug.[ch]. Though most daemons don't need it, they still link
against ccan/io, so it's harmess (suggested by @ZmnSCPxj).
This was tested manually to make sure we get backtraces still.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we're going to simply take() a pointer, don't allocate it off a random
object. Using NULL makes our intent clear, particularly with allocating
packets we're going to take() onto a queue.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now we have wirestring, this is much more natural. And with the
24M length limit, we needn't be so concerned about dumping 64k peer
messages in hex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These are now logically arrays of pointers. This is much more natural,
and gets rid of the horrible utxo array converters.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We log at a higher level when we have problems, and also just fail if
master behaves, rather than complaining and hanging.
Closes: #335
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
All other users of derive_simple_privkey(...) check the return value:
```
channeld/channel.c: if (!derive_simple_privkey(&peer->our_secrets.htlc_basepoint_secret,
lightningd/test/run-commit_tx.c: if (!derive_simple_privkey(&x_remote_htlc_basepoint_secret,
lightningd/test/run-commit_tx.c: if (!derive_simple_privkey(&x_local_delayed_payment_basepoint_secret,
lightningd/test/run-commit_tx.c: if (!derive_simple_privkey(&x_local_htlc_basepoint_secret,
lightningd/test/run-key_derive.c: if (!derive_simple_privkey(&base_secret, &base_point,
onchaind/onchain.c: if (!derive_simple_privkey(&secrets->delayed_payment_basepoint_secret,
onchaind/onchain.c: if (!derive_simple_privkey(&secrets->payment_basepoint_secret,
onchaind/onchain.c: if (!derive_simple_privkey(&secrets->htlc_basepoint_secret,
onchaind/onchain.c: if (!derive_simple_privkey(&secrets->payment_basepoint_secret,
onchaind/onchain.c: if (!derive_simple_privkey(&secrets->htlc_basepoint_secret,
```
Our handling of SIGPIPE was incoherent and inconsistent, and we had much
cut & paste between the daemons. They should *ALL* ignore SIGPIPE, and
much of the rest of the boilerplate can be shared, so should be.
Reported-by: @ZmnSCPxj
Fixes: #528
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
So far we have been generating the tx both in the HSM and in the
caller, and had to rely on them generating exactly the same
transaction. This makes it a lot simpler by fully signing and
serializing the TX on the HSM side and the caller just needs to unpack
and broadcast it.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Firstly, not every output is a P2SH (our change outputs aren't, and in
future we'll have native incoming segwit txs).
Secondly, withdraw_tx() permutes the utxo array, so we can't use a
temporary: we got away with it because we were always using the same
key!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Change all calls to use the correct serialization and deserialization
functions, include the correct headers and remove the control
messages.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We had a number of entry points into the HSM, all with different
behavior, so this is my attempt at unifying the way we handle
clients. Every client, except master, now takes the same path entry
point to the HSM and we use capability bit flags to indicate whether
the client is allowed to execute a set of operations.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This change is really to allow us to have a --dev-fail-on-subdaemon-fail option
so we can handle failures from subdaemons generically.
It also neatens handling so we can have an explicit callback for "peer
did something wrong" (which matters if we want to close the channel in
that case).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
