This matters: if we connected, the address is probably usable for future connections.
But if they connected, the port is probably not (but the IP address may be).
Changelog-Added: JSON-RPC: `connect` returns "direction" ("in": they iniatated, or "out": we initiated)
Changelog-Added: plugins: `peer_connected` hook and `connect` notifications have "direction" field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
We should actually be including this (as it may define _GNU_SOURCE
etc) before any system headers. But where we include <assert.h> we
often didn't, because check-includes would complain that the headers
included it too.
Weaken that check, and include config.h in C files before assert.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Took me a while (stressing under valgrind) to reproduce this,
then longer to figure out how it happened.
Turns out io_new_conn() can fail if the init function fails.
In our case, this can happen if connect() immediately returns
an error (inside io_connect). But we've already set the finish
function, which (if this was the last address), will free connect,
making the assignment `connect->conn = ...` write to a freed address.
Either way, if it fails, try_connect_one_addr() has taken care to
update connect->conn, or free connect, and the caller should not do it.
Here's the valgrind trace:
```
==384981== Invalid write of size 8
==384981== at 0x11127C: try_connect_one_addr (connectd.c:880)
==384981== by 0x112BA1: destroy_io_conn (connectd.c:708)
==384981== by 0x141459: destroy_conn (poll.c:244)
==384981== by 0x14147F: destroy_conn_close_fd (poll.c:250)
==384981== by 0x149EB9: notify (tal.c:240)
==384981== by 0x149F8B: del_tree (tal.c:402)
==384981== by 0x14A51A: tal_free (tal.c:486)
==384981== by 0x140036: io_close (io.c:450)
==384981== by 0x1400B3: do_plan (io.c:401)
==384981== by 0x140134: io_ready (io.c:423)
==384981== by 0x141A57: io_loop (poll.c:445)
==384981== by 0x112CB0: main (connectd.c:1703)
==384981== Address 0x4d67020 is 64 bytes inside a block of size 160 free'd
==384981== at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==384981== by 0x14A020: del_tree (tal.c:421)
==384981== by 0x14A51A: tal_free (tal.c:486)
==384981== by 0x1110C5: try_connect_one_addr (connectd.c:806)
==384981== by 0x112BA1: destroy_io_conn (connectd.c:708)
==384981== by 0x141459: destroy_conn (poll.c:244)
==384981== by 0x14147F: destroy_conn_close_fd (poll.c:250)
==384981== by 0x149EB9: notify (tal.c:240)
==384981== by 0x149F8B: del_tree (tal.c:402)
==384981== by 0x14A51A: tal_free (tal.c:486)
==384981== by 0x140036: io_close (io.c:450)
==384981== by 0x1405DC: io_connect_ (io.c:345)
==384981== Block was alloc'd at
==384981== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==384981== by 0x149CF1: allocate (tal.c:250)
==384981== by 0x14A3C6: tal_alloc_ (tal.c:428)
==384981== by 0x1114F2: try_connect_peer (connectd.c:1526)
==384981== by 0x111717: connect_to_peer (connectd.c:1558)
==384981== by 0x1124F5: recv_req (connectd.c:1627)
==384981== by 0x1188B2: handle_read (daemon_conn.c:31)
==384981== by 0x13FBCB: next_plan (io.c:59)
==384981== by 0x140076: do_plan (io.c:407)
==384981== by 0x140113: io_ready (io.c:417)
==384981== by 0x141A57: io_loop (poll.c:445)
==384981== by 0x112CB0: main (connectd.c:1703)
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Occasional crash in connectd due to use-after-free
Fixes: #4343
This is stolen from the bitcoind source, so we use their style
(no tabs, 4-wide spaces). Clean it up and add the emacs magic to
maintain it in future.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
As reported by Wladimir J. van der Laan. Valgrind will complain
about padding and unset fields, so memset the structs.
```
==42653== Syscall param socketcall.connect(serv_addr..sa_len) points to uninitialised byte(s)
==42653== at 0x4C7D19A: _connect (in /lib/libc.so.7)
==42653== by 0x4EE1F35: ??? (in /lib/libthr.so.3)
==42653== by 0x249D57: get_local_sockname (netaddress.c:212)
==42653== by 0x249CDB: guess_address (netaddress.c:242)
==42653== by 0x2473D0: public_address (connectd.c:1003)
==42653== by 0x246CE4: setup_listeners (connectd.c:0)
==42653== by 0x246566: connect_init (connectd.c:1311)
==42653== by 0x270CEB: next_plan (io.c:59)
==42653== by 0x2713EE: io_ready (io.c:417)
==42653== by 0x2726B1: io_loop (poll.c:445)
==42653== by 0x24618A: main (connectd.c:1703)
==42653== Address 0x7fc000690 is on thread 1's stack
==42653== in frame #3, created by guess_address (netaddress.c:231)
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!
Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Avoids much cut & paste. Some tests don't need any of it, but most
want at least some of this infrastructure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we're already attempting to connect to a peer, we would ignore
new connection requests. This is problematic if your node has bad
connection details for the node -- you can't update it while inflight.
This patch appends new connection suggestions to the list of connections
to try.
Fixes#4154
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.
This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is simple, and we now can multifundchannel to every node on testnet
(one simply hangs once we connect).
Changelog-Fixed: Protocol: We now hang up if peer doesn't respond to init message after 60 seconds.
And add secp_recovery to headers, while we're at it.
```
./wire/wire.h:7:10: fatal error: secp256k1_recovery.h: No such file or directory
#include <secp256k1_recovery.h>
^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:254: recipe for target 'connectd/test/run-initiator-success.o' failed
make: *** [connectd/test/run-initiator-success.o] Error 1
m
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
dual funding needs the max-witness-len and utxo fields set for every
input. we should add them when we create a 'fundpsbt', so that every
psbt that c-lightning generates is dual-funding ready
There's a lot of it, and it means we can't `make check-source` on
these files.
Also bring bolt quotes up-to-date.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids overwriting the ones in git, and generally makes things neater.
We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're going to make experimental versions of these completely separate files.
Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn. We can always force-rebuild when we change a rule.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS. Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This simplifies our test matrix, as we never have to handle talking
to peers that specify one but not the other.
This is particularly important for option_anchor_outputs which
assumes option_static_remotekey.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't preserve detailed asset information at the moment, so provide a
way to convert from a sat to an amount_asset struct.
We also need a way to convert from an 'amount_asset' to a 'value' for
elements, which for explicit (i.e. non-blinded) asssets is a 0x01 prefix
plus the big-endian encoded value.
@thestick613 noticed that since tor version below 0.3.2.2-alpha
will not support V3 ed25519 address formats, the error handling
is not that helpful in the error message from cli.
So now we add an hint.
Changelog-None:
Signed-off-by: Saibato <saibato.naga@pm.me>
connectd/connectd.h; Add helper function to update conn error list
Signed-off-by: Saibato <saibato.naga@pm.me>
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)
Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
It returns NULL, so you can simply `return fromwire_fail(...)`
if you want to return NULL in this case. Use that more.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core
Changelog-None
We did this originally because these types are referred to in the bolts, and we
had no way of injecting the correct include lines into those. Now we do, so
there's less excuse for this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When we have only a single member in a TLV (e.g. an optional u64),
wrapping it in a struct is awkward. This changes it to directly
access those fields.
This is not only more elegant (60 fewer lines), it would also be
more cache friendly. That's right: cache hot singles!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that it's channeld which calculates the shared secret, too. This
minimizes the work that lightningd has to do, at cost of passing this
through.
We also don't yet save the blinding field(s) to the database.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
common/onion is going to need to use this for the case where it finds a blinding
seed inside the TLV. But how it does ecdh is daemon-specific.
We already had this problem for devtools/gossipwith, which supplied a
special hsm_do_ecdh(). This just makes it more general.
So we create a generic ecdh() interface, with a specific implementation
which subdaemons and lightningd can use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's almost always "their_features" and "our_features" respectively, so
make those names clear.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Turns out that unnecessary: all callers can access the feature_set,
so make it much more like a normal primitive.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This will help with the next patch, where we wean off using a global
for features: connectd.c has access to the feature bits.
Since connectd might now want to send a message, it needs the crypto_state
non-const, which makes this less trivial than it would otherwise be.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is to prepare for dynamic features, including making plugins first
class citizens at setting them.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This restriction was removed from the spec as of
86c2ebcc5973a4133d3ce4d80ae1c203061a1646.
We also fix up some strange formatting in that part of the documentation.
Changelog-changed: We now announce multiple addresses of the same type, if given.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a common thing to do, so create a macro.
Unfortunately, it still needs the type arg, because the paramter may
be const, and the return cannot be, and C doesn't have a general
"(-const)" cast.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The `init_featurebits` are computed at startup, and then cached
indefinitely. They are then used whenever a new `init` handshake is performed.
We could add a new message to push updates to `connectd` whenever a plugin is
added or removed, but that's up for discussion.
GCC 10 defaults to `-fno-common`. no longer automatically sharing
global variable definitions, which makes it important to define
them in only one place (otherwise there will be duplicate definition
errors). Add `extern` qualifiers where (I think) is the best place for
them.
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.
To resolve this:
* Introduce an error code type with a known size:
`typedef s32 errcode_t`.
* Change all error code macros to constants of type `errcode_t`.
Constants also play better with gdb - it would visualize the name of
the constant instead of the numeric value.
* Change all functions that take error codes to take the new type
`errcode_t` instead of `int`.
* Introduce towire / fromwire functions to send / receive the newly added
type `errcode_t` and use it instead of `towire_int()`.
In addition:
* Remove the now unneeded `towire_int()`.
* Replace a hardcoded error code `-2` with a new constant
`INVOICE_EXPIRED_DURING_WAIT` (903).
Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
This makes it clear we're dealing with a message which is a wrapped error
reply (needing unwrap_onionreply), not an already-wrapped one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Add towire_int() and fromwire_int() functions to "(de)serialize"
"int". This will only work as long as both the caller of towire_int()
and the caller of fromwire_int() use the same in-memory representation
of signed integers and have the same sizeof(int).
Changelog-None
The tor proxy might want auth 0x2 or answer what ever not defined
in https://tools.ietf.org/html/rfc1928.
Changelog-Fixed: TOR: We don't send any further request if the return code of connect is not zero or error.
Signed-off-by: Saibato <saibato.naga@pm.me>
For example:
"Tor connect out for host example.com error: 4 "
becomes:
"Error connecting to example.com: Tor server reply: host unreachable"
Also clarify the error:
"Connected out for example.com error"
to:
"Connected out for example.com error: authentication required"
because the only reason it could have failed is that the Tor socks5
server did not like any of our proposed auth methods and we only
proposed "no auth".
Changelog-None
We want to have a static Tor service created from a blob bound to
our node on cmdline
Changelog-added: persistent Tor address support
Changelog-added: allow the Tor inbound service port differ from 9735
Signed-off-by: Saibato <saibato.naga@pm.me>
Add base64 encode/decode to common
We need this to encode the blob for the tor service
Signed-off-by: Saibato <saibato.naga@pm.me>
The spec is (RSN!) going to explicitly denote where each feature should
be presented, so create that infrastructure.
Incorporate the new proposed bolt11 features, which need this.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was decided at a recent spec meeting: in particular, mpp and
var_onion_optin options will be used here.
We enhanced "features_supported" into "features_unsupported" so it
can return the first un-handlable bit number.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is ignored in subdaemons which are per-peer, but very useful for
multi-peer daemons like connectd and gossipd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Even if it is on startup only once ...
Like @bitcoin-software indicated the expected UX should be in
line with what a user expects the software will do
so we should not dns if we say so with a flag that suggest that.
Changelog-Fixed: We disable all dns even on startup the scan for bogus dns servers, if --always-use-proxy is set true
Signed-off-by: Saibato <saibato.naga@pm.me>
I was wondering why TAGS was missing some functions, and finally
tracked it down: PRINTF_FMT() confuses etags if it's at the start
of a function, and it ignores the rest of the file.
So we put PRINTF_FMT at the end, but that doesn't work for
*definitions*, only *declarations*. So we remove it from definitions
and add gratuitous declarations in the few static places.1
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were using `i` as index variable in two nested loops. This works as long as
the DNS seed resolves to a single address, but will crash if the node has both
an A as well as an AAAA entry, at which point we'll try to index the hostname
without a matching entry.
Signed-off-by: Christian Decker <@cdecker>
This is mainly an internal-only change, especially since we don't
offer any globalfeatures.
However, LND (as of next release) will offer global features, and also
expect option_static_remotekey to be a *global* feature. So we send
our (merged) feature bitset as both global and local in init, and fold
those bitsets together when we get an init msg.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently the only source for amount_asset is the value getter on a tx output,
and we don't hand it too far around (mainly ignoring it if it isn't the
chain's main currency). Eventually we could bubble them up to the wallet, use
them to select outputs or actually support assets in the channels.
Since we don't hand them around too widely I thought it was ok for them to be
pass-by-value rather than having to allocate them and pass them around by
reference. They're just 41 bytes currently so the overhead should be ok.
Signed-off-by: Christian Decker <@cdecker>
