Core Lightning — Lightning Network implementation focusing on spec compliance and performance
Go to file
Rusty Russell 4ee55acc71 connectd: don't start connecting in parallel in peer_conn_closed.
The crash below from @zerofeerouting left me confused.  The invalid
value in fmt_wireaddr_internal is a telltale sign of use-after-free.

This backtrace shows us destroying the conn *twice*: what's happening?

Well, tal carefully protects against destroying twice: it's not that
unusual to free something in a destructor which has already been freed.
So this indicates that there are *two* io_conn hanging off one
struct connecting, which isn't supposed to happen!  We deliberately
call try_connect_one_addr() initially, then inside the io_conn destructor.

But due to races in connectd vs lightningd connection state, we added
a fix which allows a connect command to sit around while the peer is
cleaning up (6cc9f37cab) and get fired
off when it's done.

But what if, in the chaos, we are already connecting again?  Now we'll
end up with *two* connections.

Fortunately, we have a `conn` pointer inside struct connecting, which
(with a bit of additional care) we can ensure is only non-NULL while
we're actually trying to connect.  This lets us check that before
firing off a new connection attempt in peer_conn_closed.

```
lightning_connectd: FATAL SIGNAL 6 (version v0.11.2rc2-2-g8f7e939)
0x5614a4915ae8 send_backtrace
	common/daemon.c:33
0x5614a4915b72 crashdump
	common/daemon.c:46
0x7ffa14fcd72f ???
	???:0
0x7ffa14dc87bb ???
	???:0
0x7ffa14db3534 ???
	???:0
0x5614a491fc71 fmt_wireaddr_internal
	common/wireaddr.c:255
0x5614a491fc7a fmt_wireaddr_internal_
	common/wireaddr.c:257
0x5614a491ea6b type_to_string_
	common/type_to_string.c:32
0x5614a490beaa destroy_io_conn
	connectd/connectd.c:754
0x5614a494a2f1 destroy_conn
	ccan/ccan/io/poll.c:246
0x5614a494a313 destroy_conn_close_fd
	ccan/ccan/io/poll.c:252
0x5614a4953804 notify
	ccan/ccan/tal/tal.c:240
0x5614a49538d6 del_tree
	ccan/ccan/tal/tal.c:402
0x5614a4953928 del_tree
	ccan/ccan/tal/tal.c:412
0x5614a4953e07 tal_free
	ccan/ccan/tal/tal.c:486
0x5614a4908b7a try_connect_one_addr
	connectd/connectd.c:870
0x5614a490bef1 destroy_io_conn
	connectd/connectd.c:759
0x5614a494a2f1 destroy_conn
	ccan/ccan/io/poll.c:246
0x5614a494a313 destroy_conn_close_fd
	ccan/ccan/io/poll.c:252
0x5614a4953804 notify
	ccan/ccan/tal/tal.c:240
0x5614a49538d6 del_tree
	ccan/ccan/tal/tal.c:402
0x5614a4953e07 tal_free
	ccan/ccan/tal/tal.c:486
0x5614a4948f08 io_close
	ccan/ccan/io/io.c:450
0x5614a4948f59 do_plan
	ccan/ccan/io/io.c:401
0x5614a4948fe1 io_ready
	ccan/ccan/io/io.c:417
0x5614a494a8e6 io_loop
	ccan/ccan/io/poll.c:453
0x5614a490c12f main
	connectd/connectd.c:2164
0x7ffa14db509a ???
	???:0
0x5614a4904e99 ???
	???:0
0xffffffffffffffff ???
	???:0
```

Fixes: #5339
Changelog-Fixed: connectd: occasional crash when we reconnect to a peer quickly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-28 13:46:59 +09:30
.github pytest: Remove all trace of python's "flaky" module. 2022-06-27 17:21:35 +09:30
bitcoin lightnind: removeDEFAULT_PORT global definition 2022-06-28 06:09:01 +09:30
ccan ccan: update to get io_sock_shutdown 2022-01-20 15:24:06 +10:30
channeld channeld: take over gossip_rcvd_filter.c and is_msg_gossip_broadcast. 2022-06-17 14:14:02 +09:30
cli psbt: use DER encoded + sighash byte for PSBT_IN_PARTIAL_SIG items 2022-06-09 18:28:35 +02:00
cln-grpc lightningd: deprecate msatoshi in sendpay route. 2022-06-21 06:52:35 +09:30
cln-rpc Remove native-tls dependency from cln-rpc 2022-06-23 15:45:06 +02:00
closingd channeld: take over gossip_rcvd_filter.c and is_msg_gossip_broadcast. 2022-06-17 14:14:02 +09:30
common lightnind: removeDEFAULT_PORT global definition 2022-06-28 06:09:01 +09:30
connectd connectd: don't start connecting in parallel in peer_conn_closed. 2022-06-28 13:46:59 +09:30
contrib pytest: fix gossipwith flake harder. 2022-06-27 17:21:35 +09:30
db db: small code cleanup 2022-03-11 16:43:15 +10:30
devtools devtools: revert changes and make sure chainparams always set. 2022-06-28 06:09:01 +09:30
doc doc: document the [] IPv6 address hack. 2022-06-28 13:46:41 +09:30
external external/Makefile: fix submodcheck for sudo make install and modern git. 2022-05-03 09:08:33 +09:30
gossipd lightnind: removeDEFAULT_PORT global definition 2022-06-28 06:09:01 +09:30
hsmd Update hsm error formatting. 2022-06-24 13:57:17 +09:30
lightningd lightnind: removeDEFAULT_PORT global definition 2022-06-28 06:09:01 +09:30
onchaind Makefile: update bolts to include remote_pubkey change. 2022-05-19 09:47:32 +09:30
openingd channeld: take over gossip_rcvd_filter.c and is_msg_gossip_broadcast. 2022-06-17 14:14:02 +09:30
plugins cln-plugin: Make the proxy-related configuration Option<> 2022-06-22 12:07:55 +02:00
tests devtools: revert changes and make sure chainparams always set. 2022-06-28 06:09:01 +09:30
tools docker: Update name from c-lightning to Core-Lightning 2022-05-14 11:43:46 +09:30
wallet lightnind: removeDEFAULT_PORT global definition 2022-06-28 06:09:01 +09:30
wire lightnind: removeDEFAULT_PORT global definition 2022-06-28 06:09:01 +09:30
.clang-format tools: Added .clang-format for formatting 2018-11-29 23:01:11 +00:00
.dir-locals.el emacs: add .dir-locals.el for linux style C 2018-01-10 04:01:56 +00:00
.editorconfig Travis is no longer used 2022-04-07 06:54:02 +09:30
.gitattributes database: pull out database code into a new module 2022-03-05 15:03:34 +10:30
.gitignore git: Ignore some more generated files and the cln-grpc plugin 2022-06-24 16:27:13 +02:00
.gitlab-ci.yml Add .gitlab-ci.yml 2016-12-11 13:24:27 +01:00
.gitmodules Update at latest version of lnprototest with the support of channel_type 2021-06-25 09:51:29 +09:30
.msggen.json lightningd: change msatoshi args to amount_msat. 2022-06-21 06:52:35 +09:30
action.yml doc: update c-lightning to Core Lightning almost everywhere. 2022-04-07 06:53:26 +09:30
Cargo.toml cln-grpc-plugin: Add scaffolding for the cln-grpc-plugin 2022-03-30 12:15:55 +10:30
ccan_compat.h check: make sure all files outside contrib/ include "config.h" first. 2021-12-06 10:05:39 +10:30
CHANGELOG.md CHANGELOG.md: release notes for 0.11.1. 2022-05-16 09:59:42 +09:30
configure "removed asterisks, in case that's what made build fail" 2022-05-01 14:36:20 +09:30
Dockerfile Fixes #5276 by using as docker base image debian bullseye instead of buster 2022-05-22 14:25:05 +02:00
LICENSE Update LICENSE 2022-03-10 10:23:40 +10:30
Makefile contrib/pylightning: remove lightning-pay helper. 2022-06-21 06:52:35 +09:30
poetry.lock pytest: Remove all trace of python's "flaky" module. 2022-06-27 17:21:35 +09:30
pyproject.toml pytest: Remove all trace of python's "flaky" module. 2022-06-27 17:21:35 +09:30
README.md README: add links to discord + telegram so people can easily find us! 2022-05-17 10:18:10 +09:30

Core Lightning (CLN): A specification compliant Lightning Network implementation in C

Core Lightning (previously c-lightning) is a lightweight, highly customizable and standard compliant implementation of the Lightning Network protocol.

Project Status

Continuous Integration Pull Requests Welcome Irc Documentation Status

This implementation has been in production use on the Bitcoin mainnet since early 2018, with the launch of the Blockstream Store. We recommend getting started by experimenting on testnet (or regtest), but the implementation is considered stable and can be safely used on mainnet.

Any help testing the implementation, reporting bugs, or helping with outstanding issues is very welcome. Don't hesitate to reach out to us on IRC at #lightning-dev @ libera.chat, #c-lightning @ libera.chat, or on the implementation-specific mailing list c-lightning@lists.ozlabs.org, or on the Lightning Network-wide mailing list lightning-dev@lists.linuxfoundation.org, or on Discord core-lightning, or on Telegram Core Lightning.

Getting Started

Core Lightning only works on Linux and Mac OS, and requires a locally (or remotely) running bitcoind (version 0.16 or above) that is fully caught up with the network you're running on, and relays transactions (ie with blocksonly=0). Pruning (prune=n option in bitcoin.conf) is partially supported, see here for more details.

Installation

There are 4 supported installation options:

For the impatient here's the gist of it for Ubuntu:

sudo apt-get install -y software-properties-common
sudo add-apt-repository -u ppa:lightningnetwork/ppa
sudo apt-get install lightningd snapd
sudo snap install bitcoin-core
sudo ln -s /snap/bitcoin-core/current/bin/bitcoin{d,-cli} /usr/local/bin/

Starting lightningd

Regtest (local, fast-start) Option

If you want to experiment with lightningd, there's a script to set up a bitcoind regtest test network of two local lightning nodes, which provides a convenient start_ln helper. See the notes at the top of the startup_regtest.sh file for details on how to use it.

. contrib/startup_regtest.sh

Note that your local nodeset will be much faster/more responsive if you've configured your node to expose the developer options, e.g.

./configure --enable-developer

Mainnet Option

To test with real bitcoin, you will need to have a local bitcoind node running:

bitcoind -daemon

Wait until bitcoind has synchronized with the network.

Make sure that you do not have walletbroadcast=0 in your ~/.bitcoin/bitcoin.conf, or you may run into trouble. Notice that running lightningd against a pruned node may cause some issues if not managed carefully, see below for more information.

You can start lightningd with the following command:

lightningd --network=bitcoin --log-level=debug

This creates a .lightning/ subdirectory in your home directory: see man -l doc/lightningd.8 (or https://lightning.readthedocs.io/) for more runtime options.

Using The JSON-RPC Interface

Core Lightning exposes a JSON-RPC 2.0 interface over a Unix Domain socket; the lightning-cli tool can be used to access it, or there is a python client library.

You can use lightning-cli help to print a table of RPC methods; lightning-cli help <command> will offer specific information on that command.

Useful commands:

  • newaddr: get a bitcoin address to deposit funds into your lightning node.
  • listfunds: see where your funds are.
  • connect: connect to another lightning node.
  • fundchannel: create a channel to another connected node.
  • invoice: create an invoice to get paid by another node.
  • pay: pay someone else's invoice.
  • plugin: commands to control extensions.

Care And Feeding Of Your New Lightning Node

Once you've started for the first time, there's a script called contrib/bootstrap-node.sh which will connect you to other nodes on the lightning network.

There are also numerous plugins available for Core Lightning which add capabilities: in particular there's a collection at:

https://github.com/lightningd/plugins

Including helpme which guides you through setting up your first channels and customizing your node.

For a less reckless experience, you can encrypt the HD wallet seed: see HD wallet encryption.

You can also chat to other users at #c-lightning @ libera.chat; we are always happy to help you get started!

Opening A Channel

First you need to transfer some funds to lightningd so that it can open a channel:

# Returns an address <address>
lightning-cli newaddr

lightningd will register the funds once the transaction is confirmed.

You may need to generate a p2sh-segwit address if the faucet does not support bech32:

# Return a p2sh-segwit address
lightning-cli newaddr p2sh-segwit

Confirm lightningd got funds by:

# Returns an array of on-chain funds.
lightning-cli listfunds

Once lightningd has funds, we can connect to a node and open a channel. Let's assume the remote node is accepting connections at <ip> (and optional <port>, if not 9735) and has the node ID <node_id>:

lightning-cli connect <node_id> <ip> [<port>]
lightning-cli fundchannel <node_id> <amount_in_satoshis>

This opens a connection and, on top of that connection, then opens a channel. The funding transaction needs 3 confirmation in order for the channel to be usable, and 6 to be announced for others to use. You can check the status of the channel using lightning-cli listpeers, which after 3 confirmations (1 on testnet) should say that state is CHANNELD_NORMAL; after 6 confirmations you can use lightning-cli listchannels to verify that the public field is now true.

Sending and Receiving Payments

Payments in Lightning are invoice based. The recipient creates an invoice with the expected <amount> in millisatoshi (or "any" for a donation), a unique <label> and a <description> the payer will see:

lightning-cli invoice <amount> <label> <description>

This returns some internal details, and a standard invoice string called bolt11 (named after the BOLT #11 lightning spec).

The sender can feed this bolt11 string to the decodepay command to see what it is, and pay it simply using the pay command:

lightning-cli pay <bolt11>

Note that there are lower-level interfaces (and more options to these interfaces) for more sophisticated use.

Configuration File

lightningd can be configured either by passing options via the command line, or via a configuration file. Command line options will always override the values in the configuration file.

To use a configuration file, create a file named config within your top-level lightning directory or network subdirectory (eg. ~/.lightning/config or ~/.lightning/bitcoin/config). See man -l doc/lightningd-config.5.

Further information

Pruning

Core Lightning requires JSON-RPC access to a fully synchronized bitcoind in order to synchronize with the Bitcoin network. Access to ZeroMQ is not required and bitcoind does not need to be run with txindex like other implementations. The lightning daemon will poll bitcoind for new blocks that it hasn't processed yet, thus synchronizing itself with bitcoind. If bitcoind prunes a block that Core Lightning has not processed yet, e.g., Core Lightning was not running for a prolonged period, then bitcoind will not be able to serve the missing blocks, hence Core Lightning will not be able to synchronize anymore and will be stuck. In order to avoid this situation you should be monitoring the gap between Core Lightning's blockheight using lightning-cli getinfo and bitcoind's blockheight using bitcoin-cli getblockchaininfo. If the two blockheights drift apart it might be necessary to intervene.

HD wallet encryption

You can encrypt the hsm_secret content (which is used to derive the HD wallet's master key) by passing the --encrypted-hsm startup argument, or by using the hsmtool (which you can find in the tool/ directory at the root of this repo) with the encrypt method. You can unencrypt an encrypted hsm_secret using the hsmtool with the decrypt method.

If you encrypt your hsm_secret, you will have to pass the --encrypted-hsm startup option to lightningd. Once your hsm_secret is encrypted, you will not be able to access your funds without your password, so please beware with your password management. Also, beware of not feeling too safe with an encrypted hsm_secret: unlike for bitcoind where the wallet encryption can restrict the usage of some RPC command, lightningd always needs to access keys from the wallet which is thus not locked (yet), even with an encrypted BIP32 master seed.

Developers

Developers wishing to contribute should start with the developer guide here. You should also configure with --enable-developer to get additional checks and options.