This is almost certainly because the HTLCs are not fully settled, so wait for that:
```
2023-06-20T11:37:56.2332158Z assert apys_2[0]['our_start_balance_msat'] == Millisatoshi(0)
2023-06-20T11:37:56.2332443Z > assert apys_1[0]['routed_out_msat'] == apys_2[0]['routed_in_msat']
2023-06-20T11:37:56.2332571Z E assert 1892216msat == 2810170msat
2023-06-20T11:37:56.2332580Z
2023-06-20T11:37:56.2332717Z tests/test_pay.py:81: AssertionError
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we reconnect before the channel is completely closed, we might get
a "reconnected" message, so mine a block after and make sure it's
processed.
```
2023-06-20T11:37:56.1302058Z if errors.has_errors():
2023-06-20T11:37:56.1302648Z # Format a nice list of everything that went wrong and raise an exception
2023-06-20T11:37:56.1303781Z request.node.has_errors = True
2023-06-20T11:37:56.1304091Z > raise ValueError(str(errors))
2023-06-20T11:37:56.1304370Z E ValueError:
2023-06-20T11:37:56.1304624Z E Node errors:
2023-06-20T11:37:56.1305042Z E - lightningd-2: had unexpected reconnections
2023-06-20T11:37:56.1305340Z E Global errors:
```
...
```
2023-06-20T11:37:56.1960525Z lightningd-2 2023-06-20T11:21:28.638Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#2: Peer has reconnected, state CLOSINGD_SIGEXCHANGE: connecting subd
```
I added a plugin arg and was surprised that compile didn't break.
This is because typesafe_cb et al are conditional casts: if the type
isn't as expected it has no effect, but we're passing plugin_option() through
varargs, so everything is accepted!
Add a noop inline to check type, and fix up the two cases where we
used `const char *` instead of `char *`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If it's a plugin opt, we'll need a callback, so reshuffle logic. Also
add infra to map option name to plugin and option.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Do it slightly intelligently, so if we had set previously using setconfig
we don't keep appending new ones, but replace it in-place.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Currently only implemented for min-capacity-sat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: new command `setconfig` allows a limited number of configuration settings to be changed without restart.
Previously, if these failed we always exited; once we have dymamic
configs this would be a (tiny) memory leak, so use tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
To test, we do min-capacity-sat which is simple. We also update the
listconfigs man page which contained some obsolete information.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We get "disconnected during connection" if we haven't finished processing
the connection when the peer sends error and hangs up.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We re-enable sendrawtransaction then mine a block to kick off RBF, but there's
a race where it can get a tx in that block, and then we timeout waiting for
both txs to get into the next block.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This isn't the cause of the test_gossip_ratelimit flake I saw (since
the final gossip msg clearly was received), but it's still good to fix
since it means we might not send the final messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were seeing hangs in disconnect in
tests/test_connection.py::test_feerate_stress, and looking at the logs
it's because we're not actually telling connectd to disconnect.
These days, we have a connect counter, so connectd knows to ignore it
if we simply haven't read the message about it already disconnecting.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
build.sh did this, but we didn't carry it across to the new test
matrix, leading to spurious CI failures:
```
lightningd-1 2023-06-06T16:15:34.931Z DEBUG plugin-bookkeeper: Setting up database at ***localhost:39061/accounts_0_7ccmg745
lightningd-1 2023-06-06T16:15:34.931Z INFO plugin-bookkeeper: Creating database
lightningd-1 2023-06-06T16:15:34.931Z **BROKEN** plugin-bookkeeper: Error vacuuming db: VACUUM command failed: ERROR: deadlock detected\nDETAIL: Process 77248 waits for AccessShareLock on relation 1260 of database 0; blocked by process 77414.\nProcess 77414 waits for RowExclusiveLock on relation 1214 of database 0; blocked by process 77248.\nHINT: See server log for query details.\n
lightningd-1 2023-06-06T16:15:34.931Z INFO plugin-bookkeeper: Killing plugin: exited before replying to init
lightningd-1 2023-06-06T16:15:34.931Z **BROKEN** plugin-bookkeeper: Plugin marked as important, shutting down lightningd!
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Clang's coverage instrumentation [1] is the best I've seen, with
precision down to the expressions within a line of code. Add an option
to use this instrumentation for better coverage reports.
[1] https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
This fixes the compile issue that we are having on
alpine.
```
cc wallet/wallet.c
cc wallet/walletrpc.c
cc wallet/reservation.c
cc wallet/db_sqlite3_sqlgen.c
cc wallet/db_postgres_sqlgen.c
cc common/addr.c
cc common/bolt11.c
cc common/bolt11_json.c
cc common/bolt12.c
cc common/configdir.c
cc common/configvar.c
cc common/scb_wiregen.c
common/configvar.c: In function 'configvar_remove':
common/configvar.c:118:9: error: unknown type name 'ssize_t'; did you mean 'size_t'?
118 | ssize_t prev = -1;
| ^~~~~~~
| size_t
make: *** [Makefile:292: common/configvar.o] Error 1
make: *** Waiting for unfinished jobs....
```
Link: https://github.com/ElementsProject/lightning/issues/6321
Reported-by: @gruve-p
Fixes: 36200a6593
Changelog-None
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
I noted in some PR that we are failing during the diff of the autogenerated files.
So this will generate the last version of them!
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
When core lightning is asking the information about
the blockchain with `getchaininfo` command lightningd
know already the information about the min and max block height.
the problem is when we have a smarter Bitcoin backend that is able
to switch between different clients in some cases is helpful
give the information about current known height by lightningd and
pass it down to the plugin.
In this way, the plugin knows what is the correct known height from lightnind, and can
try to fix some problems if any exit.
This is particularly useful when you are syncing a new backend from scratch
like https://github.com/cloudhead/nakamoto and we avoid returning the
lower height from the known, and avoid the crash of core lightning.
With this information, the plugin can start to sync the chain and return
the answer back only when the chain is in sync with the current status of
lightningd.
Another reason to add this field and not wait the correct block in core
lightning itself is because Bitcoin Core is extremely slow to sync up,
so the question here is, how long should we wait? The time depends
on various factors.
With this approach of informing the plugin about the height, in some cases,
you can start the syncing but move the execution to another backend until
the previous one is ready.
The problem I want to solve is that I don't want to be left in the dark when
we run `getchaininfo`, and I want to have the opportunity to wait for
the blockchain sync or decide to dispatch the request elsewhere.
Changelog-Added: Pass the current known block height down to the getchaininfo call.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
option_scid_alias inside a channel_type allows for more private
channels: in particular, it tells the peer that it MUST NOT allow
routing via the real short channel id, and MUST use the alias.
It only makes sense (and is only permitted!) on unannounced channels.
Unfortunately, we didn't set this bit in the channel_type in v12.0
when it was introduced, instead relying on the presence of the feature
bit with the peer. This was fixed in 23.05, but:
1. Prior to 23.05 we didn't allow it to be set at all, and
2. LND has a limited set of features they allow, and this isn't allowed without
option_anchors_zero_fee_htlc_tx.
We could simply drop this channel_type until we merge anchors, *but*
that has nasty privacy implications (you can probe the real channel id).
So, if we don't negotiate anchors (we don't!), we don't set this
channel_type bit even if we want it, and *intuit* it, based on:
1. Is this a non-anchor channel_type?
2. Did we both send channel_type?
3. Is this an unannounced channel?
4. Did both peers announce support for scid aliases?
In addition, while looking at the previous backwards-compat code, I
realized that v23.05 violated the spec and send accept_channel with
OPT_SCID_ALIAS if it intuited it, even if it wasn't offered. Stop
doing this, but allow our peers to.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Fix incompatibility with LND which prevented us opening private channels
Fixes: #6208
Due to a security report by github, we should increase
our cryptography lib version.
This may impact potential another version that is stuck
with a cryptography version.
Link: https://github.com/ElementsProject/lightning/issues/6164
Reported-by: @dni
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Detected by ASan in test_hsmtool_generatehsm:
==58698==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 120 byte(s) in 1 object(s) allocated from:
#0 0x4e6247 in malloc
#1 0x7f078452d672 in getdelim
SUMMARY: AddressSanitizer: 120 byte(s) leaked in 1 allocation(s).
defaults were deprecated in 0df97547dd, but that was a bit
harsh as several plugins do that (summary, for example). So allow false, but warn
that we ignore anything else.
Reported-by: @microsatoshi on Discord.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: ...actually, `default` `false` still accepted on `flag` type parameters.
Previously any attempt would result in "is not an integer field"; we
now recognize valid JSON integers as integer fields.
Changelog-Fixed: Plugins: `commando` runes can now compare integer parameters using '<' and '>' as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This PR:
- adds all the guides (in markdown format) that is published at https://docs.corelightning.org/docs
- adds a github workflow to sync any future changes made to files inside the guides folder
- does not include API reference (json-rpc commands). Those will be handled in a separate PR since they're used as manpages and will require a different github workflow
Note that the guides do not exactly map to their related files in doc/, since we reorganized the overall documentation structure on readme for better readability and developer experience. For example, doc/FUZZING.md and doc/HACKING.md#Testing are merged into testing.md in the new docs. As on the creation date of this PR, content from each of the legacy documents has been synced with the new docs. Until this PR gets merged, I will continue to push any updates made to the legacy documents into the new docs.
If this looks reasonable, I will add a separate PR to clean up the legacy documents from doc/ (or mark them deprecated) to avoid redundant upkeep and maintenance.
Changelog-None
This is a simplification for our build system
that allows for managing and cleaning the external
build artefacts in an easy way.
We have a hard time with the suffix `-modded` inside the
version with some architecture when building a tagged version.
Link: https://discord.com/channels/899980449231814676/899989729183940629/1110957992158965770
Reported-by: @ctrlbreak-
Suggested-by: Rusty Russell <rusty@rustcorp.com.au>
Co-developed-by: @jsarenik
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
