Commit Graph

14107 Commits

Author SHA1 Message Date
Erik De Smedt
557b5fee46 Allow dynamic option in python plugin 2023-12-14 16:48:02 +01:00
Christian Decker
b6c486c2ba gha: Add --skip-existing option to publish to testpypi
Changelog-None
2023-12-14 13:03:55 +01:00
Rusty Russell
36b631699b plugins: re-enable listchannels local info in deprecated mode.
We only deprecated this, we didn't actually remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
c8e30de513 topology: deprecate overriding active in listchannels with listpeerchannels info.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: `listchannels` no longer uses local knowledge to set `active` to false if disconnected.
2023-12-14 09:16:56 +10:30
Rusty Russell
4b92c773df common: gossmap now always ignores private gossip_store messages.
In the next PR, they'll be removed, but for now all our code doesn't
want them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
58a1c4c012 topology: don't show private nodes in listchannels.
This alters a few remaining tests, as well.

[ Inclused even more test fixes from Alex Myers <alex@endothermic.dev>! ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listchannels` listing private channels: use listpeerchannels
2023-12-14 09:16:56 +10:30
Alex Myers
3a6b9f2ec6 pytest: skip test_hook_crash on liquid CI 2023-12-14 09:16:56 +10:30
Rusty Russell
b318811a2d pytest: fix tests in assumption that listchannels will no longer show private gossip.
Some can only be changed once that is true, but some can be removed/amended already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
ea0d00e677 topology: don't show private nodes in listnodes.
This breaks our tests a bit, which assumed we can always see ourselves
even if we don't have a proper channel.

Usually we would deprecate this first, but it's unlikely to break
anyone since it's a bit obscure that this worked at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listnodes` no longer shows private (peer) nodes: use listpeers
2023-12-14 09:16:56 +10:30
Rusty Russell
f61ba7b332 pytest: remove test waits for routehints.
Now listincoming sees local channels in listpeerchannels, we don't need these waits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
6ee59629f9 lightningd: make listincoming use gossmods_from_listpeerchannels().
We temporarily use a second gossmap so we can just switch private info off
for listincoming and not listchannels.

Note that listchannels now uses the local alias (if no scid), so we have
to change that in the routehint caller.

Since we now *always* use a channel alias in hints if one exists, a
test broke, so fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
0c949dcd04 lightningd: permit wrong-alias lookups for private channel_update.
This is wrong, but we send them for now in zeroconf!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
f3b4821d36 pytest: fix test_zeroconf_open()
l2 gave us a routehint, but it should have seen l1 as a dead-end.  It
didn't due to the presence of a channel alias, which was a bug.  We're
about to fix this, which breaks the test.

Add a dummy node off l1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
bac02b462b plugins/renepay: use gossmods_from_listpeerchannels instead of private gossip_store records.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
c8eb46fe30 plugins/renepay: add localmods later.
We will get localmods from gossmods_from_listpeerchannels in the next commit,
so we need to save routehints to add to that later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
7c26f140d8 common: add tests that json_scan can omit an entire object member.
We can, but I had a typo and thought we couldn't!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
8738d78c86 plugins/pay: use gossmods_from_listpeerchannels instead of private gossip_store records.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
3fdae99524 libplugin-pay: make sure get_gossmap calls are balanced by put_gossmap.
This will become important when we apply localmods to the gossmap at these points.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
a5c0563612 plugins/fetchinvoice: use gossmods_from_listpeerchannels instead of private gossip_store records.
[ Includes use-after-free fix from Alex Myers <alex@endothermic.dev> ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
5256690c6f plugins/topology: temporary hack to allow per-access private gossmap suppression.
We have multiple different routines here, and we want to wean them off private
gossip one at a time.  This converts `getroute`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
cdaad1bf2a common: generic routine to turn listpeerchannels into gossmap local updates.
This is more thorough than the minimal one required for getroute(), including the feerates
and cltv deltas.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
f2fff4de55 gossmap: insert temporary per-caller flag to turn off private gossip.
This lets us convert one user at a time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
1d9a6d1efa pyln-testing: when waiting for active channel, make sure remote has sent fee info.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
c3f33eb6dd listpeerchannels: show gossip updates.
This is redundant if it's a public channel, but vital if it's not.  Publishing unconditionally makes
it easier for gossmap: we create a local modification all the time, even if redundant (and we can
have the actual capacity ceiling accurate in this case, since we know it for local channels).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-Added: JSON-RPC: `listpeerchannels` now shows gossip update contents (even if channel unannounced).
2023-12-14 09:16:56 +10:30
Rusty Russell
5583f730de gossipd: always send information about our channels' gossip updates.
Not just when it's a private channel.  This is useful for listpeerchannels in the next patch.
Most of this is renaming.

It also means that source can be NULL, so move it out of the struct and put it in the message,
where it logically belongs, and make it an optional field.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Alex Myers
8e897746e2 gossipd: pass remote private channel update to ld
and stash in the database.

Rusty: I added the bad gossip message so we would see unknown updates in CI, and made sure we don't send our own generated updates to lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Alex Myers
c1beb4b745 db: add private channel inbound fields
This will be used for generating route hints rather than private gossip
2023-12-14 09:16:56 +10:30
Alex Myers
014cc46328 gossipd: correctly attribute our own channel update to ourselves 2023-12-14 09:16:56 +10:30
Alex Myers
a8995caa8b pytest: use wait_local_channel_active 2023-12-14 09:16:56 +10:30
Rusty Russell
674684369f common: prepare unit tests for lack of private channels.
Created a new canned gossmap without private channels, updated tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
a12e41a930 compiler: fix for -O3 errors.
CI revealed one:

```
cc plugins/libplugin-pay.c
plugins/libplugin-pay.c: In function ‘payment_getroute’:
plugins/libplugin-pay.c:888:17: error: ‘errstr’ may be used uninitialized [-Werror=maybe-uninitialized]
  888 |                 payment_fail(p, "%s", errstr);
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
plugins/libplugin-pay.c:851:21: note: ‘errstr’ was declared here
  851 |         const char *errstr;
      |                     ^~~~~~
cc1: all warnings being treated as errors
```

My local compiler gave another:

```
channeld/channeld.c: In function ‘resume_splice_negotiation’:
channeld/channeld.c:3734:23: error: ‘final_tx’ may be used uninitialized [-Werror=maybe-uninitialized]
 3734 |                 msg = towire_channeld_splice_confirmed_signed(tmpctx, final_tx,
      |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 3735 |                                                               chan_output_index);
      |                                                               ~~~~~~~~~~~~~~~~~~
channeld/channeld.c:3461:28: note: ‘final_tx’ was declared here
 3461 |         struct bitcoin_tx *final_tx;
      |                            ^~~~~~~~
cc1: all warnings being treated as errors
make: *** [Makefile:298: channeld/channeld.o] Error 1
```

So fix both.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Erik De Smedt
a48e2258c7 Implement is_some for cln_rpc::options::Value
Can be used to check if a configuration is set
2023-12-12 17:17:10 +01:00
Erik De Smedt
c404ebedcc Doc: Formatting
Fixed formatting of json.

Combination of tabs and spaces made the json looks bad on
my device
2023-12-12 16:53:59 +01:00
Erik De Smedt
acce3b1b32 Fix docs for invoice_payment hook and notification
The data-format changed in 2019.
The field `amount_msat` was renamed to `msat`
(See 2907e430d5)

The data-format changed again in november 2023.
The content of the field `amount_msat` was changed.
It was a string `"1000msat"` and became a number `1000`.
(See 0b23133ab2)

The current documentation still mentions the data-format before
2907e430d5.

I've updated the docs to reflect the latest changes.
I've also incluced a short mention of 0b23133ab2 to help help
developers create plugins that work both pre 23.11 and post 23.11.
2023-12-12 16:53:59 +01:00
ShahanaFarooqui
4d22f5af2d doc: Documentation update
- Separated Docker documentation from repro builds
- Separated release checklist documentation from contributor workflow
2023-12-12 16:31:44 +01:00
ShahanaFarooqui
78518344f4 docker: Creating docker latest image
- Create docker multiarch image with/without push
- Create docker latest image and publish if asked

Changelog-None
2023-12-12 16:31:44 +01:00
Christian Decker
a9aa93d822 subd: Do not send feerate updates to non-channeld subds
Turns out we were sending feerate updates to daemons that do not
understand it. Don't do that!

Closes #6932

Changelog-Fixed: channeld: We could crash `closingd` by sending it a `channeld` message
2023-12-12 16:27:56 +01:00
Alex Myers
b0377d2323 misc: Add Alex's gpg key
Changelog-None
2023-12-11 18:46:43 +01:00
ShahanaFarooqui
f89e91332f misc: Add Shahana's gpg key
Changelog-None
2023-12-11 18:46:31 +01:00
Alex Myers
9585b7c238 startup_regtest: test for clnrest dependencies before writing config 2023-12-05 20:31:42 -06:00
Alex Myers
66e2bf2947 startup_regtest: configure clnrest and generate rune
This allows use of clnrest + websocket in the regtest environment.
If clnrest starts up, a rune is generated and provided along with
the connection info.

Also uses the snap installed bitcoind path if appropriate.

Format of the start_ln output is:
Commands:
	l1-cli, l1-log,
	l2-cli, l2-log,
	bt-cli, stop_ln, fund_nodes
Node Info:
	l1 rest: https://127.0.0.1:3110  rune: "lgkWSp0PQK-pkbElLpjcNVQnX7yfEjouJaJHjLuW8w89OA=="
	l2 rest: https://127.0.0.1:3111  rune: "sUqGqkevGYG2r1e_JUiz8Me00GhtTv5-IuGk4o9Beyc9OQ=="

Changelog-None
2023-12-05 20:31:42 -06:00
niftynei
df71f2ffa0 coin_mvt: use the lightning_hrp for *all* coin movement currency
We were (dumbly?) using the `onchain_hrp` for 'chain_mvts' and the
`lightning_hrp` for 'channel_mvts'.

This works fine everywhere *except* for on a signet, where we use
different prefixes.

Since the lightning-hrp set is more diversified (testnet btc
+ signet btc use the same HRP 'onchain'), let's use that.

Should have zero impact on anything other than nodes running on signet.

To preserve your current accounts database without needing to delete,
restart, execute the following: (note preferrably when your node isn't
running).

```
UPDATE chain_events SET currency = 'tbs' WHERE currency = 'tb';
```

Fixes #6534

Changelog-Fixed: `bkpr-listbalances` would crash for nodes on signet with payments in channels, because onchain events were using a different currency than inchannel events.
2023-12-05 20:30:09 -06:00
niftynei
5a29baafdd startup-regtest: fund_nodes make balanced, funded channels
Changelog-Changed: startup_regtest.sh: `fund_nodes` will now make balanced channels
2023-12-05 01:02:43 -06:00
niftynei
a1cecf044c startup-regtest: flag every use as developer
Now that 'developer' isn't a compile time flag, let's always use it
when using the startup regtest nodes.

Should always make gossip + polling bitcoind fast
2023-12-05 01:02:43 -06:00
niftynei
8476a15a44 startup-regtest: add ability to set bitcoin-bin + lightning-dir
We were using similar variables for different things (directories vs binary).

Update this to separate them out, while also adding ability to use a different directory
for lightning nodes

Changelog-Changed: startup_regtest.sh PATH_TO_LIGHTNING + PATH_TO_BITCOIN are no more. Use LIGHTNING_BIN and BITCOIN_DIR
2023-12-05 01:02:43 -06:00
Christian Decker
be52f2f21e ci: Force flaky of tests
We are tracking the flakiness of tests [here][1], so no need to suffer
through fail-looping CI while trying to fix a PR or get it merged.

Changelog-None

[1]: http://35.239.136.52:3170/run
2023-12-04 20:22:04 +01:00
Christian Decker
a96e64ed56 chore(CI): Add manually triggerable PyPI publication
We used to run the publication on each PR, which could cause conflicts
due to the naming convention being based on the commit history length,
and multiple PRs potentially hitting the same name.

Adding a `workflow_dispatch` trigger allows us to still test the
workflow in PRs but only do so manually (when we're watching anyway).

Changelog-None
2023-12-04 20:22:04 +01:00
Christian Decker
66bf0a7bbb chore(CI): Increase the timeout to publish to test.pypi.org 2023-12-04 20:22:04 +01:00
Peter Neuroth
84f22066f2 misc: Add peter's gpg key
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>

Changelog-None
2023-11-29 12:56:45 +01:00
Peter Neuroth
5887be000c CHANGELOG.md: Update for 23.11 final
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2023-11-28 19:01:39 +01:00