Commit Graph

10401 Commits

Author SHA1 Message Date
Rusty Russell
36baef22e2 doc: document shutdown plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Rusty Russell
516c38a750 lightningd: call shutdown plugin when we dynamic shutdown a single one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Rusty Russell
2063049559 libplugin: plugin_exit helper which flushes stdout.
We weren't actually getting the last log out; this does that.

We have to fix test_bitcoin_failure which now notices the BROKEN
log message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: libplugin: Fatal error messages from plugin_exit() now logged in lightningd.
2021-09-05 15:16:56 +02:00
Rusty Russell
1d8aecb44f lightningd: call "shutdown" notification on plugins at shutdown.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `shutdown` notification for clean exits.
2021-09-05 15:16:56 +02:00
Rusty Russell
89c3541c83 lightningd: disable topology timers on shutdown.
Not necessary yet, but it will be once shutdown starts waiting for
plugins to respond: we don't want these to try to access the bcli
plugin once it's freed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Rusty Russell
395bf96d15 lightningd: don't keep timer ptr inside struct chain_topology.
It's a legacy from when it didn't have an ld pointer to access ld's
timer structure.  Now it's just confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:16:56 +02:00
Rusty Russell
6d1897b796 doc: fix GH/readthedocs rendering of manual pages.
They render the comment as if it's in the list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-05 15:02:17 +02:00
niftynei
0f05238e97 psbt: set the witness script to null, also
Otherwise libwally pushes the psbt-key for 'witness script' onto the
serialized version and we fail the 'is this identical' check.

Relevant line from libwally, where if bytes, we push a psbt_key.

```
static void push_typed_varbuff(unsigned char **cursor, size_t *max,
                               uint64_t type,
                               const unsigned char *bytes, size_t bytes_len)
{
    if (bytes) {
        push_psbt_key(cursor, max, type, NULL, 0);
        push_varbuff(cursor, max, bytes, bytes_len);
    }
}
```

Reported-By: @grubles
Changelog-Fixed: openchannel_signed would fail on PSBT comparison of materially identical PSBTs
2021-09-05 14:24:14 +02:00
niftynei
1782ae8590 dual-fund psbt: hoist up single use method so we can test it
Also remove all the unused code (checking for differences for a side
etc)
2021-09-05 14:24:14 +02:00
Rusty Russell
d6192c2cb0 db: simple index addition to speed listpeers.
Fixes: #4718
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-04 13:41:35 +02:00
William Casarin
105a88dde4 docs/fetchinvoice: document payer_note
Signed-off-by: William Casarin <jb55@jb55.com>
2021-09-04 13:18:29 +02:00
Michael Schmoock
f6709a7a90 pytest: test new commitment_revocation hook values 2021-09-04 11:54:33 +09:30
Michael Schmoock
37a7dab549 doc: new values for commitment_revocation hook 2021-09-04 11:54:33 +09:30
Michael Schmoock
1db0ec8b75 hook: adds channel_id and commitnum to commitment_revocation
If commitment_revocation hook is not being used for a whatchtower but
something else, the channel_id is missing.

Changelog-Added: addes channel_id and commitnum to commitment_revocation hook
2021-09-04 11:54:33 +09:30
Christian Decker
7e43109e33 doc: Regenerated man-pages to fix hash mismatch. 2021-09-03 20:03:02 +09:30
Rusty Russell
1c8294ef1c doc: fix listnodes schema, doc.
Shows the dangers of "additionalProperties": true.  We didn't have an else
clause, so our incorrect (and, with DF, incomplete!) schema was accepted.

I despair of getting anyone else to write a decent schema with these
semantics :(

Changelog-Fixed: doc: listnodes fields now correctly documented.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-03 20:03:02 +09:30
Rusty Russell
372fffac69 Anchor outputs are enabled by dual-funding, so this was reported by
users there (and reliably detected now).

Reported-by: @larsschenk
Reported-by: @nickfarrow
Fixes: #4728
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: EXPERIMENTAL: crash for some users while requesting dual funding leases.
2021-09-02 15:06:18 +02:00
Rusty Russell
2e01d26588 configure: update from -fstack-protector to -fstack-protector-strong.
This would have caught the bug found in the next commit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-02 15:06:18 +02:00
Vincenzo Palazzo
218875a38d adding check for the deprecated api.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-02 14:19:42 +09:30
Vincenzo Palazzo
3b65a4caca Fixed order parameter in the listforwards command
Changelog-Changed: Change order parameters in the listforwards command

Changelog-Deprecated: Change order of the status parameter in the listforwards rpc command.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-02 14:19:42 +09:30
Rusty Russell
0a05bcd05e wire: fix patch file to match actual peer_wire.csv.
Running `make extract-peer-csv` changes peer_wire.csv, @niftynei
points out that the patch file is wrong (peer_wire.csv is correct).

(Any attempt to regenerate the csv files from the specs hits this issue)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-02 14:04:32 +09:30
Christian Decker
df70bbcc6a docs: Remove explicit markdown configuration for rtd 2021-08-31 11:46:17 -05:00
Rusty Russell
2770bcb158 lightningd: don't reject attempts to forward 2^32 msat out of hand.
The outgoing channeld still does that, for now, but that will change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-30 15:56:20 +09:30
Rusty Russell
ebb0b5791e Don't limit channel_updates to 2^32 msat.
We're in a post-Wumbo world now!  See also https://github.com/lightningnetwork/lightning-rfc/pull/902

Fixes: #4746
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-30 15:56:20 +09:30
Vincenzo Palazzo
bc1eab8646 Skip following import and skip flask type to mypy code analysis.
As suggested in this issue https://github.com/python/mypy/issues/7484#issuecomment-529363083 we skip following import becuase with the recent version of mypy the __init__.py file make confusione inside the analysis (in the python issue it is unclear the main motivation of this issue. At list unclear to me).

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-08-30 12:12:32 +09:30
Vincenzo Palazzo
b40812494d Move to minimum required mypy dependeces
This fix some dependencies error with lnprototest

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-08-30 12:12:32 +09:30
Vincenzo Palazzo
f2d0e93ce0 Added deprecated phase to rpc framework rules changes.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

Changelog-Deprecated: RPC framwork now require the "jsonrpc" propriety inside the request.
Changelog-Fixed: RPC framwork now required the "jsonrpc" propriety to be specified inside each request.
2021-08-30 12:10:37 +09:30
Vincenzo Palazzo
dca9bb8638 Required jsonrpc inside the method request.
As the json rpc specification tell, the "jsonrpc": "2.0" MUST be required.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-08-30 12:10:37 +09:30
Rusty Russell
8f782b06f7 pytest: don't copy db to query it.
Should be safe to do in-place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-26 12:44:02 +09:30
Rusty Russell
5bf2b5c0ba wallet: set a timeout if the db is busy.
This is recommended for litestream, which allows for easy async backup,
and harmless otherwise.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: db: we now set a busy timeout to safely allow others to access sqlite3 db (e.g. litestream)
2021-08-26 12:44:02 +09:30
Alex
c066378576 added Litestream Replication
Litestream enables easy SQLite replication
2021-08-26 12:44:02 +09:30
Michael Schmoock
da0b159323 pylightning: fix requirements to work with pyln-client
Without this, we cannot pip install pylightning and pyln-client in the
same environment anymore, as it tries to pull in an incompatible version
of pyln-client.

Changelog-None
2021-08-26 10:13:01 +09:30
Rusty Russell
fe86c117d9 datastore: turn keys into arrays
After some discussion with @shesek, and my own usage, we agreed that
a more comprehensive interface, which explicitly supports grouping,
is desirable.

Thus keys are now arrays, with the semantic that a key is either a
parent or has a value, never both.

For convenience in the JSON schema, we always return them as arrays,
though we accept simple strings as arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
533571a655 datastore: add generation, simple atomicity.
We add a generation counter, and allow update or del conditional
on a given generation.

Formalizes error codes, too, since we have more now.

Suggested-by: @shesek
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
432508e65e datastore: allow replace/append.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
e711f6c589 datastore: allow strings.
It's common, and the simplest case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
dfe2693bbd datastore: docs, schemas, and a test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
1b48545e5b JSON-RPC: add interfaces to access datastore.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `datastore`, `deldatastore` and `listdatastore` for plugins to store simple persistent key/value data.
2021-08-25 10:06:31 +02:00
Rusty Russell
2fb8735f04 wallet: add datastore access routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
df17387e05 wallet: add datastore for plugins to use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
LightningHelper
720a4d7ae7 exact bk file has to be in path
The exact file that will be the backup has to be in the backup output path.
2021-08-25 11:58:56 +09:30
LightningHelper
1b705f0cc9 missing --lightning-dir
It seems that the plugin require `--lightning-dir` , using without `--lightning-dir` result with an error.
2021-08-25 11:58:56 +09:30
Rusty Russell
cf1dd779d1 gossmap: don't spew to stderr, include counter for callers.
Fixes: #4722
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: don't drop complaints about silly channels to stderr.
2021-08-24 06:42:32 +09:30
Sebastian Falbesoner
c970195b93 pytest: remove only_one() duplicate
The function is already provided in
contrib/pyln-testing/pyln/testing/utils.py (which is also
imported in this module), so there is no need to define it
a second time.
2021-08-24 06:42:16 +09:30
Sebastian Falbesoner
48473eb3e9 wallet-test: fix segfault due to uninitialized block
The variable `block` (instace of `struct block`) is
allocated on the stack without being initialized, i.e. its
member `prev` points to nowhere. This causes a segmentation
fault on my machine on the binding of "prev_hash" on running
`wallet_block_add`, as the following core-dump analysis
shows:

    $ egdb ./wallet/test/run-wallet ./run-wallet.core
    [...]
    Core was generated by `run-wallet'.
    Program terminated with signal SIGSEGV, Segmentation fault.
    ---Type <return> to continue, or q <return> to quit---
    #0  0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97
    97                      TLOOP1(*dst++ = *src++);
    (gdb) bt
    #0  0x000008f67a04b660 in memcpy (dst0=<optimized out>, src0=0x100007f8c, length=32) at /usr/src/lib/libc/string/memcpy.c:97
    #1  0x000008f73e838f60 in sqlite3VdbeMemSetStr () from /usr/local/lib/libsqlite3.so.37.12
    #2  0x000008f73e83cb11 in bindText () from /usr/local/lib/libsqlite3.so.37.12
    #3  0x000008f44bc91345 in db_sqlite3_query (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:77
    #4  0x000008f44bc91122 in db_sqlite3_exec (stmt=0x8f6845bf028) at wallet/db_sqlite3.c:110
    #5  0x000008f44bcbb3b2 in db_exec_prepared_v2 (stmt=0x8f6845bf028) at ./wallet/db.c:2055
    #6  0x000008f44bcc6890 in wallet_block_add (w=0x8f688b5bba8, b=0x7f7ffffca788) at ./wallet/wallet.c:3556
    #7  0x000008f44bce2607 in test_wallet_outputs (ld=0x8f6a35a7828, ctx=0x8f6a35c0268) at wallet/test/run-wallet.c:1104
    #8  0x000008f44bcddec0 in main (argc=1, argv=0x7f7ffffcaaf8) at wallet/test/run-wallet.c:1930

Fix by explicitely setting the whole structure to zero.

[ Rebuilt generated files, too --RR ]
2021-08-23 19:46:04 +09:30
Michael Schmoock
24ea498350 cleanup: rename use_proxy_always to always_use_proxy to match cfg
This renames all occurences of use_proxy_always to always_use_proxy
to keep it inline with config values. This was a bit confusing.

Only significant change is that the payload in the plugins init
requests also contained the old name. No plugin currently seems to make
use of this variable yet. The old name 'use_proxy_always' is added when
deprecated APIs is enabled.

Changelog-Deprecated: Plugins: Renames plugin init 'use_proxy_always' to 'always_use_proxy'
2021-08-23 14:43:40 +09:30
Michael Schmoock
0550ffa9db cleanup: nits 2021-08-23 14:43:40 +09:30
Michael Schmoock
f31c5b7662 connectd: prefer non-TOR connections and filter wireaddr hint duplicate
This does two things:
 - It moves non-tor addresses upfront so it prefers normal connection
   which are less laggy and more reliable.
 - It prevents connectd from trying the same wire_addr twice when the
   addr_hint was given and gossip also added the same address.

Changelog-Changed: connectd: try non-TOR connections first
Changelog-Fixed: connectd: do not try address hint twice
2021-08-23 14:43:40 +09:30
Rusty Russell
f1e0fe506d fetchinvoice: add developer option to send raw invoice_request.
This will be used for bootstrap.bolt12.org to provide a raw request API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-20 13:45:00 +09:30
Rusty Russell
12fbb23322 pytest: fix flake in test_channel_lease_unilat_closes
We actually were waiting for l3 to disconnect, not l2.

But in general we should be looking for status rather than grovelling
in the logs where possible, so I changed all those.

```
2021-08-17T04:40:34.9015538Z         # l2 leases a channel from l3
2021-08-17T04:40:34.9016520Z         l2.rpc.connect(l3.info['id'], 'localhost', l3.port)
2021-08-17T04:40:34.9017570Z         rates = l2.rpc.dev_queryrates(l3.info['id'], amount, amount)
2021-08-17T04:40:34.9018724Z         l3.daemon.wait_for_log('disconnect')
2021-08-17T04:40:34.9019851Z         l2.rpc.connect(l3.info['id'], 'localhost', l3.port)
2021-08-17T04:40:34.9021467Z         l2.rpc.fundchannel(l3.info['id'], amount, request_amt=amount,
2021-08-17T04:40:34.9022865Z                            feerate='{}perkw'.format(feerate), minconf=0,
2021-08-17T04:40:34.9024000Z >                          compact_lease=rates['compact_lease'])
...
2021-08-17T04:40:34.9103422Z >           raise RpcError(method, payload, resp['error'])
2021-08-17T04:40:34.9106829Z E           pyln.client.lightning.RpcError: RPC call failed: method: fundchannel, payload: {'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d', 'amount': 500000, 'feerate': '2000perkw', 'announce': True, 'minconf': 0, 'request_amt': 500000, 'compact_lease': '029a00640064000000644c4b40'}, error: {'code': 400, 'message': 'Unable to connect, no address known for peer', 'data': {'id': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d', 'method': 'connect'}}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-18 22:11:47 -05:00