Commit Graph

8285 Commits

Author SHA1 Message Date
ZmnSCPxj jxPCSnmZ
6fadd5aea2 Makefile: Remove gen_version.h from ALL_GEN_HEADERS.
This prevents recompiling everything when you are changing just a doc, or
touching only one file among hundreds of sources, just because the
`gen_version.h` is changed, especially since only one source actually
depends on that header.
2020-08-07 10:41:37 +09:30
ZmnSCPxj jxPCSnmZ
a5fc66c382 plugins/Makefile: Only link in libplugin-pay for plugins that need it.
Changelog-None: cleanup only.

Before this change:

```
$ ls -l plugins/bcli
-rwxrwxr-x 1 zmnscpxj zmnscpxj 1914976 Aug  5 21:54 plugins/bcli
```

After this change:

```
$ ls -l plugins/bcli
-rwxrwxr-x 1 zmnscpxj zmnscpxj 1830552 Aug  5 22:00 plugins/bcli
```

We already duplicate a lot of code between `lightningd` and eeach
builtin plugin because we do not use .so for `common/`, but including
an object file with code that is not referenced in the executable is
fairly low-hanging size optimization fruit.
2020-08-07 10:40:10 +09:30
Jon Griffiths
908a8399e2 psbt: Allocate correctly sized buffer for psbt_to_bytes
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths
fef155a9e2 psbt: Use wallys function to check PST finalization status
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths
1fe53880a7 psbt: use new wally functions to add PSBT inputs/outputs
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths
68ffecba63 psbt: Use the newly exposed wally function to clone PSBTs
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths
a51c6550ec psbt: avoid allocations when adding psbt outputs
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths
499395636c psbt: remove now-unneeded const casts
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths
21ee2c3a9c psbt: Remove workaround for now-fixed wally tx flag behaviour
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Jon Griffiths
95d3d65c62 wally: update to the latest wally version
Includes:
psbt: Use renamed functions for new wally version
psbt: Set the transaction directly to avoid script workarounds
psbt: Use low-S grinding when computing signatures
tx: Use wally_tx_clone from libwally now that its exported

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2020-08-07 10:28:01 +09:30
Rusty Russell
06372e13d8 pyln.proto.message: don't let Message() init set implicit lengths.
We'll override them from field length anyway!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 17:53:25 +02:00
ZmnSCPxj jxPCSnmZ
4fde45669c plugins/pay.c: Fix the routehints/presplit conflict.
Changelog-Fixed: pay: Fixed a bug where routehints would be ignored if the payment exceeded 10,000 satoshi. This is particularly bad if the payee is only reachable via routehints in an invoice.
2020-08-06 17:36:58 +02:00
ZmnSCPxj jxPCSnmZ
a9b992a94a tests/test_pay.py: Add test for conflict between presplit and routehints paymods. 2020-08-06 17:36:58 +02:00
Sjors Provoost
2e10729690 Add missing test dependencies to requirements.txt
Supplements the consolidation in b68066e
2020-08-06 09:46:04 +09:30
Rusty Russell
fa829f23db amount: add amount_msat_scale, amount_msat_ratio, amount_{msat,sat}_div
It's not all that rare to do these operations, and requiring annotations
for it is a little painful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
Rusty Russell
ffbb409b47 amount: use initializers everywhere.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
Rusty Russell
36d43b871f amount: add amount_msat and amount_sat initializers.
Generally, importing amounts needn't be checked, and it cuts down on
the warnings we get.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
niftynei
28b839f306 amount: add helper to convert u64 sats to amount_sat type 2020-08-06 09:36:47 +09:30
ZmnSCPxj jxPCSnmZ
1ded3fc52f lightningd/plugin.c: Add a --dev-builtin-plugins-unimportant for developers who want to mess around with the builtin plugins. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
1aa076845a tests/test_plugin.py: Test builtin plugins are important. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
4ca2e49812 lightningd/plugin.c: Make builtin plugins important.
Changelog-Changed: plugin: Builtin plugins are now marked as important, and if they crash, will cause C-lightning to stop as well.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
48f36904c8 tests/test_plugin.py: Add test for --important-plugin. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
a847487bbe lightningd/plugin.c: Add important plugins, which if they terminate, lightningd also terminates.
Changelog-Added: New option `--important-plugin` loads a plugin is so important that if it dies, `lightningd` will exit rather than continue.  You can still `--disable-plugin` it, however, which trumps `--important-plugin` and it will not be started at all.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
50600dce95 lightningd/lightningd.c: Create API to exit lightningd with an exit code. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
3df2333d5d lightningd/plugin.c: Add specific function to give the directory for built-in plugins. 2020-08-04 13:27:51 -05:00
Christian Decker
3dafddd717 channel: Base the channel forget timeout on the headercount
We were basing the 2016 block timeout on the blockchain height that we had
processed at the time completed the funding, which could lag considerably
behind the network-wide blockheight. For example this could happen if we
started with `--rescan` from a low height, taking some time to catch up.

This means that we could end up forgetting channels even before reaching the
network blockheight. This patch instead uses the headercount reported by the
backend plugin if we aren't caught up yet. While the chances of this happening
are still there, the window it might happen are much reduced, since headers
can be synced in a couple of minutes.

Reported-by: Riccardo Masutti
Changelog-Changed: Funding timeout is now based on the header count reported by the bitcoin backend instead of our current blockheight which might be lower.
2020-08-04 09:14:17 +09:30
ZmnSCPxj jxPCSnmZ
094eac4e95 plugins/libplugin-pay.c: Properly handle exclusions for routehints with two hops or more.
Arguably a low-priority bug since no current node ever generates routehints longer
than one hop.

However, it is possible as an edge case, if the destination is directly accessible
*and* supports multiple channels, that we route through the destination, one of the
*other* channels it has not in the routehint, to the entry point, and then through
the routehint.

This change removes the risk of the above edge case.

Changelog-None: arguably a low-priority bug.
2020-08-03 15:20:29 +09:30
Antoine Poinsot
3aad86ff9d pytest: test the blocksonly sanity checkin bcli
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Antoine Poinsot
5b45334d93 pyln: allow to signal may_fail in get_node()
This or `expect_fail`, as it implies that, well, the node may fail.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Antoine Poinsot
905730341a bcli: explicitly check at startup that bitcoind does relay transactions
We've had problems with blocksonly in the past and bitcoind still allows
to use outdated (thus potentially dangerous estimates) while running
bitcoin with -blocksonly.

ZmnSCPxj mentionned that we still don't document this, but i figured
that in this specific case an explicit check and error seems preferable.

Changelog-Added: We now explicitly check at startup that our default Bitcoin backend (bitcoind) does relay transactions.

Proposed-by: ZmnSCPxj <zmnscpxj@protonmail.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Antoine Poinsot
5be07c5fe3 bcli: poll getnetworkinfo at startup
This allows us to kill two birds with one stone: once connected, we use
the output of the successful call to do some sanity checks (only
checking the version for now, but more are yet to come!).

Changelog-Added: We now explicitly check at startup the version of our default Bitcoin backend (bitcoind).
Co-Authored-by: ZmnSCPxj <zmnscpxj@protonmail.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Christian Decker
a456d08ad0 pay: Be less aggressive when estimating channel capacity
We'd previously take the failed attempt and estimate the failing channel's
capacity at 3/4 of the attempted amount, which is rather aggressive. This
reduces this aggressiveness to use the exact amount tried, but excluding on
equality. This still skips attempting the same route with the same amount, but
also permits attempts that are in the range [3/4, 1] of the failed attempt
amount to still be attempted.
2020-08-03 12:15:08 +09:30
ZmnSCPxj jxPCSnmZ
e7d89cd7de lightningd/invoice.c: Improve programmatic error reporting for delinvoice.
Changelog-Changed: JSON-RPC: `delinvoice` will now report specific error codes: 905 for failing to find the invoice, 906 for the invoice status not matching the parameter.
2020-07-31 23:57:32 +00:00
ZmnSCPxj jxPCSnmZ
b646b96756 lightningd/chaintopology.h: Remove unused txnums field from struct block.
Changelog-None internal cleanup
2020-07-31 23:55:23 +00:00
Christian Decker
af4eec7b66 release: Update changelog for release v0.9.0 2020-07-31 18:41:04 +02:00
Christian Decker
d9680414cb release: Add changes since v0.9.0rc3 to changelog 2020-07-29 14:27:36 +02:00
niftynei
d87f31f9a8 utxo: clean up NULL handling of scriptpubkey
Now that we're *guaranteed* to have a scriptpubkey entry in the
database, we remove the NULL handling for it.
2020-07-29 13:13:46 +02:00
niftynei
90b393ca1a hsmd/db: backfill pubkey information so that psbts signing works
the way we use PSBTs to sign things requires that we have the
scriptpubkey available on the utxo so we can populate the witness-utxo
field with it.

this causes problems if we don't already have the scriptpubkey cached in
the database, as in *some* cases we require a round trip to the HSM to
populate them

to get over this hump, we backfill any and all missing scriptpubkey
information for the utxo's that we hold in our wallet.

this will allow us to clean up the NULL handling of missing
scriptpubkeys.
2020-07-29 13:13:46 +02:00
niftynei
65c2bac2f3 hsmd/wallet: pass the bip32_key down into migrations
we're about to add a migration that requires access to the bip32_key
in order to calculate missing scriptpubkeys.

prior to this patch, we don't have access to the bip32 key in the db
migration, as it's set on the wallet but after the db migrations are
run.

here we patch it through so that every migration can access it
2020-07-29 13:13:46 +02:00
Rusty Russell
4517435810 pytest: fix flake in tests/test_pay.py::test_pay_exclude_node
```
        # Excludes channel, then ignores routehint which includes that, then
        # it excludes other channel.
>       assert len(status) == 2
E       assert 1 == 2
E         -1
E         +2
```

The invoice we use at the end has a routehint: 50% of the time it's
to l2 (which fails), 50% to l5 (which succeeds).

Change it to create invoice before channel with l5 so it does the
retry like we expect here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-29 11:42:09 +02:00
Christian Decker
14d08b75cd doc: Document the msatoshi parameter for sendonion 2020-07-28 16:17:39 +09:30
Christian Decker
311594b2cd jsonrpc: Add amount received by recipient to listpays result
We sum up the amounts just like we do with `amount_sent`, however we may have
parts that don't have that annotation, so make it optional.

Suggested-by: Rusty Russell <@rustyrussell>
2020-07-28 16:17:39 +09:30
Christian Decker
d7cca0781d jsonrpc: Add msatoshi argument to sendonion to annotate
While not directly necessary, it still feeds the `listpays` result, and so we
should pass it along if we can, so we don't have to rely solely on the
`amount_sent` field, which includes the fees.

Reported-by: Rusty Russell <@rustyrussell>
2020-07-28 16:17:39 +09:30
Christian Decker
1da977a04c db: Guard against accessing NULL partid and total_msat
These were spamming my logs and could result in misleading results being
returned on `listpays` and `listsendpays`.
2020-07-28 16:17:39 +09:30
Christian Decker
d12ea51e63 pytest: Replicate the missing amount_msat in listpays
Since we started using `sendonion` in the `pay` plugin we no longer
automatically have the `amount` annotation on (partial) payments. This
replicates the issue so we can fix it.

Reported-by: Rusty Russell <@rustyrussell>
2020-07-28 16:17:39 +09:30
ZmnSCPxj jxPCSnmZ
8e02998161 doc/PLUGINS.md: Correct misspelling of getchaininfo.
Changelog-None
2020-07-27 22:35:30 +02:00
Rusty Russell
b4aff493f1 pay: always send onion error message to gossipd.
There were no channel updates in my log; because sendonion doesn't know the
actual node_ids or channel_ids, we can't tell gossipd what node/channel it was
so it can no longer remove them on PERM errors.

However, we can tell it the error message so it can apply the update.

Fixes: #3877
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-27 15:22:38 +02:00
Vincent
81fd552e84 plugins/pay: hand bolt11 arg to sendonion if we have one (i.e. for pay)
[ Extracted into standalone patch and comment added by RR ]
2020-07-27 13:11:14 +02:00
Rusty Russell
71a2aefafd sendonion: add bolt11 arg.
And document the partid arg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `sendonion` has a new optional `bolt11` argument for when it's used to pay an invoice.
2020-07-27 13:11:14 +02:00
Vincent
f4f2853ded pytest: test to show that listpays doesn't remember bolt11 string.
[Separated into separate commit by RR]
2020-07-27 13:11:14 +02:00