Commit Graph

8100 Commits

Author SHA1 Message Date
Christian Decker
cb9debc229 tlv: Add generic getters and setters for tlvstream
Trying to rework the TLV streams to have a more homogenous interface to work
with. This is by no means a complete implementation, just the groundwork that
is going to be used by the wire code generator to generate the specific
accessors, but it's enough so we can manipulate TLV streams in the onion and
later just switch to the generated ones.
2020-07-01 12:19:02 +02:00
Christian Decker
958eb41cbb paymod: Do not pass cmd to sub-payments, plugin suffices
We were passing the `cmd` instance around where `plugin` suffices (used to
start RPC calls and logs).
2020-07-01 12:19:02 +02:00
Christian Decker
df94ec35ba paymod: Do not reply to the same JSON-RPC command multiple times 2020-07-01 12:19:02 +02:00
Christian Decker
c35df400b2 paymod: Maintain a list of current and past payments
We need to keep them around so we can inspect them later. We'll also need a
background cleanup every once in a while to free some memory. More on that in
a future commit.
2020-07-01 12:19:02 +02:00
Christian Decker
8207b4eac8 paymod: Use reasonable partids
We were just handwaving the partid generation, which broke some tests that
expected the first payment attempt to always have partid=0, so here we just
track the partids assigned in the payment tree, starting at 0.
2020-07-01 12:19:02 +02:00
Christian Decker
04c9c5e0d1 paymod: Collect and return results of a tree of partial payments
The status of what started as a simple JSON-RPC call is now spread across an
entire tree of partial payments and payment attempts. So we collect the status
in a single struct in order to report back success of failure.
2020-07-01 12:19:02 +02:00
Christian Decker
4fec969062 paymod: Parse and store the waitsendpay result
This is necessary so we can later aggregate across an entire tree of attempts
and report success or failure to the RPC caller.
2020-07-01 12:19:02 +02:00
Christian Decker
6fb81bf203 paymod: Call waitsendpay on the sendonion we just queued 2020-07-01 12:19:02 +02:00
Christian Decker
d8ebe6fbcd paymod: Parse createonion result and call sendonion 2020-07-01 12:19:02 +02:00
Christian Decker
d37b8548fb paymod: Call createonion to generate the onion and associated data
After we gave each modifier a chance to have its say, we can now proceed to
generate the onion that's going to be sent in the next step.
2020-07-01 12:19:02 +02:00
Christian Decker
5f32cfe9cd paymod: Implement legacy onion payload computation
This is just for testing for now, TLV payload computation will come next. We
stage all the payloads in deserialized form so modifiers can modify them more
easily and serialize them only before actually calling `createonion`.
2020-07-01 12:19:02 +02:00
Christian Decker
9d36e49770 doc: Fix a documentation error in sendonion
Te `sendonion` docs where claiming that the `first_hop` needs to specify a
`channel_id` whereas it should really be the `node_id` of the peer we are
trying to contact.
2020-07-01 12:19:02 +02:00
Christian Decker
5ef7297e51 paymod: Add a getinfo call on payment_start to get the blockheight
This is necessary so we can build the absolute locktimes in the next step. For
now just fetch the blockheight on each (sub-)payment, later we can reuse the
root blockheight if it ends up using too much traffic.
2020-07-01 12:19:02 +02:00
Christian Decker
3c3e2c9c8f paymod: Implement getroute call 2020-07-01 12:19:02 +02:00
Christian Decker
ce0025c95c paymod: Implement bubbling results up the hierarchy of payments
A payment is considered finished if it is in a final state (success or
failure) or all its sub-payments are finished. If that's the case we notify
`payment_finished` and bubble up through `payment_child_finished`, eventually
bubbling up to the root, which can then report success of failure back to the
RPC command that initiated the whole process.
2020-07-01 12:19:02 +02:00
Christian Decker
5c32e33aab paymod: Add a retry modifier that retries payments on failure
This is likely a bit of overkill for this type of functionality, but it is a
nice first use-case of how functionality can be compartmentalized into
modifiers. If makes swapping retry mechanisms in and out really simple.
2020-07-01 12:19:02 +02:00
Christian Decker
aeb5cc0b7c paymod: Generate type-safe accessor functions for modifier data
This should make it easy for JSON-RPC functions and modifiers to get the
associated data for a given modifier name. Useful if a modifier needs to
access its parent's modifier data, or in other functions that need to access
modifier data, e.g., when passing destination pointers into the `param()`
call.
2020-07-01 12:19:02 +02:00
Christian Decker
23b4dca3c7 paymod: Add a simple test-command to test the paymod state-machine
This commit can be reverted/skipped once we have implemented all the logic and
have feature parity with the normal `pay`. It's main purpose is to expose the
unfinished functionality to test it, without completely breaking the existing
`pay` command.
2020-07-01 12:19:02 +02:00
Christian Decker
5d1725209d paymod: Implement base state-machine transitions
The actual steps are mocked out, but we can already run through the various
phases of a payment, and the modifiers should be called for each state.
2020-07-01 12:19:02 +02:00
Christian Decker
e108ebe559 paymod: Add base state-machine for payment flow 2020-07-01 12:19:02 +02:00
Christian Decker
8138ef136f plugin: Add a data backend for the new payment state-machine 2020-07-01 12:19:02 +02:00
Saibato
eea6b33154 connect/tor: Set errno, if connection failed we want to reflect this.
Changelog-None:

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
Saibato
2b0aba13a8 connectd/connectd: Display an error hint for V3 tor onions when connect fail.
@thestick613 noticed that since tor version below 0.3.2.2-alpha
will not support V3 ed25519 address formats, the error handling
is not that helpful in the error message from cli.
So now we add an hint.

Changelog-None:

Signed-off-by: Saibato <saibato.naga@pm.me>

connectd/connectd.h; Add helper function to update conn error list

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
Saibato
e4cd2148ab Reflect in TOR.md that we need a Tor version >= 0.3.2.2-alpha to reach V3 onions
Changelog-None

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
ZmnSCPxj jxPCSnmZ
5d720536e2 Makefile: Install tools/hsmtool as lightning-hsmtool.
Changelog-Added: We now install `lightning-hsmtool` for your `hsm_secret` needs.

See: https://github.com/ElementsProject/lightning/issues/3717#issuecomment-644844594

It seems reasonable to add this to the standard install, and to document it properly as well, hopefully we can fill in the documentation better later on.
2020-07-01 09:24:03 +09:30
Christian Decker
7b899da801 db: Retrieve peer ID if it exists or create the peer if not
We were assuming `wallet_channel_insert` that there cannot be a matching peer
if our in-memory representation isn't bound to it (`dbid == 0`). If we then
attempt to create the peer, and we already had one it'd cause a unique
constraint violation. As far as I can tell this could end up happening if we
have an uncommitted channel, and then exited without cleanup (`tal_destructor`
on the uncommitted channel not running). This could then leave the peer in the
DB. This is because the constraint that every peer has at least one channel is
not enforce at DB level, but rather in destructors that may or may not run.

Changelog-Fixed: Fixed a failing assertion if we reconnect to a peer that we had a channel with before, and then attempt to insert the peer into the DB twice.
2020-07-01 09:22:29 +09:30
Rusty Russell
ae0cccb293 wallet: fix memleak if we get shut down before finishing utxo cleanup.
"backtrace": [
                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
                       "wallet/wallet.c:154 (wallet_stmt2output)",
                       "wallet/wallet.c:275 (wallet_get_utxos)",
                       "wallet/wallet.c:3792 (wallet_clean_utxos)",
                       "lightningd/lightningd.c:914 (main)"
                   ],
                   "label": "wallet/wallet.c:154:struct utxo",
                   "parents": [
                       "wallet/wallet.c:273:struct utxo*[]"
                   ],
                   "value": "0x24c1be8"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-01 09:19:39 +09:30
ZmnSCPxj jxPCSnmZ
d14460ef3d wallet/walletrpc.c: Show input annotations for inputs.
Changelog-None
2020-06-30 17:59:17 +02:00
Rusty Russell
6b3bcd49d2 tests: fix flake in test_penalty_htlc_tx_fulfill
If the daemon already knows about the channel before it was stopped,
it won't get this message from gossipd.  That's OK, since we explicitly
test for the channel being active two lines down.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-30 13:50:30 +00:00
Rusty Russell
dd4a1a3510 tests: fix flake in test_txprepare_restart
Detection of reserved outputs is async, going via bitcoind.  Wait for them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-30 13:50:30 +00:00
Rusty Russell
74abd30da5 pyln-proto, pyln-spec: fix 'make prod-release' target.
rusty$ make prod-release
  make: *** No rule to make target 'test', needed by 'prod-release'.  Stop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-29 18:01:12 +02:00
niftynei
2e9c387f45 coin_moves: update withdrawal logic to account for 'variable owner' txs
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)

Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
2020-06-29 16:10:05 +02:00
niftynei
ee549a2af9 tx: fix case where input amounts are less than total outputs
when attempting to calculate the fees for a tx where we don't own all of
the outputs, we can overshoot the feerate
2020-06-29 16:10:05 +02:00
niftynei
9830c94778 rpc: new signpsbt + sendpsbt rpcs
Changelog-Added: JSON-RPC: new call `signpsbt` which will add the wallet's signatures to a provided psbt
Changelog-Added: JSON-RPC: new call `sendpsbt` which will finalize and send a signed PSBT
2020-06-29 16:10:05 +02:00
niftynei
fd8a716695 wallet: have wallet_extract_outputs take wally_tx, not bitcoin_tx
With the incursion of PSBTs, we're moving away from bitcoin_tx
2020-06-29 16:10:05 +02:00
niftynei
0388fe6db4 tx: add helper for extracting script from a wally_tx
the bitcoin_tx version is basically a wrapper for the wally_tx script
extraction -- here we pull it apart so we can easily get a tal'd script
for a wally_tx_output
2020-06-29 16:10:05 +02:00
niftynei
a68d79e390 add manpages for reserve/unreserve 2020-06-29 16:10:05 +02:00
niftynei
103dce63ef reserve/unreserve input: new RPC commands for reserving inputs/outputs
Reserve and unreserve wallet UTXOs using a PSBT which includes those
inputs.

Note that currently we unreserve inputs everytime the node restarts.
This will be addressed in a future commit.

Changelog-Added: JSON-RPC: Adds two new rpc methods, `reserveinputs` and `unreserveinputs`, which allow for reserving or unreserving wallet UTXOs
2020-06-29 16:10:05 +02:00
niftynei
b90be4f6c8 prepare-tx: pass back the feerate, as json_tx_prepare sometimes sets it
Unused here, but we'll use it in the next commit so that we can always
pass back the effective / used feerate to the caller of `reserveinputs`

This makes opening a channel much easier if we've internally determined
the feerate
2020-06-29 16:10:05 +02:00
Rusty Russell
57f6f74ee0 external/Makefile: fix spurious rebuilds.
Quote marks are not special to make: as it can't find
external/"x86_64-linux-gnu"/libwally-core-build/src/libwallycore.la
it always insists on rebuilding it (which rebuilds the world).

If we have spaces in TARGET_DIR, we're in trouble already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-None
2020-06-29 15:48:09 +09:30
Wladimir J. van der Laan
32f9805a30 build: Make it possible to build without gcc
The external Makefile hardcodes gcc to get the machine spec.
This should use the configured C compiler instead.

This bug was introduced in 601464416b.
2020-06-29 09:08:39 +09:30
Vasil Dimov
a3a37ea4ce build: fix compilation on i386
```
plugins/keysend.c:136:47: error: format specifies type 'unsigned long' but the argument has type 'time_t' (aka 'int') [-Werror,-Wformat]
        ki->label = tal_fmt(ki, "keysend-%lu.%09lu", now.ts.tv_sec, now.ts.tv_nsec);
                                         ~~~         ^~~~~~~~~~~~~
                                         %d
```

Changelog-None
2020-06-28 09:36:30 +00:00
Rusty Russell
42bf230bfc doc: regenerate and update.
My node tried to rebuild one of them because it was lagging (I noticed
because mrkd was not installed).

Turns out a few were lagging, and lightning-dev-sendcustommsg.7 was totally
empty!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-28 01:14:08 +00:00
Rusty Russell
8ee8161fb4 pyln-spec: add .gitignore file for build detritus.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
447231f52e pyln.spec.bolt*: change version numbering to include specific csv subversion.
Now they look like 1.0.1.137, so you can explicitly depend on a csv change
(without caring about a textual change).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
1c1c3349b4 pyln.spec.bolt*: unify setups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
450fb43612 pyln.spec.bolt*: test-release and prod-release targets, fix requirements.txt
We depend on packages, not modules within them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
93ae190c19 pyln.proto: bump version to 0.8.3.
Changelog-Changed: pyln.proto version now 0.8.3 to indicate pyln.proto.message
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
014ede0585 pyln.proto: fix test-release target
1. version was 0.0.2 in setup.py, which means we didn't get the dist/ files we expected.
2. We need 'bdist_wheel' to make the .whl file.
3. --no-site-packaged was apparently removed in 0.20.0, and was default long before that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
902d8f7dab pyln.proto.message: remove incorrect fundamental types now spec update.
See 9e8e29af9b

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00