Commit Graph

7979 Commits

Author SHA1 Message Date
Saibato
eea6b33154 connect/tor: Set errno, if connection failed we want to reflect this.
Changelog-None:

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
Saibato
2b0aba13a8 connectd/connectd: Display an error hint for V3 tor onions when connect fail.
@thestick613 noticed that since tor version below 0.3.2.2-alpha
will not support V3 ed25519 address formats, the error handling
is not that helpful in the error message from cli.
So now we add an hint.

Changelog-None:

Signed-off-by: Saibato <saibato.naga@pm.me>

connectd/connectd.h; Add helper function to update conn error list

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
Saibato
e4cd2148ab Reflect in TOR.md that we need a Tor version >= 0.3.2.2-alpha to reach V3 onions
Changelog-None

Signed-off-by: Saibato <saibato.naga@pm.me>
2020-07-01 11:21:58 +02:00
ZmnSCPxj jxPCSnmZ
5d720536e2 Makefile: Install tools/hsmtool as lightning-hsmtool.
Changelog-Added: We now install `lightning-hsmtool` for your `hsm_secret` needs.

See: https://github.com/ElementsProject/lightning/issues/3717#issuecomment-644844594

It seems reasonable to add this to the standard install, and to document it properly as well, hopefully we can fill in the documentation better later on.
2020-07-01 09:24:03 +09:30
Christian Decker
7b899da801 db: Retrieve peer ID if it exists or create the peer if not
We were assuming `wallet_channel_insert` that there cannot be a matching peer
if our in-memory representation isn't bound to it (`dbid == 0`). If we then
attempt to create the peer, and we already had one it'd cause a unique
constraint violation. As far as I can tell this could end up happening if we
have an uncommitted channel, and then exited without cleanup (`tal_destructor`
on the uncommitted channel not running). This could then leave the peer in the
DB. This is because the constraint that every peer has at least one channel is
not enforce at DB level, but rather in destructors that may or may not run.

Changelog-Fixed: Fixed a failing assertion if we reconnect to a peer that we had a channel with before, and then attempt to insert the peer into the DB twice.
2020-07-01 09:22:29 +09:30
Rusty Russell
ae0cccb293 wallet: fix memleak if we get shut down before finishing utxo cleanup.
"backtrace": [
                       "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
                       "wallet/wallet.c:154 (wallet_stmt2output)",
                       "wallet/wallet.c:275 (wallet_get_utxos)",
                       "wallet/wallet.c:3792 (wallet_clean_utxos)",
                       "lightningd/lightningd.c:914 (main)"
                   ],
                   "label": "wallet/wallet.c:154:struct utxo",
                   "parents": [
                       "wallet/wallet.c:273:struct utxo*[]"
                   ],
                   "value": "0x24c1be8"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-01 09:19:39 +09:30
ZmnSCPxj jxPCSnmZ
d14460ef3d wallet/walletrpc.c: Show input annotations for inputs.
Changelog-None
2020-06-30 17:59:17 +02:00
Rusty Russell
6b3bcd49d2 tests: fix flake in test_penalty_htlc_tx_fulfill
If the daemon already knows about the channel before it was stopped,
it won't get this message from gossipd.  That's OK, since we explicitly
test for the channel being active two lines down.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-30 13:50:30 +00:00
Rusty Russell
dd4a1a3510 tests: fix flake in test_txprepare_restart
Detection of reserved outputs is async, going via bitcoind.  Wait for them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-30 13:50:30 +00:00
Rusty Russell
74abd30da5 pyln-proto, pyln-spec: fix 'make prod-release' target.
rusty$ make prod-release
  make: *** No rule to make target 'test', needed by 'prod-release'.  Stop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-29 18:01:12 +02:00
niftynei
2e9c387f45 coin_moves: update withdrawal logic to account for 'variable owner' txs
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)

Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
2020-06-29 16:10:05 +02:00
niftynei
ee549a2af9 tx: fix case where input amounts are less than total outputs
when attempting to calculate the fees for a tx where we don't own all of
the outputs, we can overshoot the feerate
2020-06-29 16:10:05 +02:00
niftynei
9830c94778 rpc: new signpsbt + sendpsbt rpcs
Changelog-Added: JSON-RPC: new call `signpsbt` which will add the wallet's signatures to a provided psbt
Changelog-Added: JSON-RPC: new call `sendpsbt` which will finalize and send a signed PSBT
2020-06-29 16:10:05 +02:00
niftynei
fd8a716695 wallet: have wallet_extract_outputs take wally_tx, not bitcoin_tx
With the incursion of PSBTs, we're moving away from bitcoin_tx
2020-06-29 16:10:05 +02:00
niftynei
0388fe6db4 tx: add helper for extracting script from a wally_tx
the bitcoin_tx version is basically a wrapper for the wally_tx script
extraction -- here we pull it apart so we can easily get a tal'd script
for a wally_tx_output
2020-06-29 16:10:05 +02:00
niftynei
a68d79e390 add manpages for reserve/unreserve 2020-06-29 16:10:05 +02:00
niftynei
103dce63ef reserve/unreserve input: new RPC commands for reserving inputs/outputs
Reserve and unreserve wallet UTXOs using a PSBT which includes those
inputs.

Note that currently we unreserve inputs everytime the node restarts.
This will be addressed in a future commit.

Changelog-Added: JSON-RPC: Adds two new rpc methods, `reserveinputs` and `unreserveinputs`, which allow for reserving or unreserving wallet UTXOs
2020-06-29 16:10:05 +02:00
niftynei
b90be4f6c8 prepare-tx: pass back the feerate, as json_tx_prepare sometimes sets it
Unused here, but we'll use it in the next commit so that we can always
pass back the effective / used feerate to the caller of `reserveinputs`

This makes opening a channel much easier if we've internally determined
the feerate
2020-06-29 16:10:05 +02:00
Rusty Russell
57f6f74ee0 external/Makefile: fix spurious rebuilds.
Quote marks are not special to make: as it can't find
external/"x86_64-linux-gnu"/libwally-core-build/src/libwallycore.la
it always insists on rebuilding it (which rebuilds the world).

If we have spaces in TARGET_DIR, we're in trouble already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-None
2020-06-29 15:48:09 +09:30
Wladimir J. van der Laan
32f9805a30 build: Make it possible to build without gcc
The external Makefile hardcodes gcc to get the machine spec.
This should use the configured C compiler instead.

This bug was introduced in 601464416b.
2020-06-29 09:08:39 +09:30
Vasil Dimov
a3a37ea4ce build: fix compilation on i386
```
plugins/keysend.c:136:47: error: format specifies type 'unsigned long' but the argument has type 'time_t' (aka 'int') [-Werror,-Wformat]
        ki->label = tal_fmt(ki, "keysend-%lu.%09lu", now.ts.tv_sec, now.ts.tv_nsec);
                                         ~~~         ^~~~~~~~~~~~~
                                         %d
```

Changelog-None
2020-06-28 09:36:30 +00:00
Rusty Russell
42bf230bfc doc: regenerate and update.
My node tried to rebuild one of them because it was lagging (I noticed
because mrkd was not installed).

Turns out a few were lagging, and lightning-dev-sendcustommsg.7 was totally
empty!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-28 01:14:08 +00:00
Rusty Russell
8ee8161fb4 pyln-spec: add .gitignore file for build detritus.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
447231f52e pyln.spec.bolt*: change version numbering to include specific csv subversion.
Now they look like 1.0.1.137, so you can explicitly depend on a csv change
(without caring about a textual change).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
1c1c3349b4 pyln.spec.bolt*: unify setups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
450fb43612 pyln.spec.bolt*: test-release and prod-release targets, fix requirements.txt
We depend on packages, not modules within them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
93ae190c19 pyln.proto: bump version to 0.8.3.
Changelog-Changed: pyln.proto version now 0.8.3 to indicate pyln.proto.message
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
014ede0585 pyln.proto: fix test-release target
1. version was 0.0.2 in setup.py, which means we didn't get the dist/ files we expected.
2. We need 'bdist_wheel' to make the .whl file.
3. --no-site-packaged was apparently removed in 0.20.0, and was default long before that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
902d8f7dab pyln.proto.message: remove incorrect fundamental types now spec update.
See 9e8e29af9b

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
41e914fb37 pyln.proto.message, pyln.proto.spec*: do magic to expose mypy types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
11a0de877e pyln.proto.message: more mypy fixes.
This includes some real bugfixes, since it noticed some places we were
being loose with different types!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
3882e8bdf7 pyln.spec.bolt*: make these separate packages, so versions can change indep.
They're almost entirely autogenerated, and we use symlinks into the
top directory to reduce replication.

They can't be under pyln.spec.message, because a package can't also
be a namespace.

We also add fulltext and desc fields, and exclude our "gen" files from
flake8, since the spec quotes contain weird whitespace.

Changelog-Added: Python: pyln.spec.bolt{1,2,4,7} packages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
013eced4ce pyln.proto: fix package list in setup.py
Remove non-existant pyln.proto.bolts.  bolts will have separate setup.py, so we
can rev the versions individually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Niklas Claesson
601464416b travis: Actually build for arm
Before this we cached the whole `external` directory, but that lead to
stale files since the cache is checked out after the repository is
cloned.
2020-06-24 17:08:40 +02:00
Antoine Poinsot
e0aef66ce2 plugins: make the autoclean plugin static
The autocleaning will only happen if the autocleaninvoice-cycle startup
option is passed, which cannot happen if the plugin is started
post-startup.
Thus, it's less misleading for users to restrict its usage to startup.

Changelog-Added: plugins: The `autoclean` plugin is now static (you cannot manage it with the `plugin` RPC command anymore).

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-06-24 15:29:39 +02:00
Rusty Russell
93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
niftynei
a66415a43e plugins-test: use pyln.client, not lightning to import Plugin 2020-06-23 14:49:32 +02:00
niftynei
0f8157a02e libwally: update to latest commit on master, which contains psbt fixes
fixes for p2sh-p2wpkh sigs in libwally etc
2020-06-23 14:49:32 +02:00
niftynei
0bd0de54fa psbt: have withdraw_tx use psbt's to create signed txs
this will allow us to add inputs that aren't ours to a tx that we sign
and finalize
2020-06-23 14:49:32 +02:00
niftynei
0a7550473b psbt: add redeemscript info to psbt for utxos that have it 2020-06-23 14:49:32 +02:00
niftynei
85dca84327 psbt-json: remove reliance on bitcoin_tx, use straight wally_psbt struct 2020-06-23 14:49:32 +02:00
niftynei
175fcf381a psbt: have wally_tx serialization methods be legible for gen'd code
our code generators expect the serialization name to match the struct
type
2020-06-23 14:49:32 +02:00
niftynei
b63abef542 psbt: add method to finalize + extract a psbt
will either use a temporary psbt (and not munge the passed in psbt)
or will finalize in place -- finalization erases most of the signature
metadata from the psbt struct
2020-06-23 14:49:32 +02:00
niftynei
0d1f1bc66e psbt: add helper method for setting a psbt input's redeemscript 2020-06-23 14:49:32 +02:00
niftynei
c3ae44e296 psbt: don't crash if we can't add a partial sig
instead return a boolean indicating the success/failure of a sig set
2020-06-23 14:49:32 +02:00
niftynei
80072b389e psbt: remove script sig info from inputs before adding them to global
PSBT's dont' serialize / unserialize if there's any sig info set on the
global transaction
2020-06-23 14:49:32 +02:00
lisa neigut
16656a85cf withdraw: refactor change output handling
We're not using the change_outnum for withdraw tx's (and the way
we were calculating it was broken as of the addition of 'multiple
outputs'). This removes the change output knowhow from withdraw_tx
entirely, and pushes the responsibility up to the caller to
include the change output in the output set if desired.

Consequently, we also remove the change output knowhow from hsmd.
2020-06-23 14:49:32 +02:00
niftynei
09815c7e7f psbt: return NULL instead of aborting on wally-lib problems
This lets us parse invalid/bad psbt data from user input without
crashing
2020-06-23 14:49:32 +02:00
niftynei
5ecacf3dd0 psbt: add type-to-string that prints b64 string
Re-uses code from what was the bitcoin_tx_to_psbt_b64
2020-06-23 14:49:32 +02:00
ZmnSCPxj jxPCSnmZ
c100de6d93 common/utils.c: Correctly handle NULL taken pointer in tal_dup_talarr.
Fixes: #3757

Reported-by: @sumBTC

Changelog-None
2020-06-23 12:48:20 +09:30