Commit Graph

5282 Commits

Author SHA1 Message Date
Christian Decker
e79fbb6812 pylightning: Fix missing encode if first pass fails
Without this the RPC will fail to continue buffering if the response does not
fit in the first read, and if we don't switch over to the non-compat
mode. This was introduced by our mitigation of the UTF-8 misalignment, but I
missed this path.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 11:05:24 -08:00
Christian Decker
998233efc0 changelog: Add changelog entry for PR #2371 2019-02-22 17:55:36 +01:00
Christian Decker
04ffa5a413 pylightning: Add some more documentation for the python lib
It was looking a bit minimal on pypi :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 17:55:36 +01:00
Christian Decker
816f20d1c0 docs: Autodetect version and copyright year in the sphinx doc
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Suggested-by: Rusty Russell <@rustyrussell>
2019-02-22 17:55:36 +01:00
Christian Decker
8fb2e6be7f fixup! jsonrpc: Arm the minconf=1 parameter and deal with the fallout 2019-02-22 10:40:59 +00:00
Christian Decker
72f1c78a1e jsonrpc: Arm the minconf=1 parameter and deal with the fallout
We want to disallow using unconfirmed outputs by default, so making the
default 1 confirmation seems a good idea. This also matches `bitcoind`s
minimum confirmation requirement.

Arming however breaks some of our tests, so I used `minconf=0` for the
breaking tests and added a new test specifically for the `minconf` parameter
for `fundchannel`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Christian Decker
683b601dc7 pylightning: Expose minconf in fundchannel and withdraw
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Christian Decker
c0d38aa99c jsonrpc: Expose the minconf parameter for fundchannel and withdraw
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Christian Decker
68fe5eacde wallet: Allow limiting the selection by confirmation height
This allows us to specify that an output must have been confirmed before the
given maximum height. This allows us to specify a minimum number of
confirmations for an output to be selected.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 10:40:59 +00:00
Michael Schmoock
4986d6b39d feat: block of newlines when attaching a logfile
This will make the logger write 4 newlines to re-attached logfiles.
The newlines wont appear on logfiles that are just created.
Additionally the server prints 50 '-' dashes before printing his
startup message, which also help increase readability on logfile.

This was inspired by the way Bitcoin Core handles logfiles.
2019-02-22 03:01:37 +00:00
Christian Decker
f622ffb087 pylightning: Add plugin dispatch tests to check-python and fix them
These weren't checked by CI yet, and they are really short so I just added
them to the check-python target.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
7f11b4854e pylightning: Split @method and @async_method decorators
Suggested-by: Rusty Russell <@rustyrussell>
Suggested-by: Conor Scott <@conscott>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
d9c3f5ec4b pylightning: Rename peer_id to node_id in getroute
Technically this is a node, not a direct peer, so this is correct.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
571fb44d20 pylightning: Add a small test for async rpcmethods
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
cc4fbfca43 pylightning: Add the background keyword to hooks and methods
This indicates that the method or hook will accepts a request
parameter, and will use that to return the result or raise an
exception instead of returning the return value. This allows the hook
or method to stash the incomplete request or pass it around, without
blocking the JSON-RPC interface.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
9670b4382f pylightning: Don't always use request ID 0
This isn't a problem for now since we don't support multithreading,
and only allow synchronous calls, but eventually this'll become
important.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
f452d00b77 pylightning: Add support for *args and **kwargs in plugin dispatch
These are a bit special and are handled separately.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
8de1a85ac0 pylightning: Exception if we have unfulfilled positional arguments
This caused me to backtrack quite a bit, so this should help debugging
in the future.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
81fa247d07 pylightning: Wrap request in an object
We well need this in the next commit to be able to return from an
asynchronous call. We also guard stdout access with a reentrant lock
since we are no longer guaranteed that all communication happens on
the same thread.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Christian Decker
71795d4d6a pylightning: Wrap the plugin methods in a class
Sending around unnamed tuples is bound to cause some issues sooner or
later, so we just create a quick class that holds all the information
about a plugin method.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 02:59:36 +00:00
Conor Scott
d96564ce3e pylightning: raise Error when missing rpc parameter 2019-02-21 14:23:01 +01:00
Rusty Russell
6ed54dd34e db: fix dangling peers on db upgrade.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
594f260de7 pytest: test starting with db corrupted by dangling peer.
db was taken from the failed test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
7887e5c00a pytest: add support for starting nodes with a pre-canned db.
With xz, the db is only 9120 bytes, vs 163840.  And lzma is a builtin
in Python 3.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
3b587a1c6d lightningd: fix db error where we can have detached peer.
An uncommitted channel should not keep the peer in the db, since the
uncommitted channel isn't in the db itself.

Fixes: #2367
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
26df586fbf pytest: add test for db constraint failure
wallet_channel_insert: UNIQUE constraint failed: peers.node_id
lightningd: Fatal signal 6 (version v0.6.3rc1-202-g4c8cb98)
0x555625b14261 crashdump
	common/daemon.c:40
0x7f8f7400e0ff ???
	???:0
0x7f8f7400e077 ???
	???:0
0x7f8f73fef534 ???
	???:0
0x555625af34f2 fatal
	lightningd/log.c:624
0x555625b3dd96 db_exec_prepared_
	wallet/db.c:448
0x555625b44fcd wallet_channel_insert
	wallet/wallet.c:1067
0x555625af64f7 wallet_commit_channel
	lightningd/opening_control.c:229
0x555625af6e86 opening_funder_finished
	lightningd/opening_control.c:394
0x555625af7bfa openingd_msg
	lightningd/opening_control.c:714
0x555625b0d421 sd_msg_read
	lightningd/subd.c:474
0x555625b0cd3b read_fds
	lightningd/subd.c:302
0x555625b548d3 next_plan
	ccan/ccan/io/io.c:59
0x555625b553ef do_plan
	ccan/ccan/io/io.c:395
0x555625b5542d io_ready
	ccan/ccan/io/io.c:405
0x555625b5700e io_loop
	ccan/ccan/io/poll.c:310
0x555625af1562 main
	lightningd/lightningd.c:827
0x7f8f73ff109a ???
	???:0
0x555625adaad9 ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20190220020526

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Conor Scott
af750d802d Switch to libsodium-dev for ubuntu install instructions 2019-02-21 09:14:39 +00:00
Rusty Russell
38e7d19dd5 Makefile: check for direct amount_sat/amount_msat access.
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
28f5da7b2f tools/generate-wire: use amount_msat / amount_sat for peer protocol.
Basically we tell it that every field ending in '_msat' is a struct
amount_msat, and 'satoshis' is an amount_sat.  The exceptions are
channel_update's fee_base_msat which is a u32, and
final_incorrect_htlc_amount's incoming_htlc_amt which is also a
'struct amount_msat'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
948ca470ad bitcoin: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
bac9a594b8 wallet: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
3ac0e814d0 daemons: use amount_msat/amount_sat in all internal wire transfers.
As a side-effect of using amount_msat in gossipd/routing.c, we explicitly
handle overflows and don't need to pre-prune ridiculous-fee channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
0d30b89043 channeld: use amount_msat for struct htlc amount.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
3412c5d392 commit_tx & htlc_tx: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
bb00deeea4 channeld: use amount_sat/amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
93dcd5fed7 channeld: avoid overflow in reloading of channel from db.
We used to just throw htlcs into the channel with a flag to tell it to
ignore overflow.  Instead, we can insert them in order (which is the same as
id order) which always must be valid.

This helps when we turn the balance into a struct amount_msat which will get
upset with overflows.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
b8e484b508 struct channel_config: use amount_sat / amount_msat.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
85b8b25749 bitcoin/chainparams: use amount_sat / amount_msat
Simple changes, but ripples through the code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
cd341b34d6 plugins/pay: use struct amount_msat.
This is particularly interesting because we handle overflow during route
calculation now; this could happen in theory once we wumbo.

It fixes a thinko when we print out routehints, too: we want to print
them out literally, not print out the effect they have on fees (which
is in the route, which we also print).

This ABI change doesn't need a CHANGELOG, since paystatus is new since
release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
cc95a56544 pylightning: handle msat fields in JSON more appropriately.
Little point having users handle the postfixes manually, this
translates them, and also allows Millisatoshi to be used wherever an
'int' would be previously.

There are also helpers to create the formatting in a way c-lightning's
JSONRPC will accept.

All standard arithmetic operations with integers work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
52c843f708 CHANGELOG, documentation: update changelog to reflect suffix changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
7e3928359a listpeers: add all the alternate "msat" and "sat" fields for channels.
These are undocumented, unfortunately, but at least that means I don't
have to update the docs!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
b5dcb93e5f wallet: use amount_msat / amount_sat.
We change struct utxo to use amount_sat, and paper over the JSON APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
83adb94583 lightningd and routing: use struct amount_msat.
We use it in route_hop, and paper over it in the JSON APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
feb92cf4e2 sendpay: allow 'amount_msat'
We're about to add 'amount_msat' to getroute, but it's common to feed
'getroute' back into 'sendpay', so sendpay should allow it.

If both are specified, make sure they're the same!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
2632cc3f34 lightningd/json: make wallet_tx functions take amount_sat.
Using param_tok is generally deprecated, as it doesn't give any sanity checking
for the JSON 'check' command.  So make param_wtx usable directly, and
also make it have a struct amount_sat.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
3ba544bfde common/bolt11: use struct amount_msat
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
28ec65fd79 lightningd: add json_add_amount_msat and json_add_amount_sat helpers.
These create two fields, one old one which is purely numeric,
and a modern on with a suffix, eg "msatoshi" and "amount_msat".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
177cfd9edc common: json_to_msat and json_to_sat helpers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00
Rusty Russell
a25e22737c common/json_tok: add param_msat / param_sat.
The current param_sat accepts "any": rename and move that to invoice.c
where it's called.  We rename it to param_msat_or_any and invoice.c
is our first (trivial) amount_msat user.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 03:44:44 +00:00