Commit Graph

433 Commits

Author SHA1 Message Date
ZmnSCPxj jxPCSnmZ
4a89819cdd plugins/Makefile: Change plugins build procedure.
Changelog-None
2020-07-07 21:32:31 +02:00
Christian Decker
6fbfe24e3e paymod: Add a COMPAT_090 flag
Since we end up consolidating some of the return values for `pay` and
`paystatus` and change the public interface we need to add the compatibility
flag and guard the switchover behind it.
2020-07-02 11:54:12 +02:00
ZmnSCPxj jxPCSnmZ
5d720536e2 Makefile: Install tools/hsmtool as lightning-hsmtool.
Changelog-Added: We now install `lightning-hsmtool` for your `hsm_secret` needs.

See: https://github.com/ElementsProject/lightning/issues/3717#issuecomment-644844594

It seems reasonable to add this to the standard install, and to document it properly as well, hopefully we can fill in the documentation better later on.
2020-07-01 09:24:03 +09:30
Rusty Russell
3882e8bdf7 pyln.spec.bolt*: make these separate packages, so versions can change indep.
They're almost entirely autogenerated, and we use symlinks into the
top directory to reduce replication.

They can't be under pyln.spec.message, because a package can't also
be a namespace.

We also add fulltext and desc fields, and exclude our "gen" files from
flake8, since the spec quotes contain weird whitespace.

Changelog-Added: Python: pyln.spec.bolt{1,2,4,7} packages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
niftynei
47c701cbe6 make: have update-mocks actually be quiet
Shhhh
2020-05-19 20:25:23 +09:30
Rusty Russell
7ed3e72b88 Makefile: don't print out what we're doing if --quiet/-s
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-19 20:25:23 +09:30
niftynei
66f59659a7 setup: add setup to make checks
- we've moved tmpctx management to setup.c from daemon.c, so we update
the `check-tmpctx`
- `common_setup(char *)` is now a valid analog for `setup_locale`, so we
check for either in check-setup_locale
2020-05-19 13:35:42 +09:30
William Casarin
d6ede8db1b build: cleanup compiler output
Hide CFLAGS and LDFLAGS line noise each time an object file is compiled
or linked.

Also add a `make show-flags` command for displaying CC, LD, CFLAGS and
LDFLAG information. This is shown at the start of each build.

Use `V=1 make` to restore original output

make

  CC: gcc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning" -Wall [..]
  LD: gcc   -Og    -Lexternal -lwallycore -lsecp256k1 -ljsmn  [..]
  ...
  cc wallet/test/run-db.c
  cc lightningd/test/run-jsonrpc.c
  cc lightningd/test/run-invoice-select-inchan.c
  cc lightningd/test/run-log-pruning.c
  cc lightningd/test/run-find_my_abspath.c
  cc cli/test/run-large-input.c
  cc cli/test/run-remove-hint.c
  ld lightningd/lightning_hsmd
  ld lightningd/lightning_gossipd
  ld lightningd/lightning_openingd
  ld lightningd/lightning_channeld
  ld lightningd/lightning_closingd
  ...

Signed-off-by: William Casarin <jb55@jb55.com>
Changelog-Changed: build: default compile output is prettier and much less verbose
2020-05-14 17:44:01 +02:00
Rusty Russell
1f2740efe8 Makefile: COMPAT_V082.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-07 08:44:58 +09:30
Christian Decker
f1633de6e8 plugin: Add the keysend plugin to the list of installable plugins
Suggested-by: Matt Whitlock <@whitslack>
Signed-off-by: Christian Decker <@cdecker>
2020-04-20 07:29:07 +09:30
Christian Decker
b68066e8e8 python: Consolidate requirements.txt files in a single place
We had them split according the separate use-cases:

 - testing
 - doc-gen
 - wire-gen

But that was causing new contributors to miss some dependencies when they
first got hacking. So this consolidates all of our own dependencies in a root
requirements.txt, with the notable exception of `pyln-client`, `pyln-testing`
and `pyln-proto` which are distributed as PyPI modules and therefore have
their own dependencies that need to be tracked in the module root.

Closes #3518
2020-04-09 15:14:06 +02:00
Rusty Russell
d881a4bd66 BOLT: update to latest version.
This is all typo/clarity fixes, no substantive changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-31 13:36:02 +02:00
Christian Decker
3e3b05e1b2 pyln: Migrate remaining uses of the deprecated pylightning module
`pylightning` is not much more than an alias for `pyln-client`, so this
removes the need to install that as well just to run the tests.
2020-03-24 09:52:33 +10:30
Christian Decker
15ca3b615f make: Add compat flag for v0.8.1 2020-03-10 11:42:58 +10:30
Vasil Dimov
820f1b2f9d build: send cmp's stderr to /dev/null also
It was probably a typo to have 2>&2

Changelog-None
2020-03-05 14:58:57 -06:00
Vasil Dimov
79ba4866ec build: unbreak if CFLAGS is defined in make's env
Don't let make pollute subprojects' environment with our own `CFLAGS`,
which are quite strict because that breaks at least libwally-core:

```sh
$ ./configure ...
$ CFLAGS=whatever_this_is_irrelevant make
...
cd external/libwally-core-build && ../libwally-core/configure ...
...
  CFLAGS                  = -DBINTOPKGLIBEXECDIR="\"../libexec/c-lightning\"" -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -std=gnu11 -g -fstack-protector  -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/libbacktrace-build -I . -I/usr/local/include   -DCCAN_TAKE_DEBUG=1 -DCCAN_TAL_DEBUG=1 -DCCAN_JSON_OUT_DEBUG=1 -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS   -DBUILD_ELEMENTS=1 -W -std=c89 -pedantic -Wall -Wextra -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wno-unused-function -Wno-long-long -Wno-overlength-strings -fvisibility=hidden -O3
...
In file included from ../../libwally-core/src/base58.c:4:
../../libwally-core/src/ccan/ccan/endian/endian.h:71:24: error: unused function 'bswap_16'
      [-Werror,-Wunused-function]
static inline uint16_t bswap_16(uint16_t val)
                       ^

```

If `CFLAGS` is set in its environment, then `make` would export our own
`CFLAGS` to any subprocesses it starts, which means subprojects would
inherit our `CFLAGS="-Wall -Werror"` in their environments.

GNU Make's documentation:
https://www.gnu.org/software/make/manual/html_node/Variables_002fRecursion.html#Variables_002fRecursion
> make exports a variable only if it is either defined in the environment initially...

Example:
```make
A = x
default:
	echo $$A
```

then:

```sh
$ make # prints nothing, A is not exported to the subprocess
$ A=y make # prints "x", our A=x is exported to the subprocess
```

Changelog-None
2020-02-22 08:49:57 +10:30
Rusty Russell
d478dc0e59 wallet: fix null column access for pre-0.7.3 dbs.
Added in d901304120, this column is null in old dbs like mine:

2020-02-15T00:08:41.444Z **BROKEN** database: Accessing a null column 12 in query SELECT  id, channel_htlc_id, msatoshi, cltv_expiry, hstate, payment_hash, payment_key, routing_onion, failuremsg, malformed_onion, origin_htlc, shared_secret, received_time FROM channel_htlcs WHERE direction= ? AND channel_id= ? AND hstate != ?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-18 10:22:58 +10:30
darosior
70a79e3998 plugins/bcli: a new plugin for gathering Bitcoin data
Most is taken from lightningd/bitcoind and adapted. This currently
exposes 5 commands:
- `getchaininfo`, currently called at startup to check the network and
  whether we are on IBD.
- `getrawblockbyheight`, which basically does the `getblockhash` +
  `getblock` trick.
- `getfeerate`
- `sendrawtransaction`
- `getutxout`, used to gather infos about an output and currently used by
  `getfilteredblock` in `lightningd/bitcoind`.
2020-02-12 11:45:07 +10:30
Christian Decker
966ac95098 pytest: Add a test for plugin featurebits 2020-02-11 13:53:31 +10:30
darosior
da476848d1 pytest: add a C testing plugin to test libplugin
We mark the test as xfail() as it exposes that libplugin's PLUGIN_RESTARTABLE
was not taken into account !
2020-02-10 09:49:15 +10:30
Niklas Claesson
f9b3b96a63 external: Reorganize to support cross-compilation
On CI it is nice to cache the external dependencies. However if we
always compile them in the same folder we cannot cache for multiple
different architectures. After this commit native compile targets will
still live in `external` but cross compiled versions will live in
`external/<arch>`.
2020-02-04 11:16:02 +10:30
Vasil Dimov
1015e3774a build: avoid repetition of LC_ALL=C sort
Put `LC_ALL=C sort` in a variable and use it everywhere. It was
forgotten in one place.

Without `LC_ALL=C` it would order `a.b` before `a-c` even though
`. (0x2E)` > `- (0x2D)`.

Changelog-None
2020-02-03 15:38:11 +00:00
Christian Decker
24219b38f9 pyln: Add unit tests from pyln-* to make check-python 2020-02-01 16:50:58 +01:00
Rusty Russell
f373cd45be Makefile: update BOLTVERSION to latest master.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell
6450a7e315 lightningd: update spec and remove allowance for final node to receive a fee.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Now correctly reject "fees" paid when we're the final hop (lightning-rfc#711)
2020-01-31 06:07:56 +00:00
Rusty Russell
ff1e1dd487 Makefile: update BOLTVERSION to flatten features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell
a0a2c257be Makefile: add a COMPAT_V080 flag.
We're going to need this in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Vasil Dimov
8adbd4d31e doc: also build docs when building everything else
Before this change `make` would build everything but the docs.
They would be built during `make install` which is unexpected.

Add a dependency to `doc-all` to the `default` target so that docs are
build together with the rest of the programs. Leave the `install-data`
as is, to still build the docs if they are not built yet during install.

Changelog-None
2020-01-15 12:34:57 +01:00
Vasil Dimov
899f5dee53 build: fix version extraction from non-git directory
https://github.com/ElementsProject/lightning/releases/download/v0.8.0/clightning-v0.8.0.zip
extracts into "clightning-v0.8.0".

https://github.com/ElementsProject/lightning/archive/v0.8.0.tar.gz
extracts into "lightning-0.8.0".

Tweak the regular expression in the Makefile to support both (extract
0.8.0 from either one). Also, downloading an arbitrary version, based on
git commit id:

https://codeload.github.com/ElementsProject/lightning/tar.gz/eb2b225c?dummy=/filename_this_is_irrelevant.tar.gz
extracts into "lightning-eb2b225c".

so, support extracting "eb2b225c" from that.

Changelog-None
2020-01-06 12:59:20 +01:00
Rusty Russell
fb040bc619 Spec: trivial update to latest.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Rusty Russell
d512bcb85f lightningd: automatically move files for existing deployments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-23 22:42:34 +00:00
Rusty Russell
eaa72edbd6 pytest: fix make pytest
Travis adds the correct PYTHONPATH, but "make check" doesn't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-13 10:59:05 +01:00
Christian Decker
b31defdf96 make: Add in-tree pyln-testing to python path
We'll start relying on the in-tree version pretty soon, so we better make
sure we find it.
2019-11-12 21:23:55 +01:00
Christian Decker
81f7978cc8 pyln: Migrate implementation from pylightning to pyln-client
This should not affect any consumer of the API since we just shift the actual
implementation from one side to the other, and keep aliases in place so
scripts don't break.

We also bump the version number from 0.0.7.3 to 0.7.4 which allows us to be in
sync with c-lightning itself, and remove the superfluous `0` in front.
2019-11-12 21:23:55 +01:00
Rusty Russell
c617f19a29 Makefile: fix static linking.
And update --enable-static, since it doesn't actually create a static
binary since 65fa1c4f39.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 14:04:43 -05:00
Christian Decker
357322d721 make: Enable the elements flag in the lightningd build
Without this we wouldn't see the elements specific fields and functions from
lightningd.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-10-03 04:32:57 +00:00
Rusty Russell
15612d269a Make option_static_remotekey non-EXPERIMENTAL now it's in spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 01:50:44 +00:00
Christian Decker
62dc8dc110 make: Add configuration detection and linking of libpq
If we have the client library for postgres configure will define HAVE_POSTGRES
the same way it already handled libsqlite3 an we start linking against it.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-22 02:03:43 +00:00
Rusty Russell
895e552475 BOLT: update to master with gossip_queries_ex.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
8fdfdcec74 BOLT: Update to include BOLT11 features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
e5564173e7 BOLT: update to cover minor changes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Jacob Rapoport
6360ffb2ac fix macOS build & update docs 2019-09-16 05:08:42 +00:00
darosior
90fa2ae3fd plugins: load fundchannel by default
Reported by @darwin
2019-09-16 11:59:32 +08:00
Rusty Russell
e467e6a5c6 Makefile: fix race if we run make update-mocks in a clean tree.
/bin/sh: 1: ccan/ccan/cdump/tools/cdump-enumstr: Text file busy
    make[1]: *** [common/Makefile:81: common/gen_htlc_state_names.h] Error 2
    make[1]: *** Waiting for unfinished jobs....

The fix is to make sure all generated headers are made first, and
thus cdump-enumstr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Christian Decker
7fafd934cf pytest: Remove --duration argument
We're not going to optimize travis any longer so let's no fill logs with
useless measurements.
2019-09-02 16:31:36 +02:00
Rusty Russell
6349222ea2 Spec: Update to latest BOLT, include our first global feature definition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
394aef2391 Makefile: fix race with 'make check'
installcheck calls $(MAKE) under the covers, which can race with the
current builds, and we can try installing something which is still
being built:

    collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
    compilation terminated.
    /usr/bin/ld: can not read symbols: File truncated
    /usr/bin/ld: .eh_frame/.stab edit: File truncated
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.interp' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.ABI-tag' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.gnu.build-id' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.hash' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynsym' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynstr' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version_r' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.dyn' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.text' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rodata' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.eh_frame' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data.rel.ro' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynamic' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_json_command' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_hooks' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_type_to_string' not in segment
    Makefile:390: recipe for target 'lightningd/lightningd' failed
    make[1]: *** [lightningd/lightningd] Error 1
    make[1]: Leaving directory '/home/rusty/lightning'
    Makefile:553: recipe for target 'installcheck' failed
    make: *** [installcheck] Error 2

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
9dfc854d94 tools/Makefile: put all tools/ stuff here.
Generally a Makefile should control make within that directory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00
Rusty Russell
0fdf22988d configure: absorb python3-mako check.
This is a better place for such checks than the Makefile, and we'll likely
have more in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-21 05:07:15 +00:00