Commit Graph

92 Commits

Author SHA1 Message Date
Sergi Delgado Segura
27e495efa5 pyln: Updates proto to use coincurve for Public and Private keys 2020-08-26 06:09:53 +09:30
Rusty Russell
06372e13d8 pyln.proto.message: don't let Message() init set implicit lengths.
We'll override them from field length anyway!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 17:53:25 +02:00
Rusty Russell
74abd30da5 pyln-proto, pyln-spec: fix 'make prod-release' target.
rusty$ make prod-release
  make: *** No rule to make target 'test', needed by 'prod-release'.  Stop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-29 18:01:12 +02:00
Rusty Russell
93ae190c19 pyln.proto: bump version to 0.8.3.
Changelog-Changed: pyln.proto version now 0.8.3 to indicate pyln.proto.message
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
014ede0585 pyln.proto: fix test-release target
1. version was 0.0.2 in setup.py, which means we didn't get the dist/ files we expected.
2. We need 'bdist_wheel' to make the .whl file.
3. --no-site-packaged was apparently removed in 0.20.0, and was default long before that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
902d8f7dab pyln.proto.message: remove incorrect fundamental types now spec update.
See 9e8e29af9b

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
41e914fb37 pyln.proto.message, pyln.proto.spec*: do magic to expose mypy types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
11a0de877e pyln.proto.message: more mypy fixes.
This includes some real bugfixes, since it noticed some places we were
being loose with different types!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
3882e8bdf7 pyln.spec.bolt*: make these separate packages, so versions can change indep.
They're almost entirely autogenerated, and we use symlinks into the
top directory to reduce replication.

They can't be under pyln.spec.message, because a package can't also
be a namespace.

We also add fulltext and desc fields, and exclude our "gen" files from
flake8, since the spec quotes contain weird whitespace.

Changelog-Added: Python: pyln.spec.bolt{1,2,4,7} packages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
013eced4ce pyln.proto: fix package list in setup.py
Remove non-existant pyln.proto.bolts.  bolts will have separate setup.py, so we
can rev the versions individually.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-25 10:08:58 +02:00
Rusty Russell
02338a6b25 pyln.proto.message: add to_py() operation.
This delivers the message contents in a much friendlier form for
manipulation: in particular, it makes it easy to compare two
messages without having to know all the message type internals.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-16 16:17:52 +02:00
Rusty Russell
ee76504e53 pyln.proto.message: fix handling of ... with subtypes.
This time, with a test!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-16 16:17:52 +02:00
Rusty Russell
2309a7a561 pyln.proto.message: don't leave 'None' in dict for missing fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-16 16:17:52 +02:00
Christian Decker
5bb1fd4205 pyln-proto: Add Makefile targets for releases 2020-06-15 16:14:53 +02:00
Christian Decker
48cee3c3d2 pyln: Bump version to 0.8.2 in order to match with binary distro
Keeping them out of sync seems like a great way to create confusion. Let's
sync them up whenever possible.
2020-06-15 16:14:53 +02:00
Rusty Russell
aaefbe2e9e pyln.proto.message: fix handling of missing optional fields.
If they don't exist, that's OK. These will eventually be going away
from the spec, but there are still some in gossip messages for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
fd3ea91b44 pyln.proto.message: expose array types, add set_field for Message class.
Exposing the array types is required for our dummyrunner in the lnprototest suite, since
it wants to be able to generate fake fields.

The set_field is similarly useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
f52065201b pyln.proto.message.*: add type annotations.
Other changes along the way:

1. In a couple of places we passed None as a dummy for for
   `otherfields` where {} is just as good.
2. Turned bytes into hex for errors.
3. Remove nonsensical (unused) get_tlv_by_number() function from MessageNamespace
4. Renamed unrelated-but-overlapping `field_from_csv` and
   `type_from_csv` static methods, since mypy thought they should have
   the same type.
5. Unknown tlv fields are placed in dict as strings, not ints, for
   type simplicity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
da070e73b2 pyln.proto.message.*: Add Makefile to do mypy checks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
acfeaebb62 pyln.proto.message: allow fields with options to be missing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
ee6c58cbd5 pyln.proto.message: export more.
FieldType lets you make new field types, and split_field helps with
parsing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
b33dc9847c pyln.proto.message: python-fluency feedback from @darosior
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
8f38bc7230 pyln.proto.message: support adding two namespaces.
They must not have duplicate names!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
85bb93618b new modules: pyln.proto.message.{bolt1,bolt2,bolt4,bolt7}
These are autogenerated, but now they export their own
MessageNamespace, as well as the raw csv.

They also expose their SubtypeTypes, MessageTypes and TlvStreamTypes,
though in theory these could clash (they don't for now, and it'd be
kinda awkward if they did).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
59bb6b90ac pyln: new module pyln.proto.message.bolts
This contains the CSVs for the current bolts (autogenerated).  It's a
separate module because I expect it to be updated alongside the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: new module pyln.proto.message.bolts
2020-06-12 15:35:35 +02:00
Rusty Russell
daa707d213 pyln.proto.message: separate fundamental types from other subtypes.
This will be useful for the next patch, which introduces per-bolt
modules.  This makes it easier for them generate variables for each
field type they parse (they don't want to export u16, for example)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
e4b5679f5d message: support option fields.
These are (probably) going away soon, but just tag them for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
9992a577b4 pyln: add (undocumented) u8 fundamental type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
42aab89b9f pyln.proto.message: expose fundamental MessageTypes as variables.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
47631cc23c pyln.proto.message: use BufferedIOBase instead of bytes for binary ops.
Instead of val_to_bin/val_from_bin which deal with bytes, we implement
read and write which use streams.  This simplifies the API. 

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
ed4eadc8f3 patch message-export-types.patch 2020-06-12 15:35:35 +02:00
Rusty Russell
eb73a0dd8f pyln: add pyln.proto.message.
This supports infrasructure for creating messages.  In particular, it
can be fed CSV from the spec's `tools/extract-formats.py` and then convert
them all to and from strings and binary formats.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: new module pyln.proto.message
2020-06-12 15:35:35 +02:00
Rusty Russell
5b4be02ed3 pyln: add Makefile
This runs flake8 and the python tests.  Helps me, at least!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-12 15:35:35 +02:00
Rusty Russell
97e3d61748 pyln: fix incorrect python syntax.
contrib/pyln-proto/pyln/proto/bech32.py:120
  /home/rusty/devel/cvs/lightning/contrib/pyln-proto/pyln/proto/bech32.py:120: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    assert decode(hrp, ret) is not (None, None)

I think this warning is correct (though I don't see the warning once I installed coincurve:
are we suppressing warnings?)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-15 15:21:50 +02:00
Michal Rostecki
db0a2c082a pyln-proto: Use only coincurve for libsecp256k1 bindings
secp256k1 Python library is not maintained anymore and coincurve was
already used in the `wire` module.

Changelog-None
Signed-off-by: Michal Rostecki <mrostecki@mailfence.com>
2020-05-07 11:37:34 +02:00
Christian Decker
9bfdf234f3 pyln-proto: Avoid circular dependency in setup.py
It seems that loading the version from the source files triggers imports that
may not yet have been installed.
2020-05-05 13:28:54 +09:30
Christian Decker
f3315ca110 pyln-proto: Add Tu32Field and Tu64Field helpers
These are necessary for the interim keysend plugin
2020-04-16 18:03:35 +09:30
Michael Schmoock
3e2607bf2d chore: updates coincurve requirement to v13
Update to v13 as v12 started to raise undefined symbol exceptions agains latest
libsecp256k1.

Note: any version of `pip install coincurve` fails if no libsecp256k1 headers
are installed, should we point this out somewhere/somehow?

Changelog-None
2020-02-18 10:10:17 +10:30
Christian Decker
966ac95098 pytest: Add a test for plugin featurebits 2020-02-11 13:53:31 +10:30
Christian Decker
6c19818314 pyln-proto: Add invoice utilities
We are about to disect a couple of invoices for features, so let's add a class
that can encode and decode invoices from bolt11 strings. This is pretty much
the lnaddr.py file created by @rustyrussell with some minor changes. I'm
planning to clean this up further which is why I'm only exporting the
`Invoice` class for now.
2020-02-01 16:50:58 +01:00
Christian Decker
d3f6ebf911 pyln-proto: Added a couple of utilities to manage onions and zbase32 2020-01-28 13:43:44 +10:30
Christian Decker
3418e59d76 pyln: Split pylightning into multiple pyln modules
This is the first step to transition to a better organized python module
structure. Sadly we can't reuse the `pylightning` module as a namespace module
since having importable things in the top level of the namespace is not
allowed in any of the namespace variants [1], hence we just switch over to the
`pyln` namespace. The code the was under `lightning` will now be reachable
under `pyln.client` and we add the `pyln.proto` module for all the things that
are independent of talking to lightningd and can be used for protocol testing.

[1] https://packaging.python.org/guides/packaging-namespace-packages/

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-30 13:27:37 +02:00