mirror of
https://github.com/ElementsProject/lightning.git
synced 2025-03-04 03:03:51 +01:00
db843159ea
We were using per-type overrides which caused some asymmetries, where conversions could end up dropping fields as we went along. Essentially each conversion would need to override a superset of the previous one, which then caused issues when attempting to close the loop. By overriding on the model level we ensure that all representations are equivalent and convertible into one another, at the expense of overriding a bit more aggressively, which should be fine anyway.
42 lines
1.6 KiB
Markdown
42 lines
1.6 KiB
Markdown
# MsgGen - Generating language bindings and docs from schemas and wire descriptions
|
|
|
|
MsgGen is a collection of tools that are used to parse schemas and
|
|
(eventually) protocol wire CSVs into an intermediate representation in
|
|
memory, and then generate language specific bindings and
|
|
documentation from it.
|
|
|
|
|
|
The dependency graph looks like this:
|
|
|
|
|
|
```dot
|
|
digraph {
|
|
"JSON-RPC Schemas" -> "msggen model";
|
|
"msggen model" -> "grpc proto file";
|
|
"msggen model" -> "Rust From<JsonRpc> Converters";
|
|
"grpc proto file" -> "Rust grpc bindings"
|
|
"Rust grpc bindings" -> "cln-grpc";
|
|
"Rust From<JsonRpc> Converters" -> "cln-grpc";
|
|
"msggen model" -> "Rust JSON-RPC structs";
|
|
"Rust JSON-RPC structs" -> "cln-rpc";
|
|
}
|
|
```
|
|
|
|
`msggen` will load the schemas in `doc/schemas` into memory, and then
|
|
use `Patches` to enrich the model before using it to generate the
|
|
bindings for the various languages as well as the converters from one
|
|
format to another. These patches can be found in `msggen/patch.py` and
|
|
perform a variety of operations:
|
|
|
|
- Annotate the model with additional data from external sources, such
|
|
as the `.msggen.json` file at the repository root to track details
|
|
that can be derived but should remain constant (grpc field
|
|
numbering and versioning information)
|
|
- Aggregate common types with type overrides and omit fields that we
|
|
can't map currently.
|
|
- Infer optionality based on the versions a field was added or
|
|
deprecated, and the currently supported range of versions.
|
|
|
|
If there is a field that is currently missing in the model, that is in
|
|
the schemas it is most likely because it has been marked as omitted in
|
|
the patch.
|