For pure TLV messages such as Gossip 1.75 and Bolt12 messages, there are
defined TLV ranges for unsigned data vs signed data. The signed ranges are
the inclusive ranges of 0 to 159 and 1000000000 to 2999999999.
What we do here in this commit is: 1) update the number of main types
generated so that they include type 160 which is the first type in the
un-signed range and we will make use of this type for signature fields
in gossip messages. Next, we add two marker types so that we can define
the rest of the ranges.
This type is useful when one wants to encode an integer as an underlying BigSize record. It wraps any integer, then handles the transformation into and out of the BigSize encoding on disk.
In this commit, we add a new type alias for a blob type. This type can be used in areas where a byte slice is used to store a TLV value, which may be a fully opaque nested TLV.
This commit introduces a RecordProducer implementation for Record
that serves as the identity function. This makes it easier for us
to mix Primitive and Dynamic records in the same RecordProducer
collections.
In this commit, we add a new `Zero` method for the `RecordT` type. This
method allows a caller to create the zero record for a type without
needing to reference the actual TLV type.
With this we go from this:
```go
sig1 := tlv.ZeroRecordT[tlv.TlvType1, Sig]()
```
To this:
```
sig1 := c.CloserNoClosee.Zero()
```
We add a WhenSome that'll pass in the actual underlying value, and not
just the record.
We also add `SomeRecord` to make it easier to create TLV records w/
`Some` set.
This will be useful when decoding optional TLV records, as we can use
this to look up in the `typeMap` for a given field to see if we decoded
it or not.
With this change, callers can now examine a struct instance of the
`TlvType` instance to figure out programmatically which integer type is
maps to. This'll be useful when decoding optional TLV types into a wire
struct (knowing when to set it to Some vs None).
In this commit, we modify the RecordT type to allow callers to re-use
the Record definition of a different type, but use the new type param to
override the integer type used on the wire.
This will let use do things like encode a signature using the same
RecordProducer instance, but with a diff type in another context.
The upcoming use for this is allowing our `lnwire.Sig` type to be
encoded in the same message using distinct TLV integer types (new co-op
close protocol).
In this commit, we add a new type, `RecordT[T, V]` to reduce some of the
common boiler plate for TLV types. This type lets you take either a
primitive type, or an existing Record, and gain common methods used to
create tlv streams.
It also serves as extra type annotation as well, since wire structs can
use this to wrap any existing type and gain the relevant record methods.
This implementation ensures that the very definition of the field also
binds the TLV type value. It does this by using the generated code to
map a struct like TlvType1 to an actually Type like Type(1).
This shouldn't need to be run again, as this implementation restricts
things to just values 0-99, due to a hard upper limit with the way Go
unions work under the hood.
In this commit, we add some new code generation to the codebase. As
we'll see in a future commit, this'll allow us to create a new Record[T,
V] type, where T is actually a concrete _struct_ that implements a
special interface that deems it as a valid TLV type.
* tlv: fuzz tests for primitives
* tlv: fuzz tests for BigSize
We use a new harness to compare decoded values instead of encoded
values, since there may be some unparsed bytes in the original data.
* tlv: fuzz tests for truncated integers
These fuzz tests are identical to non-truncated integers, except that we
allow the fuzzer to choose decode lengths shorter than the length of
normal integers.
* tlv: fuzz tests for streams
* fixup! tlv: fuzz tests for truncated integers
loop over decode length
* fixup! tlv: fuzz tests for streams
better documentation
Bump all build go versions to v1.21.0
Bump the minimum build package version to v1.19.0
Debian "buster" is not longer supported. Security updates have been
discontinued since June 30th 2022. We will build using the latest
version, "bookworm".
This commit adds a pair of encoder/decoder to take the advantage of the
BigSize format when encoding an uint64 and possibly an uint32.
Often the time an uint64 value is not big enough to fill all the 8
bytes, thus using BigSize can save extra bytes when save it to db. And
for uint32, if we know most of the values do not exceed 65536, we can
also save at least 1 byte using BigSize format.
This commit introduces `MakeBigSizeRecord` that can be used optionally
where db space is a concern.
This commit was previously split into the following parts to ease
review:
- 2d746f68: replace imports
- 4008f0fd: use ecdsa.Signature
- 849e33d1: remove btcec.S256()
- b8f6ebbd: use v2 library correctly
- fa80bca9: bump go modules
The benchmark tests import both the tlv and watchtower packages. To make
it possible to extract the tlv package into its own submodule, this test
is better located in the watchtower package.
Tlv is used more widely in lnd than just for the onion payload. This
commit isolated the protocol-specific odd/even logic, so that tlv can be
used freely elsewhere. An example of this use is db serialization.
These encoders can be composed to create composite types without
incurring additional allocations that would be required to pass the
truncated types through the generic interface.
In this commit, we create a new chainfee package, that houses all fee
related functionality used within the codebase. The creation of this new
package furthers our long-term goal of extracting functionality from the
bloated `lnwallet` package into new distinct packages. Additionally,
this new packages resolves a class of import cycle that could arise if a
new package that was imported by something in `lnwallet` wanted to use
the existing fee related functions in the prior `lnwallet` package.
This commit adds an additional return value to
Stream.DecodeWithParsedTypes, which returns the set of types that were
encountered during decoding. The set will contain all known types that
were decoded, as well as unknown odd types that were ignored.
The rationale for the return value (rather than an internal member) is
so that the stream remains stateless.
This return value can be used by callers during decoding to make
assertions as to whether specific types were included in the stream.
This is need, for example, when parsing onion payloads where certain
fields must be included/omitted depending on the hop type.
The original Decode method would incur the additional performance hit of
needing to track the parsed types, so we can selectively enable this
functionality when a decoder requires it by using a helper which
conditionally tracks the parsed types.
In this commit, we extend the Hop struct to carry an arbitrary set of
TLV values, and add a new field that allows us to distinguish between
the modern and legacy TLV payload.
We add a new `PackPayload` method that will be used to encode the
combined required routing TLV fields along any set of TLV fields that
were specified as part of path finding.
Finally, the `ToSphinxPath` has been extended to be able to recognize if
a hop needs the modern, or legacy payload.
