This was not properly enforced and would be a spec violation on the
peer's end. Also re-use a pong buffer to save on heap allocations if
there are a lot of peers. The pong buffer is only read from, so this
is concurrent safe.
To simplify the message signing API even further, we refactor the
lnwallet.MessageSigner interface to use a key locator instead of the
public key to identify which key should be signed with.
If these bits are present, then both sides can examine the new
CommitmentType TLV field that's present and use this in place of the
existing implicit commiment type negotiation. With this change, it's now
possible to actually deprecate old unsupported commitment types
properly.
In this commit, we add a new ChannelType field as a new TLV record to
the OpenChannel message. During this change, we make a few tweaks to the
generic TLV encode/decode methods for the ExtraOpaqueData struct to have
it work on the level of tlv.RecordProducer instead of tlv.Record, as
this reduces line noise a bit.
We also partially undo existing logic that would attempt to "prepend"
any new TLV records to the end of the ExtraOpaqueData if one was already
present within the struct. This is based on the assumption that if we've
read a message from disk to order to re-send/transmit it, then the
ExtraOpaqueData is fully populated so we'll write that as is. Otherwise,
a message is being encoded for the first time, and we expect all fields
that are known TLV fields to be specified within the struct itself.
This change required the unit tests to be modified slightly, as we'll
always encode a fresh set of TLV records if none was already specified
within the struct.
In this commit, we add a new TLV record that's intended to be used as an
explicit channel commitment type for a new form of funding negotiation,
and later on a dynamic commitment upgrade protocol. As defined, we have
3 channel types: base (the OG), tweakless, and anchors w/ zero fee
HTLCs. We omit the original variant of anchors as it was never truly
deployed from the PoV of lnd.
This commit refactors the remaining usage of WriteElements. By
replacing the interface types with concrete types for the params used in
the methods, most of the encoding of the messages now takes zero heap
allocations.
This commit changes the WriteElement and WriteElements methods to take a
write buffer instead of io.Writer. The corresponding Encode methods are
changed to use the write buffer.
This commit changes the method WriteMessage to use bytes.Buffer to save
heap allocations. A unit test is added to check the method is
implemented as expected.
Removes the MaxPayloadLength function from the Message interface
and checks that each message payload is not greater than MaxMsgBody.
Since all messages are now allowed to be 65535 bytes in size, the
MaxPayloadLength is no longer needed.
In this commit, we convert the delivery address in the open and accept
channel methods to be a TLV type. This works as an "empty" delivery
address is encoded using a two zero bytes (uint16 length zero), and a
tlv type of 0 is encoded in the same manner (byte for type, byte for
zero length). This change allows us to easily extend these messages in
the future, in a uniform manner.
When decoding the message we snip the bytes from the read TLV data.
Similarly, when encoding we concatenate the TLV record for the shutdown
script with the rest of the TLV data.
Messages:
- UpdateFulfillHTLC
- UpdateFee
- UpdateFailMalformedHTLC
- UpdateFailHTLC
- UpdateAddHTLC
- Shutdown
- RevokeAndAck
- ReplyShortChanIDsEnd
- ReplyChannelRange
- QueryShortChanIDs
- QueryChannelRange
- NodeAnnouncement
- Init
- GossipTimestampRange
- FundingSigned
- FundingLocked
- FundingCreated
- CommitSig
- ClosingSigned
- ChannelUpdate
- ChannelReestablish
- ChannelAnnouncement
- AnnounceSignatures
lnwire: update quickcheck tests, use constant for Error
multi: update unit tests to pass deep equal assertions with messages
In this commit, we update a series of unit tests in the code base to now
pass due to the new wire message encode/decode logic. In many instances,
we'll now manually set the extra bytes to an empty byte slice to avoid
comparisons that fail due to one message having an empty byte slice and
the other having a nil pointer.
In this commit, we create a new `ExtraOpaqueData` based on the field
with the same name that's present in all the announcement related
messages. In later commits, we'll embed this new type in each message,
so we'll have a generic way to add/parse TLV extensions from messages.
In order to prep for allowing TLV extensions for the `ReplyChannelRange`
and `QueryChannelRange` messages, we'll need to remove the struct
embedding as is. If we don't remove this, then we'll attempt to decode
TLV extensions from both the embedded and outer struct.
All relevant call sites have been updated to reflect this minor change.
In this commit, we add a new RequiresFeature method to the feature
vector struct. This method allows us to check if the set of features
we're examining *require* that the even portion of a bit pair be set.
This can be used to check if new behavior should be allowed (after we
flip new bits to be required) for existing contexts.
This change was largely motivated by an increase in high disk usage as a
result of channel update spam. With an in memory graph, this would've
gone mostly undetected except for the increased bandwidth usage, which
this doesn't aim to solve yet. To minimize the effects to disks, we
begin to rate limit channel updates in two ways. Keep alive updates,
those which only increase their timestamps to signal liveliness, are now
limited to one per lnd's rebroadcast interval (current default of 24H).
Non keep alive updates are now limited to one per block per direction.
This fixes a decoding error when the list of short channel ids within a
QueryShortChanIDs message started with a zero sid.
BOLT-0007 specifies that lists of short channel ids should be sorted in
ascending order. Previously, this was checked within lnwire by comparing
two consecutive sids in the list, starting at the empty (zero) sid.
This meant that a list that started with a zero sid couldn't be decoded
since the first element would _not_ be greater than the last one
(namely: also zero).
Given that one can only check for ordering starting at the second
element, we add a check to ensure the proper behavior.
A unit test is also added to ensure no future regressions on this
behavior.
Before this commit, both writing and reading an encoded empty set of
short channel IDs from the wire would fail. Prior to this commit, we
treated decoding an empty set as a caller error, and failed to write out
the zlib encoding of an empty set in a way that us and the other
implementations were able to read.
To fix this, rather than giving zlib an empty buffer to write out (which
results in an encoding with the zlib header data and the rest), we just
write a blank slice. When decoding, if we have an empty query body, then
we'll return a `nil` slice.
With the above changes, we'll now always write out an empty short
channel ID set as:
```
0001 (1 byte follows) || <encoding_type>
```
A new test has also been added to exercise this case for both known
encoding types.
The number and the name will be separate on the rpc level, so we remove
the feature bit from the string. Currently this method is unused apart
from maybe in some rare logging instances.
This commit removes an unnecessarely large 32 byte buffer in favor of
a small 2 byte buffer and cleans up type conversion between uint16
and uint32 values.
This commit adds the feature bit and additional fields
required in `open_channel` and `accept_channel` wire
messages for `option_upfront_shutdown_script`.
This commit introduces a feature.Manager, which derives feature vectors
for various contexts within the daemon. The sets can be described via a
staticly compiled format, which makes any runtime adjustments to the
feature sets when the manager is initialized.
In this commit we change path finding to no longer consider all channels
between a pair of nodes individually. We assume that nodes forward
non-strict and when we attempt a connection between two nodes, we don't
want to try multiple channels because their policies may not be identical.
Having distinct policies for channel to the same peer is against the
recommendation in the spec, but it happens in the wild. Especially since
we recently changed the default cltv delta value.
What this commit introduces is a unified policy. This can be looked upon
as the greatest common denominator of all policies and should maximize
the probability of getting the payment forwarded.
Extends the invalid payment details failure with the new accept height
field. This allows sender to distinguish between a genuine invalid
details situation and a delay caused by intermediate nodes.
Align naming better with the lightning spec. Not the full name of the
failure (FailIncorrectOrUnknownPaymentDetails) is used, because this
would cause too many long lines in the code.
Methods on failure message types used to be defined on value receivers.
This allowed assignment of a failure message to ForwardingError both as
a value and as a pointer. This is error-prone, especially when using a
type switch.
In this commit the failure message methods are changed so that they
target pointer receivers.
Two instances where a value was assigned instead of a reference are
fixed.
In this commit, we modify the decoding of the FailUnknownPaymentHash
message to ensure we're able to fully decode the legacy serialization of
the onion error. We do this by catching the `io.EOF` error as it's
returned when _no_ bytes are read. If this is the case, then only the
error type was serialized and not also the optional amount.
In this commit, we fix a bug in the way we defined our even/odd features
for a particular feature. The check for if a feature bit is part of a
pair assumes that the pair bit has the exact same name as the bit being
queried. The way we defined our feature map didn't take note of this
assumption, as a result, any attempts to require a new bit moving from
optional to required would fail since the bit would be found, but the
names differed.
In this commit, we deprecate the `IncorrectHtlcAmount` onion error.
We'll still decode this error to use when retrying paths, but we'll no
longer send this ourselves. The `UnknownPaymentHash` error has been
amended to also include the value of the payment as well. This allows us
to worry about one less error.
In this commit, we add a field to the ChannelUpdate
denoting the maximum HTLC we support sending over
this channel, a field which was recently added to the
spec.
This field serves multiple purposes. In the short
term, it enables nodes to signal the largest HTLC
they're willing to carry, allows light clients who
don't verify channel existence to have some guidance
when routing HTLCs, and finally may allow nodes to
preserve a portion of bandwidth at all times.
In the long term, this field can be used by
implementations of AMP to guide payment splitting,
as it becomes apparent to a node the largest possible
HTLC one can route over a particular channel.
This PR was made possible by the merge of #1825,
which enables older nodes to properly retain and
verify signatures on updates that include new fields
(like this new max HTLC field) that they haven't yet
been updated to recognize.
In addition, the new ChannelUpdate fields are added to
the lnwire fuzzing tests.
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
In this commit, we fix the problem where it's annoying to parse a
bitfield printed out in decimal by writing a String method for the
ChanUpdate[Chan|Msg]Flags bitfield.
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
In this commit:
* we partition lnwire.ChanUpdateFlag into two (ChanUpdateChanFlags and
ChanUpdateMsgFlags), from a uint16 to a pair of uint8's
* we rename the ChannelUpdate.Flags to ChannelFlags and add an
additional MessageFlags field, which will be used to indicate the
presence of the optional field HtlcMaximumMsat within the ChannelUpdate.
* we partition ChannelEdgePolicy.Flags into message and channel flags.
This change corresponds to the partitioning of the ChannelUpdate's Flags
field into MessageFlags and ChannelFlags.
Co-authored-by: Johan T. Halseth <johanth@gmail.com>
In this commit we fix a compatibility issue with other implementations.
Before this commit, when writing out an onion error that includes a
`ChannelUpdate` we would use the `MaxPayloadLength` to get the length to
encode. However, a recent update has modified that to be the max
`brontide` payload length as it's possible to pad out the message with
optional fields we're unaware of. As a result, we would always write out
a length of 65KB or so. This didn't effect our parser as we ignore the
length and decode the channel update directly as we don't need the
length to do that. However, other implementations depended on the length
rather than just reading the channel update, meaning that they weren't
able to decode our onion errors that had channel updates.
In this commit we fix that by introducing a new
`writeOnionErrorChanUpdate` which will write out the precise length
instead of using the max payload size.
Fixes#2450.
In this commit, we ensure that when we read node aliases from the wire,
we ensure that they're valid. Before this commit, we would read the raw
bytes without checking for validity which could result in us writing in
invalid node alias to disk. We've fixed this, and also updated the
quickcheck tests to generate valid strings.
In this commit, we export the ReadElements and WriteElements functions.
We do this as exporting these functions makes it possible for outside
packages to define serializations which use the BOLT 1.0 wire format.
In this commit we add a check to HtlcSatifiesPolicy to verify that the
time lock for the outgoing htlc that is requested in the onion packet
isn't too far in the future.
Without this check, anyone could force an unreasonably long time lock on
the forwarding node.
This commit adds the required feature name to our
set of local known features. This will allow other
peers connecting to us to set the required gossip
queries feature bit. This is required for the
subsequent commits, which instruct the server to
set the bit depending on user configured preferences.
In this commit, we add a new field to all the existing gossip messages:
ExtraOpqueData. We do this, as before this commit, if we came across a
ChannelUpdate message with a set of optional fields, then we wouldn't be
able to properly parse the signatures related to the message. If we
never corrected this behavior, then we would violate the forwards
compatible principle we use when parsing existing messages.
As these messages can now be padded out to the max message size, we've
increased the MaxPayloadLength value for all of these messages.
Fixes#1814.
In this commit, we add a compatibility mode for older version of
clightning to ensure that we're able to properly parse all their channel
updates. An older version of c-lightning would send out encapsulated
onion error message with an additional type byte. This would throw off
our parsing as we didn't expect the type byte, and so we always 2 bytes
off. In order to ensure that we're able to parse these messages and make
adjustments to our path finding, we'll first check to see if the type
byte is there, if so, then we'll snip off two bytes from the front and
continue with parsing. if the bytes aren't found, then we can proceed as
normal and parse the request.
In this commit, we alter the behavior of the regular
short channel id encoding, such that it returns a nil
slice if the decoded number of elements is 0. This is
done so that it matches the behavior of the zlib
decompression, allowing us to test both in using the
same corpus.
Modifies the behavior of the quick test for
MsgQueryShortChanIDs, such that the generated
slice of expected short chan ids is always nil
if no elements are returned. This mimics the
behavior of the zlib decompression, where
elements are appended to the slice, instead of
assigning to preallocated slice.
In this commit, we add a new package level mutex. Each time we decode a
new set of chan IDs w/ zlib, we also grab this mutex. The purpose here
is to ensure that we only EVER allocate the maxZlibBufSize globally
across all peers. Otherwise, it may be possible for us to allocate up to
64 MB for _each_ peer, exposing an easy OOM attack vector.
In this commit, we implement zlib encoding and decoding for the channel
range queries. Notably, we utilize an io.LimitedReader to ensure that we
can enforce a hard cap on the total number of bytes we'll ever allocate
in a decoding attempt.
In this commit, we fix a slight bug in the parsing of encoded short
channel ID's. Before this commit, we would always assume that the remote
peer was sending us the sorted+encoded variant of the short channel
ID's. In the case that they weren't (as there isn't yet a feature bit),
we would assert this check and fail early as atm we don't support any
sort of compression.
In this commit, we add recognition of the data loss protected feature
bit. We already implement the full feature set, but then never added the
bit to our set of known features.
In this commit, we fix a minor deviation in our implementation from the
specification. Before if we encountered an unknown error type, we would
disconnect the peer. Instead, we’ll now just continue along parsing the
remainder of the messages. This was flared up recently by some
c-lightning related incompatibilities that emerged on main net.
In this commit, we modify lnwire.MilliSatoshi to be an unsigned
integer. We do this as all values within the specification are meant to
be unsigned unless otherwise specified. Our usage of signed integers to
this date has caused some compatibility issues with the other
implementations, so this is the first step to reconciling these
compatibility issues.
In this commit, we replace all instances of *btcec.PublicKey within the
announcement messages with a simple [33]byte. We do this as usually we
don’t need to immediately validate an announcement, therefore we can
avoid the scalar multiplications during decoding.
In this commit, we add a new signature type. We’ll use this type to
avoid fully decoding a signature on the wire into a btcec.Signature.
This type is only really needed when we need to do signature
validation, as a result, always encoding it is a waste. Several helper
methods have been added to the new struct in order to ensure that we
can use it in the existing codebase without substantial issues.
This commit adds the `lnnet` package which contains an
implementation of the newly created LightningNet interface which
multiplexes the Dial and DNS-related functions to use net
by default and torsvc if a flag is specified. This modularization
makes for cleaner code.
This commit alters the NodeAlias String method to trim null-bytes from
the end of the alias. This is helpful for presentation in contexts such
as the GetInfo response.
In this commit, we add additional detail in the error return hen were’e
unable to properly decode an encapsulated error. This additional
logging was added with the goal of being able to track down a lingering
bug, where at times lnd cannot decode a TemporaryChannelFailure
message.
This changes the circuit map internals and API to reference circuits
by a primary key of (channel ID, HTLC ID) instead of paymnet
hash. This is because each circuit has a unique offered HTLC, but
there may be multiple circuits for a payment hash with different
source or destination channels.
In this commit, we add a new type to the lnwire package:
ChanUpdateFlag. This type represent the bitfield that’s used within the
ChannelUpdate message to give additional details as how the message
should be interpreted.
In this commit we correct an off by one error on the Shutdown and
ClosingSigned message types. Additionally, we fix an error related to
the numbering of the UpdateFee message. With this we, gain proper
compatibility with the other implementations when it comes to
commitment hygiene, and also the cooperative closure process.
Early in the lifetime of the project here were a few files we either
copied entirely, or used as the basis for code within lnd. Before this
PR, this was not recognized by retaining the original copyright. With
this commit, we remedy that by explicitly noting the copyright in the
relevant files.
Fixes#423.
In this commit, we add support within lnwire for the optional dataloss
fields in ChannelReestablish. With these fields, it’s possible to:
verify that the remote node really knows of the state of our prior
local commitment, and also that they’ve sent us the current commitment
point for their current state.
In the event of dataloss, it’s possible for the party which lost data
to claim their commitment output in the remote party’s commitment if
they broadcast their current commitment transaction.
In this commit, we begin implementing the latest spec change to reduce
the attack surface on online channels. In this commit, we introduce a
distinct HTLC base point which will be used to sign the second-level
HTLC transactions for each active HLTC on the commitment transaction of
the remote node. With this, we allow the commitment key to remain
offline, as it isn’t needed in routine channel updates, unless we need
to go to chain.
For Part 1 of Issue #275. Create isolated private struct in
networkHandler goroutine that will de-duplicate
announcements added to the batch. The struct contains maps
for each of channel announcements, channel updates, and
node announcements to keep track of unique announcements.
The struct has a Reset method to reset stored announcements, an
AddMsg(lnwire.Message) method to add a new message to the current
batch, and a Batch method to return the set of de-duplicated
announcements.
Also fix a few minor typos.
In this commit the reestablish message have been added, which serves as
channel state synchronization message. Before exchanging the messages
for particular channel peers have to send it to each other as the
first message in order to be sure that non of the updates have been
lost because of the previous disconnect.
In this commit we add a new type to the lnwire package: FundingFlag.
This type will serve as an enum to describe the possible flags that can
be used within the ChannelFlags field in the OpenChannel struct.
We also define the first assigned flag: FFAnnounceChannel, which
indicates if the initiator of the funding flow wishes to announce the
channel to the greater network.
This is a rewrite of feature vectors in lnwire. This has a few
benefits:
1) a simpler interface
2) separate structs for a plain set of feature bits and a feature
vector with associated feature names
their respective feature sets
3) loosened requirements that bits MUST be assigned in pairs
4) fix endianness of encoding/decoding
This commit fixes an incorrectly calculated size of a
*FeatureVector in the serializedSize() function. go-fuzz
found that when calling NewFeatureVectorFromReader, if
a flag is invalid, it is not added to f.flags. However,
it will skip the index that wasn't included. This
becomes a problem when serializedSize() calculates the
length of f.flags via len() which can lead to an index
out of range since certain flags may be missing.
This commit fixes an existing deviation in the way we encode+decode the
addresses within the NodeAnnouncement message with that of the
specification. Prior to this commit, we would encode the _number_ of
addresses, rather than the number of bytes it takes to encode all the
addresses.
In this commit, we fix this mistake by properly writing out the total
number of bytes, modifying our parsing to take account of this new
encoding.
This commit fixes an existing, unnoticed bug within the lnwire.
NewChanIDFromOutPoint function. Two lingering issues cause the function
to not do anything at all, meaning that the channel ID, would be the
exact same as the actual txid passed in.
The first issue was that the xorTxid function wasn’t actually XOR’ing
the last two bytes. This was due to the fact that the function wasn’t
taking a pointer to the target ChannelID, meaning that the mutation
wouldn’t be seen outside of the scope of the function. Second, we had
our slicing reversed, rather than buf[30:], we were using buf[:30],
meaning that we were weren’t properly filling the buffer with the lower
2-bytes of the passed index.
This commit modifies the main test loop within
TestChannelIDOutPointConversion stricter. With this commit, we now
ensure that the conversion function actually does _something_. This is
the first in a series of commits to fix a recently discovered bug
within the outpoint to channel ID conversion routines.
This is a very simple bug that go-fuzz found. If length of an address
within CloseSigned is greater than 34, a runtime error: slice bounds out
of range happens. An error should be returned instead.
In this commit we reverse the ordering of the MinAcceptDepth and
HltcMinimum fields within the AcceptChannel message. Previously, the
order of these two fields were reversed, meaning the remote peer
would’ve attempted to parse the MinAcceptDepth (as we intended), as
part of the HtlcMinimum, leading to a garbage value.
This commit fixes a diversion from the way the UpdateAddHTLC message is
defined within the specification. We had the HTLC expiry value in the
wrong place on the wire, which meant that we couldn’t parse the
messages as sent by the other LN implementations.
This corrects the fuzz test in TestLightningWireProtocol for
MsgCommitSig to avoid creating an empty slice since the decoded message
only creates a slice when there are greater than zero signatures and an
empty slice is not considered equal to a nil slice under reflection.
This can be tested by running the TestLightningWireProtocol 1000 times
in a loop with and without this change.
