It is very difficult for the fuzzer to create a valid checksum for each
serialized invoice, and we were therefore unable to fuzz deeper than
invoice decoding. We can help the fuzzer generate valid serialized
invoices by calculating and appending the checksum ourselves.
We also switch to using mainnet invoices to make it easier to find valid
invoices for seeding the fuzzer. We prepend the required "lnbc" prefix
ourselves to further help the fuzzer generate valid invoices.
The message signer from invoice_test.go is identical to the one created
in the fuzz test. We're already using the private key from
invoice_test.go, so we may as well use the complete message signer for
simplicity.
The fuzz tests call inv.MinFinalCLTVExpiry() and inv.Expiry() supposedly
to ensure the invoice is well-formed. However, those methods can never
panic or return errors and therefore provide no benefit for this
purpose.
In this commit, the bugs demonstrated in prior commits are fixed. In the
case where an session has persisted a CommittedUpdate and the tower is
being removed, the session will now replay that update on to the main
task pipeline so that it can be backed up using a different session.
Add a new DeleteCommittedUpdate method to the wtdb In preparation for an
upcoming commit that will replay committed updates from one session to
another.
This commit demonstrates that if a session has persisted committed
updates and the client is restarted _after_ these committed updates have
been persisted, then removing the tower will fail.
In this commit, we demonstrate the situation where a client has
persisted CommittedUpdates but has not yet recieved Acks for them from
the tower. If this happens and the client attempts to remove the tower,
it will with the "tower has unacked updates" error.
This commit does a few things:
- First, it gives the sessionQueue access to the TowerClient task
pipeline so that it can replay backup tasks onto the pipeline on Stop.
- Given that the above is done, the ForceQuit functionality of the
sessionQueue and TowerClient can be removed.
- The bug demonstrated in a prior commit is now fixed due to the above
changes.
In preparation for an upcoming commit where multiple threads will have
access to the TowerClient sessionQueueSet, we turn it into a thread safe
struct.
This commit demonstrates a bug. It shows that if backup tasks have been
bound to a session with a tower (ie, the tasks are in the session's
pendingQueue) and then the tower is removed and a new one is added, then
the tasks from the pendingQueue are _not_ replayed to the session with
the new tower. Instead, they are silently lost. This will be fixed in an
upcoming commit.
This commit adds a new watchtower client test to demonstrate that a
client is able to successfully switch to a new tower and continue
backing up updates to that new tower.
Details of race:
- A global testInvoice is used in various invoice related tests.
- All tests are running with t.Parallel.
- TestMppPaymentWithOverpayment passes a reference to the testInvoice
to AddInvoice which writes the AddIndex.
- TestMultipleSetHeightExpiry reads the testInvoice to make a copy.
With the latest Golang Docker base image we are using the new gpg
version 2.4 is now being installed in the lnd Docker base image.
Apparently the expected value for the --keyring flag is just a file name
and not an absolute path. The path of the file is indicated either by
the $HOME environment variable or the --homedir flag. It looks like 2.4
now finally stopped supporting an absolute path in the --keyring flag
and we need to update our gpg command to make the script work again.
This should be backward compatible and still work on older versions of
gpg.
Schema for AMP invocies.
AMP invoices can be paid multiple times and each payment to an AMP invoice
is identified by a `set_id`.
The A in AMP stands for `Atomic`. All the htlcs belonging to the same
AMP payment (share the same set_id) will be resolved at the same time
with the same result: settled/canceled.
AMP invoices do not have an "invoice preimage". Instead, each htcl has
its own hash/preimage. When a new htlc is added the hash for that htlc
is attached to it. When all the htlcs of a set_id have been received we
are able to compute the preimage for each one of them.
Set of queries to deal with invoices. A couple of things to take into
account:
- Because the queries are not rewritten at runtime, we cannot have a
generic `INSERT` with different tuples.
- Because the queries are not rewritten at runtime, we cannot build
one query with only the filters that matter for that queries. The
two options are a combinatorial approach (a new query for every
permutation) or a generic query using the pattern
```
SELECT *
FROM table
WHERE (
-- Can be read as:
-- Match the filter 1 value if filter_1 != nil
column_1 >= sqlc.narg('filter_1') OR
sqlc.narg('filter_1') IS NULL
) AND (
column_2 >= sqlc.narg('filter_2') OR
sqlc.narg('filter_2') IS NULL
) ...
```
This is the schema for "ordinal" BOLT11 invoices.
The invoices table aims to keep an entry for each invoice, BOLT11 or not,
that will be supported.
Invoice related HTLCs will be stored in a separete table than forwarded
htlcs.
SQLite does not support `SEQUENCE`. We achieve atomic autoincrementals
using primary keys with autoincrement/serial. An invoice `AddIndex`
translates to `invoices(id)` while `SettleIndex` is `invoice_payments(id)`.
sqlc is a tool that generates fully type-safe idiomatic code from SQL.
The result is Go code can then used execute the queries in the database.
The noraml flow looks like:
- The developer write some sql that will update the schema in the
database: new tables, indices, etc
- The developer updates the set of queries that will use the new schema.
- `sqlc` generates type-safe interfaces to those queries.
- The developer can then write application code that calls the methods
generated by sqlc.
The tool configuration needs to live in the repo's root and its name is
`sqlc.yaml`.
LND will support out of the box sqlite and postgres. The sql code needs to
be (almost) the same for both engines, so we cannot use custom functions
like `ANY` in postgres.
The SQLC config file needs to define what is the target engine, we will
set postgres but the generated code can be executed by sqlite too.
In some specific cases, we will `match and replace` some sql lines to be
sure the table definitions are valid for the targeted engine.
