SendToRoute previously didn't accept invoice routing hints.
This was a limitation when paying a wallet invoice.
Invoice hints are now correctly taken into account.
Writing the PaymentRelayed event to the DB is not atomic.
Each part is written independently of the others (one row per part).
This is fine as nothing relies on this event being written atomically.
However tests were expecting that and we would observe rare
test failures on travis.
When coming back online, re-send private channels' `channel_update`.
This makes sure it gets rebroadcast regularly in case it was missed.
Since it's a private channel, it won't spam the network.
We use the socks5 proxy that is defined in the configuration and is typically used to connect to LN nodes running as TOR hidden services.
This should allow users to connect to Electrum servers that are running behind TOR.
* Refactor payment errors
When sending payments, it makes it easier for a wallet to display
the correct localized error message to users.
* Faster Trampoline payments fulfill
We were previously waiting for the whole downstream payment
to be settled (all individual HTLCs).
We can do better and fulfill upstream as soon as we get the preimage
(which only needs one downstream fulfill).
We currently rely on `require`, which is convenient, but doesn't allow
fine-grained exception control.
Also, in case of errors, logging is done at the supervisor level, where
we lose the remote `node_id`.
Instead, we type some crypto-related errors and log them in the
`TransportHandler`, which already has the correct MDC.
The DB ordering is not deterministic.
For multi-part payments, the first timestamp is taken.
This can vary depending on which record is listed first.
Using the same timestamp avoids a failed assertion.
Instead of emitting this event when we send a signature, we emit it when
our `availableBalanceForSend` actually changes. This happens:
- when we send a new `update_*`;
- when we receive a `commit_sig`, which may acknowledge one or several
`update_*` sent by our peer.
We choose to only emit this event in `NORMAL` state, because its goal is
to give information about what payments we can make, which can only
happen in that state.
NB: other events `ChannelSignatureSent` and `ChannelSignatureReceived` give
a different type of information, and are sent in all states where
signatures are exchanges, not only in `NORMAL`.
The field `localBalance` has been removed because it was ambiguous, and so is
the balance tracking in the database, which wasn't very useful.
Co-Authored-By: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
* Electrum: allow watcher to watch for mempool transactions
Watcher now handles WatchConfirmed watches where min depth
is set to 0: the watch event will be sent when the tx enters the
mempool of the bitcoin node our Electrum server is connected to.
For 0-conf channel, use scids with a height of 0 and a tx index
generated from the first 16 bytes of the funding txid. This gives us
unique ids that can still be identified as 0-conf channel.
Let a sender manually split a payment and specify a trampoline route.
Fix two flaky tests where the order of payment parts could be
different, resulting in a failed equality test.
If we're relaying multiple HTLCs for the same payment_hash,
we need to list all of those.
The previous code only handled that when Trampoline was used.
Comparing with the router ActorRef simply didn't work.
The reason is probably because Peers receive the router's supervisor ref
which doesn't match what `self` is inside `Router`.
Checking that the origin was the router felt brittle anyway.
We're now correctly typing the gossip origin.
We don't implement the upfront_shutdown_script feature.
However we update our encoding to always specify it.
This allows extending OpenChannel/AcceptChannel with tlv streams.
There is one caveat: Phoenix shipped with a version that's incompatible.
So we use a workaround to identify unpatched Phoenix versions
and send them the old encoding.
With MPP and Trampoline (and particularly the combination of the two),
we need to keep track of multiple amounts, recipients and fees.
There's a trampoline fee and a fee to reach the first trampoline node.
The trampoline nodes must appear in the route, but not as payment recipients.
Adding new fields to payment events and DB structs lets us distinguish those.
We also relax the spec requirement about feature graph dependencies.
The requirement to include `var_onion_optin` in invoice feature bits
was added after the first Phoenix release.
Phoenix users will thus have non spec-compliant invoices in their
payment history.
We accept invoices that don't set this field; this is a harmless
spec violation (as long as we set it in new invoices).
There was a rounding issue with the availableForSend/Receive calculation.
Because CommitTx fee and Htlc fee were computed separately,
but each was individually rounded down to Satoshis, we could
end up with an off-by-one error.
This resulted in an incapacity to send/receive the maximum amount available.
We now allow computing fees in msat, which removes rounding issues.
c-lightning fails to decode empty arrays of scids or timestamps with an encoding type set to COMPRESSED_ZLIB.
The spec is not specific enough on whether this is valid or not, so we'll set the encoding type of empty arrays to UNCOMPRESSED.
When paying an invoice, we weren't properly checking our own features.
If the invoice supported MPP, we would use it all the time.
If MPP isn't enabled in our features, we now default to a legacy payment.
Add new errors that let senders know they need to raise the trampoline fee/ctlv.
When the error is downstream, select the best error to forward.
Implement retry with more fees for trampoline payments.
This process is currently quite manual: the sender decides upfront on
each attempt's fee/cltv.
If our initial random deconnnection delay is 0 (unlikely but possible) then all "exponential backoff" reconnection delays will be 0 too, so we set a minimum value of 200 milliseconds.
The term "non-segwit UTXOs" appears in the error message that results from having such outputs in your bitcoin core wallet, and now the readme contains that same term and the solution to make troubleshooting easier.
lnd expects ids ranges in reply_channel_range messages to strictly follow each other, without gaps.
For example, using block heights and not ids, [1,2,4,5] would be split into (first=1, num=2, [1,2]) :: (first=3, num=2, [4, 5])
This is arguably a limitation of lnd (c-lightning does not requires this and it's not needed to properly process replies) but is easy to implement.
This is needed to make sure we broadcast our own gossip.
Otherwise we will try to gossip at the beginning of the connection,
when the peer hasn't set any timestamp, so our gossip will be dropped.
See https://github.com/lightningnetwork/lightning-rfc/pull/684
Otherwise eclair-mobile can't pay using MPP.
This heuristic was only here to help Trampoline nodes with a lot of
channels relay using MPP, but we disabled that in #1271 anyway.
We will reactivate Trampoline-MPP once split is done inside the router.
* Add test to check that we split short channel ids correctly
reply_channel_range messages should not overlap i.e different replies should not contain
channel ids that have the same block height.
The test in this commit fails, because our 'split' function needs to be updated.
* Channel Queries: make sure that our replies match the request range (fixes#1269)
Even though it's not completely explicit in the specs, we should make sure that
the [firstBlock, numBlock] range that we cover in our replies is not computed
from the ids that we actually have but instead matches the [firstBlock, numBlock] range
that was requested.
* Make sure that serialised replies stay below the 65Kb limit
We prune short channel id chunks to make sure that serialised replies stay below the 65 Kb limit.
The pruning algo is very simple: for each chunk we randomly keep the first or last 3200 ids
Selection is random so peers that re-connect will eventually receive all channel info.
The limit of 3200 was chosen for the worst case where replies are not compressed and include timestamps and checksum.
It is a fairly conservative boundary, the highest number of public channels in a single block so far is <300, and
there 3200 is roughly the currently observed number of transactions in a "full" block.
* Set default ids chunk size to 1500
Have smaller chunks (smaller than 3200 / 2) reduces the probability of merging 2 chunks and having to prune the result because the encoded reply would be over 65K.
* Smarter algo for enforcing max chunk size policy
Instead of keeping either the first or last items, we use a random offset. This way peers will eventually receive info about all channels even if chunks are much larger than the max chunk size and are pruned.
There is currently a backwards-compatibility issue with eclair-mobile.
Eclair-mobile mistakes feature bit 15 (payment_secret) for the
gossip_queries_ex prototype (which is incompatible with the spec-ed version).
To temporarily avoid this issue (until eclair-mobile is patched and all users have updated),
we never advertize those ambiguous bits in Init.
They're only really needed in the invoice so it's ok.
Implement https://github.com/lightningnetwork/lightning-rfc/pull/666
Keep the global/local split in Commitments to avoid backwards incompatibility in the codec.
Remove allowMultiPart API field: we instead rely on the MPP feature being set in nodeParams.
That means MPP-enabled nodes need to update their reference.conf.
Rework features:
* Add types to allow cleaner dependency validation.
* Most of the time we don't care whether a feature is activated as optional or mandatory, which caused duplicate code. This is now handled more cleanly.
* It also paves the way to annotate features with the places they should be advertised (Init vs NodeAnn vs ChannelAnn vs invoice).
This is safer for now since the splitting algorithm isn't working
well on nodes with a large number of channels and we don't
expect too many payments from Phoenix to non-Phoenix to
actually need MPP in the short term.
Mockito sometimes throws an unnecessary stubbing exception, it's unclear whether the test is faulty or mockito has issues with our parallel setup.
Rewrite switchboard tests without mockito makes them more flexible.
In case they randomly fail we should get more useful data to help troubleshooting.
