Note that this doesn't mean that we will buffer 1M objects in memory:
those are just pointers to (mostly) network announcements that already
exist in our routing table.
Routing table has recently gone over 100K elements (nodes,
announcements, updates) and this causes the connection to be closed when
peer requests a full initial sync.
Until now, if the peer is unresponsive (typically doesn't respond to
`open_channel` or `funding_created`), we waited indefinitely, or until the
connection closed.
It translated to an API timeout for users, and uncertainty about the
state of the channel.
This PR:
- adds an optional `--openTimeoutSeconds` timeout to the `open` endpoint, that will
actively cancel the channel opening if it takes too long before reaching
state `WAIT_FOR_FUNDING_CONFIRMED`.
- makes the `ask` timeout configurable per request with a new `--timeoutSeconds`
- makes the akka http timeout slightly greater than the `ask` timeout
Ask timeout is set to 30s by default.
Locks held on utxos that are used in unpublished funding transactions should not be persisted.
If the app is stopped before the funding transaction has been published the channel is forgotten
and so should be locks on its funding tx utxos.
There is no unique identifier for payments in LN protocol. Critically,
we can't use `payment_hash` as a unique id because there is no way to
ensure unicity at the protocol level.
Also, the general case for a "payment" is to be associated to multiple
`update_add_htlc`s, because of automated retries. We also routinely
retry payments, which means that the same `payment_hash` will be
conceptually linked to a list of lists of `update_add_htlc`s.
In order to address this, we introduce a payment id, which uniquely
identifies a payment, as in a set of sequential `update_add_htlc`
managed by a single `PaymentLifecycle` that ends with a `PaymentSent` or
`PaymentFailed` outcome.
We can then query the api using either `payment_id` or `payment_hash`.
The former will return a single payment status, the latter will return a
set of payment statuses, each identified by their `payment_id`.
* Add a payment identifier
* Remove InvalidPaymentHash channel exception
* Remove unused 'close' from paymentsDb
* Introduce sent_payments in PaymentDB, bump db version
* Return the UUID of the ongoing payment in /send API
* Add api to query payments by ID
* Add 'fallbackAddress' in /receive API
* Expose /paymentinfo by paymentHash
* Add id column to audit.sent table, add test for db migration
* Add invoices to payment DB
* Add license header to ExtraDirective.scala
* Respond with HTTP 404 if the corresponding invoice/paymentHash was not found.
* Left-pad numeric bolt11 tagged fields to have a number of bits multiple of five (bech32 encoding).
* Add invoices API
* Remove CheckPayment message
* GUI: consume UUID reply from payment initiator
* API: reply with JSON encoded response if the queried element wasn't found
* Return a payment request object in /receive
* Remove limit of pending payment requests!
* Avoid printing "null" fields when serializing an invoice to json
* Add index on paymentDb.sent_payments.payment_hash
* Order results in descending order in listPaymentRequest
Our `open` API calls expects an optional fee rate in satoshi/byte, which is the most widely
used unit, but failed to convert to satoshi/kiloweight which is the standard in LN.
We also check that the converted fee rate cannot go below 253 satoshi/kiloweight.
There was an obscure Docker error when trying to start an Electrum
server in tests. [1]
It appears that there is a conflict between Docker and Hyper-V on some
range of ports.
A workaround is to just change the port we were using.
[1] https://github.com/docker/for-win/issues/3171
* Fix eclair cli argument passing
* Modify eclair-cli to work with equals in arguments
* Eclair-cli: show usage when wrong params are received
* Remove deprecated call from eclair-cli help message [ci skip]
* send events when htlc settle on-chain
* send events when a payment is received/relayed/sent
* send events when a payment is failed
* Add test for websocket
* Use nicer custom type hints when serializing to json (websocket)
* Fix bech32 prefix for regtest
* Separate cases for bech32 testnet and regtest for BOLT11 fallback address
* Electrum: use 3.3.4 as client name
* Electrum Pool: more specific message on disconnect
Specify wether we lost connection to our master server or to a backup server.
* Electrum: Update mainnet servers list
* Electrum: make pool address selection more readable
We connect to a random server we're not already connected to.
* Electrum Tests: increase "wait for ready" test timeout
If was a bit short and sometimes failed on travis.
* Electrum: better parsing of invalid responses
On testnet some Electrum servers are not compliant with the protocole version they advertise
and will return responses formatted with 1.0 rules.
Port the existing API functionalities over a new structure of HTTP endpoints, with the biggest difference being the usage of **named parameters** for the requests (responses are unchanged). RPC methods have become endpoints and the parameters for each are now passed via form-params (clients must use the header "Content-Type" : "multipart/form-data"), this allows for a clearer interpretation of the parameters and results in more elegant parsing code on the server side. It is possible to still use the old API version via a configuration key.
Old API can be used by setting `eclair.api.use-old-api=true`.
* Treat channels with fees=0 as if they had feeBase=1msat while we compute a route
* Add test to ensure we build the onion attaching no fees if they were not required by the channel_update
* Initialize the database outside the node param constructor
* Do not create folders during StartupSpec
* Simplify syntax for instantiating test Databases
* Rework parameter passing to database initialization
* Force UTF-8 file encoding on all platform.
* Use bitcoin-lib 0.11, which embeds libsecp256k1
* Unit tests: generate dummy sig from 32 random bytes
We now use a version of bitcoin-lib which embeds JNI bindings for libsecp256k1,
and it will only sign data that is 32 bytes long (in Bitcoin and LN you always
sign data hashes, not the actual data).
* Use maven 3.6.0 and a different mirror
* RoutingSyncSpec: don't create databases at init time
We called nodeParams which created a new in-memory sqlite database everytime we created "fake" routing info
* Bitcoin tests: generate 150 blocks instead of 500
We don't need to generate 432 blocks to activate segwit but we still need to have
spendable coins and coinbase maturity is 100 blocks even on regtest.
* Electrum client: test against mainnet Electrum servers
Previous test against testnet servers was flaky because testnet Electrum
servers are unrelable. Here we test against our own server on mainnet (and
2 servers from our list for the pool test).
Bitcoin Core 0.18 is about to enter RC cycle and should be release soon (initial target was April). It is not compatible with 0.16 (some of the RPC calls that we use have been removed. They're still available in 0.17 but tagged as deprecated).
With this PR, eclair will be compatible with 0.17 and the upcoming 0.18, but not with 0.16 any more so it will be a breaking change for some of our users. Supporting the last 2 versions is the right option and we should be ready before 0.18 is actually released (its initial target was April).
* don't spam with channel_updates at startup
Previous logic was very simple but naive:
- every time a channel_update changed we would send it out
- we would always make a new channel_update with the disabled flag set
at startup.
In case our node was simply restarted, this resulted in us re-sending a
channel_update with the disabled flag set, then a second one with the
disabled flag unset a few seconds later, for each public channel.
On top of that, this opened way to a bug: if reconnection is very fast,
then the two successive channel_update will have the same timestamp,
causing the router to not send the second one, which means that the
channel would be considered disabled by the network, and excluded from
payments.
The new logic is as follows:
- when we do NORMAL->NORMAL or NORMAL->OFFLINE or OFFLINE->NORMAL, we
send out the new channel_update if it has changed
- in all other case (e.g. WAIT_FOR_INIT_INTERNAL->OFFLINE) we do nothing
As a side effect, if we were connected to a peer, then we shut down
eclair, then the peer goes down, then we restart eclair: we will make a
new channel_update with the disabled flag set but we won't broadcast it.
If someone tries to make a payment to that node, we will return the
new channel_update with disabled flag set (and maybe the payer will then
broadcast that channel_update). So even in that corner case we are good.
* quick reconnection: bump channel_update timestamp
In case of a disconnection-reconnection, we first generate a
channel_update with disabled bit set, then after we reconnect we
generate a second channel_update with disabled bit not set.
If this happens very quickly, then both channel_updates will have the
same timestamp, and the second one will get ignored by the network.
A simple fix is to bump the second timestamp in this case.
* set channel_update refresh timer at reconnection
We only care about this timer when connected anyway. We also cancel it
when disconnecting.
This has several advantages:
- having a static task resulted in unnecessary refresh if the channel
got disconnected/reconnected in between 2 weeks
- better repartition of the channel_update refresh over time because at
startup all channels were generated at the same time causing all refresh
tasks to be synchronized
- less overhead for the scheduler (because we cancel refresh task for
offline channels (minor, but still)
