* use 64B representation instead of DER for sigs
It is more compact, and as an added bonus it frees us from the
completely unrelated Bitcoin-specific `0x01` trailing sig hash.
Note that we already used the 64B representation for storage everywhere,
except in `ChannelCodecs.htlcTxAndSigsCodec`, which required a backward
compatibility codec. Added a nonreg test for this.
* Use updated secp256k1 JNI bindings
* Replace scalar with private key and point with public key
We now use the simplified/unified design proposed in bitcoin-lib where:
- there are no more specific types for scalar/point
- private and public keys are compressed unless explicitly requested
* Generate and use 32 bytes seeds (and not 33)
We used serialized random private keys which were represented a 33 bytes (with a 01 suffix).
Using random 32 bytes values is more consistent.
We must make sure that upgraded apps that already have a 33 bytes seed will still generate the same secrets, which is why LocalKeyManager still uses the 01 suffix when needed
* Fix eclair-cli to work with equal sign in arguments (#926)
* Fix eclair cli argument passing
* Modify eclair-cli to work with equals in arguments
* Eclair-cli: show usage when wrong params are received
* Remove deprecated call from eclair-cli help message [ci skip]
* Make Electrum tests pass on windows (#932)
There was an obscure Docker error when trying to start an Electrum
server in tests. [1]
It appears that there is a conflict between Docker and Hyper-V on some
range of ports.
A workaround is to just change the port we were using.
[1] https://github.com/docker/for-win/issues/3171
* API: fix fee rate conversion (#936)
Our `open` API calls expects an optional fee rate in satoshi/byte, which is the most widely
used unit, but failed to convert to satoshi/kiloweight which is the standard in LN.
We also check that the converted fee rate cannot go below 253 satoshi/kiloweight.
* Expose the websocket over HTTP GET to work properly with basic auth (#934)
* Expose the websocket over HTTP GET
* Add test for basic auth over websocket endpoint
* Set max payment attempts from configuration (#931)
With a default to `5`.
* Add a proper payments database (#885)
There is no unique identifier for payments in LN protocol. Critically,
we can't use `payment_hash` as a unique id because there is no way to
ensure unicity at the protocol level.
Also, the general case for a "payment" is to be associated to multiple
`update_add_htlc`s, because of automated retries. We also routinely
retry payments, which means that the same `payment_hash` will be
conceptually linked to a list of lists of `update_add_htlc`s.
In order to address this, we introduce a payment id, which uniquely
identifies a payment, as in a set of sequential `update_add_htlc`
managed by a single `PaymentLifecycle` that ends with a `PaymentSent` or
`PaymentFailed` outcome.
We can then query the api using either `payment_id` or `payment_hash`.
The former will return a single payment status, the latter will return a
set of payment statuses, each identified by their `payment_id`.
* Add a payment identifier
* Remove InvalidPaymentHash channel exception
* Remove unused 'close' from paymentsDb
* Introduce sent_payments in PaymentDB, bump db version
* Return the UUID of the ongoing payment in /send API
* Add api to query payments by ID
* Add 'fallbackAddress' in /receive API
* Expose /paymentinfo by paymentHash
* Add id column to audit.sent table, add test for db migration
* Add invoices to payment DB
* Add license header to ExtraDirective.scala
* Respond with HTTP 404 if the corresponding invoice/paymentHash was not found.
* Left-pad numeric bolt11 tagged fields to have a number of bits multiple of five (bech32 encoding).
* Add invoices API
* Remove CheckPayment message
* GUI: consume UUID reply from payment initiator
* API: reply with JSON encoded response if the queried element wasn't found
* Return a payment request object in /receive
* Remove limit of pending payment requests!
* Avoid printing "null" fields when serializing an invoice to json
* Add index on paymentDb.sent_payments.payment_hash
* Order results in descending order in listPaymentRequest
* Electrum: do not persist transaction locks (#953)
Locks held on utxos that are used in unpublished funding transactions should not be persisted.
If the app is stopped before the funding transaction has been published the channel is forgotten
and so should be locks on its funding tx utxos.
* Added a timeout for channel open request (#928)
Until now, if the peer is unresponsive (typically doesn't respond to
`open_channel` or `funding_created`), we waited indefinitely, or until the
connection closed.
It translated to an API timeout for users, and uncertainty about the
state of the channel.
This PR:
- adds an optional `--openTimeoutSeconds` timeout to the `open` endpoint, that will
actively cancel the channel opening if it takes too long before reaching
state `WAIT_FOR_FUNDING_CONFIRMED`.
- makes the `ask` timeout configurable per request with a new `--timeoutSeconds`
- makes the akka http timeout slightly greater than the `ask` timeout
Ask timeout is set to 30s by default.
* Set `MAX_BUFFERED` to 1,000,000 (#948)
Note that this doesn't mean that we will buffer 1M objects in memory:
those are just pointers to (mostly) network announcements that already
exist in our routing table.
Routing table has recently gone over 100K elements (nodes,
announcements, updates) and this causes the connection to be closed when
peer requests a full initial sync.
* Fix Dockerfile maven binary checksum (#956)
The Maven 3.6.0 SHA256 checksum was invalid and caused the docker build to fail.
* Add channel errors in audit db (#955)
We now keep track of all local/remote channel errors in the audit db.
* Added simple plugin support (#927)
Using org.clapper:classutil library and a very simple `Plugin` interface.
* Live channel database backup (#951)
* Backup running channel database when needed
Every time our channel database needs to be persisted, we create a backup which is always
safe to copy even when the system is busy.
* Upgrade sqlite-jdbc to 3.27.2.1
* BackupHandler: use a specific bounded mailbox
BackupHandler is now private, users have to call BackupHandler.props() which always
specifies our custom bounded maibox.
* BackupHandler: use a specific threadpool with a single thread
* Add backup notification script
Once a new backup has been created, call an optional user defined script.
* Update readme with bitcoin 0.17 instructions (#958)
This has somehow been missed by PR #826.
* Backup: explicitely specify move options (#960)
* Backup: explicitely specify move options
We now specify that we want to atomically overwrite the existing backup file with the new one (fixes
a potential issue on Windows).
We also publish a specific notification when the backup process has been completed.
* Print stack trace when crashing during boot sequence (#949)
* Print stack trace when crashing during boot sequence
* Use friendly message when db compatibility check fails
* ElectrumWallet should not send ready if syncing (#963)
This commit is already embedded in version `0.2-android-beta22`.
* Channel: Log additional data (#943)
* Channel: Log additional data
Log local channel parameters, and our peer's open or accept message.
This should be enough to recompute keys needed to recover funds in case of unilateral close.
* Electrum: make debug logs shorter (#964)
* Better handling of closed channels (#944)
* Remove closed channels when application starts
If the app is stopped just after a channel has transition from CLOSING to CLOSED, when the application starts again if will be restored as CLOSING. This commit checks channel data and remove closed channels instead of restoring them.
* Channels Database: tag closed channels but don't delete them
Instead we add a new `closed` column that we check when we restore channels.
* Document how we check and remove closed channels on startup
* Do not print the stacktrace on stderr when there is an error at boot (#966)
* Do not print the stacktrace on stdout when there is an error at boot
* Fix flaky test in PaymentLifecycleSpec (#967)
* Use local random pamentHash for each test in paymentlifecyclespec, intercept the route request before the router.
* Rename `eclair.bak` to `eclair.sqlite.bak` (#968)
This removes any ambiguity about what the content of the file is about.
* Fixed concurrency issue in `IndexedObservableList` (#961)
Update map with new indexes after element is removed
Fixes#915
* Various fix and improvements in time/timestamp handling (#971)
This PR standardizes the way we compute the current time as unix timestamp
- Scala's Platform is used and the conversion is done via scala's concurrent.duration facilities
- Java's Instant has been replaced due to broken compatibility with android
- AuditDB events use milliseconds (fixes#970)
- PaymentDB events use milliseconds
- Query filters for AuditDB and PaymentDB use seconds
* API: Support query by `channelId` or `shortChannelId` everywhere (#969)
Add support for querying a channel information by its `shortChannelId`.
* Smarter strategy for sending `channel_update`s (#950)
The goal is to prevent sending a lot of updates for flappy channels.
Instead of sending a disabled `channel_update` after each disconnection,
we now wait for a payment to try to route through the channel and only
then reply with a disabled `channel_update` and broadcast it on the
network.
The reason is that in case of a disconnection, if noone cares about that
channel then there is no reason to tell everyone about its current
(disconnected) state.
In addition to that, when switching from `SYNCING`->`NORMAL`, instead
of emitting a new `channel_update` with flag=enabled right away, we wait
a little bit and send it later. We also don't send a new `channel_update` if
it is identical to the previous one (except if the previous one is outdated).
This way, if a connection to a peer is unstable and we keep getting
disconnected/reconnected, we won't spam the network.
The extra delay allows us to remove the change made in #888, which was
a workaround in case we generated `channel_update` too quickly.
Also, increased refresh interval from 7 days to 10 days. There was no
need to be so conservative.
Note that on startup we still need to re-send `channel_update` for all
channels in order to properly initialize the `Router` and the `Relayer`.
Otherwise they won't know about those channels, and e.g. the
`Relayer` will return `UnknownNextPeer` errors.
But we don't need to create new `channel_update`s in most cases, so
this should have little or no impact to gossip because our peers will
already know the updates and will filter them out.
On the other hand, if some global parameters (like relaying fees) are
changed, it will cause the creation a new `channel_update` for all
channels.
* Fixed overflow issue with max duration (#975)
This is a regression caused by #971, because `Duration` has a max value of `Long.MaxValue` *nanoseconds*, not *seconds*.
* Use proper closing type in `ChannelClosed` event (#977)
There was actually a change introduced by #944 where we used
`ClosingType.toString` instead of manually defining types, causing a
regression in the audit database.
* Update bash autocompletion for eclair-cli (#983)
* Update bash autocompletition file to suggest all the endpoints
* Update list of commands in eclair-cli help message
* Replace `UnknownPaymentHash` and `IncorrectPaymentAmount` with `IncorrectOrUnknownPaymentDetails` (#984)
See https://github.com/lightningnetwork/lightning-rfc/pull/516 and https://github.com/lightningnetwork/lightning-rfc/pull/544
* Wireshark dissector support (#981)
* Transport: add support for encryption key logging.
This is the format the wireshark lightning-dissector uses to be able to decrypt lightning messages.
* Enrich test for internal eclair API implementation (fr.acinq.eclair.Eclair.scala) (#938)
* Add test to EclairImpl for `/send`, `/allupdates` and `/forceclose/`
* Set default chain to "mainnet" (#989)
Eclair is now configured to run on mainnet by default.
* Set tcp client timeout to 20s (#990)
So that it fails before the ask/api time out.
* Add bot support for code coverage (codecov) (#982)
* Add scoverage-maven-plugin dependency
* Update travis build to generate a scoverage report
* Add custom codecov configuration to have nice PR comments
* Add badge for test coverage in readme
* Accept `commit_sig` without changes (#988)
LND sometimes sends a new signature without any changes, which is a
(harmless) spec violation.
Note that the test was previously not failing because it wasn't specific
enough. The test now fails and has been ignored.
* Ignore subprojects eclair-node/eclair-node-gui in the codecov report (#991)
* Use bitcoind fee estimator first (#987)
* use bitcoind fee provider first
* set default `smooth-feerate-window`=6
* Configuration: increase fee rate mismatch threshold
We wil accept fee rates that up to 8x bigger or smaller than our local fee rate
* Updated license header (#992)
* Release v0.3 (#994)
* gui: include javafx native libraries for windows, mac, linux
* Release v0.3
* Set version to 0.3.1-SNAPSHOT
* Improved test coverage of `io` package (#996)
* improved test coverage of `NodeURI`
* improved test coverage of `Peer`
* Fix TextUI
* BackupHandler: use renameTo() on Android
Most Path methods are not available at our current API level
* Add scoverage-maven-plugin dependency
* Update travis build to generate a scoverage report
* Add custom codecov configuration to have nice PR comments
* Add badge for test coverage in readme
* Initialize the database outside the node param constructor
* Do not create folders during StartupSpec
* Simplify syntax for instantiating test Databases
* Rework parameter passing to database initialization
* Force UTF-8 file encoding on all platform.
* Use bitcoin-lib 0.11, which embeds libsecp256k1
* Unit tests: generate dummy sig from 32 random bytes
We now use a version of bitcoin-lib which embeds JNI bindings for libsecp256k1,
and it will only sign data that is 32 bytes long (in Bitcoin and LN you always
sign data hashes, not the actual data).
* Use maven 3.6.0 and a different mirror
* RoutingSyncSpec: don't create databases at init time
We called nodeParams which created a new in-memory sqlite database everytime we created "fake" routing info
* don't spam with channel_updates at startup
Previous logic was very simple but naive:
- every time a channel_update changed we would send it out
- we would always make a new channel_update with the disabled flag set
at startup.
In case our node was simply restarted, this resulted in us re-sending a
channel_update with the disabled flag set, then a second one with the
disabled flag unset a few seconds later, for each public channel.
On top of that, this opened way to a bug: if reconnection is very fast,
then the two successive channel_update will have the same timestamp,
causing the router to not send the second one, which means that the
channel would be considered disabled by the network, and excluded from
payments.
The new logic is as follows:
- when we do NORMAL->NORMAL or NORMAL->OFFLINE or OFFLINE->NORMAL, we
send out the new channel_update if it has changed
- in all other case (e.g. WAIT_FOR_INIT_INTERNAL->OFFLINE) we do nothing
As a side effect, if we were connected to a peer, then we shut down
eclair, then the peer goes down, then we restart eclair: we will make a
new channel_update with the disabled flag set but we won't broadcast it.
If someone tries to make a payment to that node, we will return the
new channel_update with disabled flag set (and maybe the payer will then
broadcast that channel_update). So even in that corner case we are good.
* quick reconnection: bump channel_update timestamp
In case of a disconnection-reconnection, we first generate a
channel_update with disabled bit set, then after we reconnect we
generate a second channel_update with disabled bit not set.
If this happens very quickly, then both channel_updates will have the
same timestamp, and the second one will get ignored by the network.
A simple fix is to bump the second timestamp in this case.
* set channel_update refresh timer at reconnection
We only care about this timer when connected anyway. We also cancel it
when disconnecting.
This has several advantages:
- having a static task resulted in unnecessary refresh if the channel
got disconnected/reconnected in between 2 weeks
- better repartition of the channel_update refresh over time because at
startup all channels were generated at the same time causing all refresh
tasks to be synchronized
- less overhead for the scheduler (because we cancel refresh task for
offline channels (minor, but still)
Use bitcoin-lib v0.10 which has finally been synced to maven central.
Fix transactions unit test (the check in the test was using the whole locktime and not
the last 24 bits).
See https://github.com/ACINQ/bitcoin-lib/pull/31.
We still have to use `Array[Byte]` for low-level cryptographic primitives, and `akka.util.ByteBuffer` for tcp connections. In order to reduce unnecessary copies, we used `ByteVector.view(...)` as much as possible.
Took the opportunity to do a project-wide optimize imports. We might as well do it now since pretty much all files have been touched already.
NB: temporarily use bitcoin-lib 0.10.1-SNAPSHOT because maven central is very slow and we can't access the recently release 0.10 for now.
* Upgrade to JDK11
Eclair can be built and used on Oracle JDK 1.8 or OpenJDK 11.
JavaFX is now embedded in eclair-node-gui and does not need to be installed separately.
* Install: update java download links
OpenJDK 11 is now our recommendation. Tell users to download java from https://jdk.java.net/11
* README: Rewrite installation instructions
* Correctly parse short channel id
* Add test for RPC APIs
* Put akka.http.version in parent project pom
Co-Authored-By: araspitzu <a.raspitzu@protonmail.com>
* Implement "GetHeaders" RPC call
* Add checkpoints and pow verification
* Don't resolve server address too soon
* Add testnet checkpoints
* Store headers in a sqlite wallet db
* Use 1.4 protocol
Request protocol version 1.4 (this is the default setting in Electrum wallet).
Retrieve and store all headers as binary blobs in bitcoin format.
* Insert headers in batch
* Optimize headers sync and persistence
We assume that there won't be a reorg of more that 2016 blocks (which
could be handled by publishing a new checkpoint) and persist our headers
except for the last 2016 we have received: when we restart, we will ask
our server for at least 2016 headers.
* Persists transactions
Transactions are persisted only when they've been verified (i.e. we've receive
a valid Merkle proof)
* Disable difficulty check on testnet and regtest
On testnet there can be difficulty adjustements even within a re-targeting window.
* Update checkpoints
* Use proper Ping message
`version` can not longer be sent as a ping as we did before.
* Don't ask for Merkle proofs for unconfirmed transactions
* Improve startup time
We now store a new checkpoint and headers up to that checkpoint as soon as our
best chain is 2016 + 500 blocks long
* Properly detect connection loss
* Update electrum mainnet servers list
Using the list from Electrum 3.3.2
* Don't open multiple connection to the same Electrum servers
We want to keep connection to 3 different servers, but when we have less than 3 different
addresses it's pointless to attempt to keep maintain 3 connections.
* Fixed regression in rebroadcast (#713)
Fixed regression caused by 2c1811d: we now don't force sending a
channel_update at the same time with channel_announcement.
This greatly simplifies the rebroadcast logic, and is what caused the
integration test to fail.
Added proper test on Peer, testing the actor, not only static methods.
* Routing sync fixes (#712)
* Router: reset sync state on reconnection
When we're reconnected to a peer we will start a new sync process and should reset our sync
state with that peer.
* Ignore 'origin htlc not found' in CLOSING (#708)
If we don't have the origin, it means that we already have forwarded the fulfill so that's not a big deal.
This can happen if they send a signature containing the fulfill, then fail the channel before we have time to sign it.
* Fix handling of born again channels (#717)
* Fix handling of born again channels
When we receive a recent update for a channel that we had marked as stale we
must send a query to the underlying transport, not the origin of the update (which
would send the query back to the router)
* Replace `update_fee` in commitments (#709)
This is a simple optimisation, we don't have to keep all `update_fee`, just the last one.
cf BOLT 2:
> An update_fee message is sent by the node which is paying the Bitcoin fee. Like any update, it's first committed to the receiver's commitment transaction and then (once acknowledged) committed to the sender's. Unlike an HTLC, update_fee is never closed but simply replaced.
* Tests: use bitcoind 0.16.3 (#715)
Bitcoind 0.16.0 is no longer available
* Make `publishTransaction` idempotent (#711)
Bitcoin core returns an error `missing inputs (code: -25)` if the tx that we want to publish has already been published and its output have been spent. When we receive this error, we try to get the tx, in order to know if it is in the blockchain, or if its inputs were spent by another tx.
Note: If the outputs of the tx were still unspent, bitcoin core would return "transaction already in block chain (code: -27)" and this is already handled.
* Improved eclair-cli (#718)
This fixes#695, and also adds the channel point in the default channel output.
```bash
$ ./eclair-cli channel 00fd4d56d94af93765561bb6cb081f519b9627d3f455eba3215a7846a1af0e46
{
"nodeId": "0232e20e7b68b9b673fb25f48322b151a93186bffe4550045040673797ceca43cf",
"shortChannelId": "845230006070001",
"channelId": "00fd4d56d94af93765561bb6cb081f519b9627d3f455eba3215a7846a1af0e46",
"state": "NORMAL",
"balanceSat": 9858759,
"capacitySat": 10000000,
"channelPoint": "470eafa146785a21a3eb55f4d327969b511f08cbb61b566537f94ad9564dfd00:1"
}
```
* Handle update relay fee in OFFLINE state (#719)
Previously it was only possible to update relay fee in NORMAL state,
which is not very convenient because most of the time there are always
some channels in OFFLINE state.
This works like the NORMAL case, except that the new `channel_update`
won't be broadcast immediately. It will be sent out next time the
channel goes back to NORMAL, in the same `channel_update` that sets the
`enable` flag to true.
Also added a default handler that properly rejects the
CMD_UPDATE_RELAY_FEE command in all other states.
* Fixed regression caused by 7a4f175 (#722)
When updating relay fee in state OFFLINE, the new channel_update must
have the disabled flag on.
This caused tests to be flaky, added necessary checks to always make
them fail in case that kind of regression happens again.
* Logging: use a rolling file appender (#721)
* Logging: use a rolling file appender
Use one file per day, keep 90 days of logs with a total maximum size
capped at 5 Gb
* Router: log routing broadcast in debug level only
* set version to 0.2-beta6
* set version back to 0.2-SNAPSHOT
* Simplify bitcoind version check (#731)
Bitcoind returns version as MMmmrr (major, minor, revision), use an int representation
and compare it to our minimum version target.
* Update scalatest and remove junit runner (#728)
* updated to scalatest 3.0.5
* use scalatest runner instead of junit
Output is far more readable, and makes console (incl. travis) reports
actually usable.
Turned off test logs as error reporting is enough to figure out what
happens.
The only downside is that we can't use junit's categories to group
tests, like we did for docker related tests. We could use nested suites,
but that seems to be overkill so I just removed the categories. Users
will only have the possibility to either skip/run all tests.
* update scala-maven-plugin to 3.4.2
NB: This requires maven 3.5.4, which means that we currently need to
manually install maven on travis.
Also updated Docker java version to 8u181 (8u171 for compiling).
* Add instructions for Bitcoin Core 0.17.0 [ci skip] (#732)
* Add instructions for Bitcoin Core 0.17.0 [ci skip]
Bitcoin Core 0.17.0 deprecates the `signrawtransaction` RPC call, which will be removed in version 0.18.0, you need to enable this call if you want your eclair node to use a 0.1.70 node.
* README: add an example of how to use the new bitcoin.conf sections [ci skip]
* Only persist trimmed htlcs (#724)
We persist htlc data in order to be able to claim htlc outputs in
case a revoked tx is published by our counterparty, so only htlcs
above remote's `dust_limit` matter.
Removed the TODO because we need data to be indexed by commit number so
it is ok to write the same htlc data for every commitment it is included
in.
* set version to 0.2-beta7
* set version to 0.2-SNAPSHOT
* Add `htlcMaximumMsat` field to `ChannelUpdate` message (#738)
* Add `htlcMaximumMsat` field to `ChannelUpdate` message
* added compatibility test with c-lightning
* Fix encoding of FinalIncorrectHtlcAmount error message (#740)
* set version to 0.2-beta8
* set version to 0.2-SNAPSHOT
* Always add 1 block to the `finalCltvExpiry` (#742)
This fixes#651.
* ignore IntegrationSpec (no server on android)
* back to SNAPSHOT
* use proper [gs]etNullableLong method for Sqlite
* updated to scalatest 3.0.5
* use scalatest runner instead of junit
Output is far more readable, and makes console (incl. travis) reports
actually usable.
Turned off test logs as error reporting is enough to figure out what
happens.
The only downside is that we can't use junit's categories to group
tests, like we did for docker related tests. We could use nested suites,
but that seems to be overkill so I just removed the categories. Users
will only have the possibility to either skip/run all tests.
* update scala-maven-plugin to 3.4.2
NB: This requires maven 3.5.4, which means that we currently need to
manually install maven on travis.
Also updated Docker java version to 8u181 (8u171 for compiling).
