* Update list of commands in eclair-cli help (#1091)
* Add missing API endpoints to eclair-cli help
* Documentation update (#1092)
* Typed amounts (#1088)
* Route computation: fix fee check (#1101)
Fee check during route computation is:
- fee is below maximum value
- OR fee is below amout * maximum percentage
The second check was buggy and route computation would failed when fees we above maximum value but below maximum percentage of amount being paid.
* Publish transactions during transitions (#1089)
Follow up to #1082.
The goal is to be able to publish transactions only after we have
persisted the state. Otherwise we may run into corner cases like [1]
where a refund tx has been published, but we haven't kept track of it
and generate a different one (with different fees) the next time.
As a side effect, we can now remove the special case that we were
doing when publishing the funding tx, and remove the `store` function.
NB: the new `calling` transition method isn't restricted to publishing
transactions but that is the only use case for now.
[1] https://github.com/ACINQ/eclair-mobile/issues/206
* Typed cltv expiry (#1104)
Untyped cltv expiry was confusing: delta and absolute expiries really need to be handled differently.
Even variable names were sometimes misleading.
Now the compiler will help us catch errors early.
* Extended queries optional (#899)
This is the implementation of https://github.com/lightningnetwork/lightning-rfc/pull/557.
* Correctly handle multiple channel_range_replies
The scheme we use to keep tracks of channel queries with each peer would forget about
missing data when several channel_range_replies are sent back for a single channel_range_queries.
* RoutingSync: remove peer entry properly
* Remove peer entry on our sync map only when we've received
a `reply_short_channel_ids_end` message.
* Make routing sync test more explicit
* Do not send channel queries if we don't want to sync
* Router: clean our sync state when we (re)connect to a peer
We must clean up leftovers for the previous session and start the sync process again.
* Router: reset sync state on reconnection
When we're reconnected to a peer we will start a new sync process and should reset our sync
state with that peer.
* Extended Queries: use TLV format for optional data
Optional query extensions now use TLV instead of a custom format.
Flags are encoded as varint instead of bytes as originally proposed. With the current proposal they will all fit on a single byte, but will be
much easier to extends this way.
* TLV Stream: Implement a generic "get" method for TLV fields
If a have a TLV stream of type MyTLV which is a subtype of TLV, and MyTLV1 and MYTLV2 are both
subtypes of MyTLV then we can use stream.get[MyTLV1] to get the TLV record of type MYTLV1 (if any)
in our TLV stream.
* Channel range queries: send back node announcements if requested (#1108)
This PR adds support for sending back node announcements when replying to channel range queries:
- when explicitly requested (bit is set in the optional query flag)
- when query flags are not used and a channel announcement is sent (as per the BOLTs)
A new configuration option `request-node-announcements` has been added in the `router` section. If set to true, we
will request node announcements when we receive a channel id (through channel range queries) that we don't know of.
This is a setting that we will probably turn off on mobile devices.
* Rework router data structures (#902)
Instead of using two separate maps (for channels and channel_updates), we now use a single map, which groups channel+channel_updates. This is also true for data storage, resulting in the removal of the channel_updates table.
* Add more numeric utilities to MilliSatoshi (#1103)
Add comparisons and postfix operators.
Update most of the codebase to leverage those.
* Use unsigned comparison for 'maxHtlcValueInFlightMsat' (#1105)
* Add a sync whitelist (#954)
We will only sync with whilelisted peer. If the whitelist is empty then
we sync with everyone.
* Move http APIs to subproject eclair-node (#1102)
* Fix regression in `Commitments.availableForSend` (#1107)
We must consider `nextRemoteCommit` when applicable.
This is a regression caused in #784. The core bug only exists when we
have a pending unacked `commit_sig`, but since we only send the
`AvailableBalanceChanged` event when sending a signature (not when
receiving a revocation), actors relying on this event to know the
current available balance (e.g. the `Relayer`) will have a wrong
value in-between two outgoing sigs.
* Bolt4: remove final_expiry_too_soon error message (#1106)
It allowed probing attacks and the spec deprecated it in favor of IncorrectOrUnknownPaymentDetails.
Also add better support for unknown failure messages.
* Fix maven mirror (#1120)
* Use Long to back the UInt64 type (#1109)
* Define comparison operators between UInt64 and MilliSatoshi
* Implement Bolt 11 invoice feature bits (#1121)
lightningnetwork/lightning-rfc#656 introduced invoice feature bits as a pre-requisite for AMP and other advanced payment use-cases.
* Update docker build (#1123)
* Update docker base image to jdk11, update maven to 3.6.2 [ci skip]
* Reject expired invoices before payment flow starts (#1117)
* Made sync params configurable (#1124)
This allows us to choose smaller parameters for tests and reduce cpu
requirement during testing.
NB: The default value of 3500 for `reply_channel_range` was wrong. Theoretical max is ~2700.
* Activate support for variable-length onion (#1087)
This is now enabled by default.
We forward variable-length onions if we receive some.
We accept variable-length payments.
However for maximum compatibility with the network, we send payments using legacy payloads.
* Add Semaphore CI (#1125)
* Router computes network stats (#1116)
* Add comments and fix warnings in graph processing
* Add small feature to set the htlcMaximumMsat for routing hints (otherwise the graph processing algorithm used a minimum value which slightly reduced the benefits of those routing hints)
* Add the computation of network statistics to the router: this will be useful for multi-part payments to decide what thresholds should be used to split a payment
* Add monitoring with Kamon (disabled by default) (#1126)
For now:
- we only track some tasks (especially in the router, but not even
`node_announcement` and `channel_update`
- all db calls are monitored
- kamon is disabled by default
* Check funds in millisatoshi when sending/receiving an HTLC (#1128)
Instead of satoshi, which could introduce rounding errors.
Also, we check first the balance before the max-inflight amount, because
it makes more sense in terms of error management.
Co-Authored-By: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
* Don't hardcode the channel version (#1129)
Instead of hardcoding the channel version when we instantiate the
`Commitments` object, we rather define it when the channel is
instantiated. This is saner and prepares future usage.
* Removed Globals class (#1127)
This is a prerequisite to parallelization of tests.
* Make tests run in parallel (#1112)
There are two level of parallelization:
- between test suites (a suite = a test file)
- within a suite (depends on tests suites, some rely on sequential execution of tests, some don't)
* Add codecov integration to semaphore CI (#1134)
* Remove codecov integration from travis CI
* Drop support for Java 8 (#1135)
We already have Java 7 (for Android) and Java 11. Supporting Java 8
would require crossbuilding, which we are not doing (two recent PRs
broke the build on Java 8).
* Sphinx: accept invalid downstream errors (#1137)
When a downstream node sends us an onion error with an invalid length, we must forward the failure.
The recipient won't be able to extract the error but at least it knows the payment failed.
* Update string to match on bitcoind while it's indexing (#1138)
* Check for bitcoind's getrawtransaction availablilty during startup
* Peer: disable kamon
* Payment lifecycle refactoring (#1130)
* Unify payment events (no more duplication between payment types and events)
* Factorize DB and eventStream interactions: this paves the way for sub-payments that shouldn't be stored in the DB nor emit events.
* Add more fields to the payments DB:
* bolt 11 invoice for sent payment
* external id (for app developers)
* parent id (AMP)
* target node id
* fees
* route (if success)
* failures (if failed)
* Re-work the PaymentsDb interface
* Clarify use of seconds / milliseconds in DB interfaces -> milliseconds everywhere
* Run SQL migrations inside transactions
* Improve error handling when we couldn't find all the channels for a supplied route in /sendtoroute API (#1142)
* Improve error handling when we couldn't find all the channels for a supplied route in /sendtoroute
* Handle fees increases when channel is OFFLINE (#1080)
* Add 'close-on-offline-feerate-mismatch' configuration to avoid closing offline channel when the feerate mismatch if over the threshold.
* Derive channel keys from the channel funding pubkey (#1097)
We now generate a random funding key for each new channel, and use its public key to deterministically derive all channel keys and secrets. This will let us easily recover funds using DLP even if we've lost everything but our seed: we just need to connect to the node we had a channel with, ask them to publish their commit tx, and once we see it on the blockchain we can extract our funding pubkey, recompute channel keys and spend our output.
* Add a "funding pubkey path" option to the channel version field
This option is checked when we need to compute channel keys. For old channels it won't be set, and we always set it for new ones.
* ChannelVersion: make sure that all bits are set to 0 for legacy channels
* ChannelVersion: USE_PUBKEY_KEYPATH is set by default
* Check if remote funder can handle an updated commit fee when sending HTLC (#1084)
If the sender of an htlc isn't the funder, then both sides will have to afford the payment:
- the sender needs to be able to afford the htlc amount
- the funder needs to be able to afford the greater commit tx fee incurred by the additional htlc output.
Fixes#1081.
Co-Authored-By: Pierre-Marie Padiou <pm47@users.noreply.github.com>
* Fix and expand channel keypath (#1147)
* Fix funding pubkey to channel key path computation
Channel key path is generated from 8 bytes computed from our funding pubkey, but we extracted 4 uint32 values instead of 2 (last 2 were always 0). We now use 128 bits to derive channel key paths.
* Add a channel key path compatibility test
This test will fail if we change the way we compute channel key paths, which would break existing channels.
* Use the same chain hash reference in all channel updates
To save memory, once we check that a channel_update's chain hash matches what
we expect we just replace it with a reference to our own chain hash.
* Commitments: take HTLC fee into account (#1152)
Our balance computation was slightly incorrect. If you want to know how much you can send (or receive), you need to take into account the fact that you'll add a new HTLC which adds weight to the commit tx (and thus adds fees).
* Android: add a spray-based API to eclair-node
This is a copy of the spray-based API developped by @araspitzu (akka-http does not
work for akka 2.3 which we use on the android branch)
* HTTP API: add type hints for payment status (#1150)
Cleans up the JSON payment status (easier to interpret for callers).
* Use "mock" Kamon library
Kamon does not work on Android and does not make much sense, so we replace
it with a basic Mock implementation that does nothing.
* Electrum: improve coin selection (fixes#1146) (#1149)
Our previous coin selection would sometimes fail when there was one wallet utxo and and low
feerate, because our first pass used a fee estimate that was too high and could sometimes not be met.
* Extend funding key path to 256 bits (#1154)
Our random funding key path is now 8 * 32 bits plus a 1' (funder) or 0' (fundee).
Channel key paths are computed from the sha256 of the funding public key (we take all 256 bits).
* Use bitcoin 0.18.1 in the test (#1148)
* Upgrade new unit tests to bitcoin 0.18.1 API (#1157)
We had 2 open PRs, one that added new tests using the 0.API, one that switched to 0.18.1, when they were merged the new tests failed since they had not been upgraded....
* Update netty dependency to 4.1.32 (#1160)
Also:
* explicitely set endpoint identification algorithm in strict mode
* force TLS protocols 1.2/1.3 in strict mode
Co-Authored-By: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>
* Add execution time limit (#1161)
* Android: wipe channels table during db migration
We already wipe the updates table, and this make upgrading much simpler since we had different structures on
android vs mater.
* Activate extended channel range queries (#1165)
By default we now set the `gossip_queries_ex` feature bit.
We also change how we compare feature bits, and will use channel queries (or extended queries) only if the corresponding feature bit is set in both local and remote init messages.
* Use guava to compute CRC32C checksums (#1166)
CRC32C is not available in JDK 7 which we target on Android.
* Add scoverage-maven-plugin dependency
* Update travis build to generate a scoverage report
* Add custom codecov configuration to have nice PR comments
* Add badge for test coverage in readme
* Backup running channel database when needed
Every time our channel database needs to be persisted, we create a backup which is always
safe to copy even when the system is busy.
* Upgrade sqlite-jdbc to 3.27.2.1
* BackupHandler: use a specific bounded mailbox
BackupHandler is now private, users have to call BackupHandler.props() which always
specifies our custom bounded maibox.
* BackupHandler: use a specific threadpool with a single thread
* Add backup notification script
Once a new backup has been created, call an optional user defined script.
Port the existing API functionalities over a new structure of HTTP endpoints, with the biggest difference being the usage of **named parameters** for the requests (responses are unchanged). RPC methods have become endpoints and the parameters for each are now passed via form-params (clients must use the header "Content-Type" : "multipart/form-data"), this allows for a clearer interpretation of the parameters and results in more elegant parsing code on the server side. It is possible to still use the old API version via a configuration key.
Old API can be used by setting `eclair.api.use-old-api=true`.
* Upgrade to JDK11
Eclair can be built and used on Oracle JDK 1.8 or OpenJDK 11.
JavaFX is now embedded in eclair-node-gui and does not need to be installed separately.
* Install: update java download links
OpenJDK 11 is now our recommendation. Tell users to download java from https://jdk.java.net/11
* README: Rewrite installation instructions
* Improved amounts readability (fixes#542) and added the Bits unit
denomination in the documentation
* Improved channel panel UI layout
* Added a confirmation dialog when closing a channel, with a short summary
of the channel to be closed
The balance bar has been shrunk in size so that it should not be mistaken as
a separator element between channels. The channel's balance, capacity and
peer node id are now more visible. Also added the short channel id to the
channel's pane.
fixes#690
* Added node's aggregated balance in the status bar
fixes#775
* Support searching backward from the target
* Use the amount+fees with testing for min/max htlc value of edges
* Build the adjacency list graph with incoming edges instead of outgoing
* Make sure we don't find routes longer than the max allowed by the spec
* Remove default amount msat, enhance 'findroute' API
* Optimize tests for ignored edges in Dijkstra
* Enhance test for max route length, fix the length to 20 channels
* Add test for routing to a target that is not in the graph (assisted routes)
* Add instructions for Bitcoin Core 0.17.0 [ci skip]
Bitcoin Core 0.17.0 deprecates the `signrawtransaction` RPC call, which will be removed in version 0.18.0, you need to enable this call if you want your eclair node to use a 0.1.70 node.
* README: add an example of how to use the new bitcoin.conf sections [ci skip]
Added a new `AuditDb` which keeps tracks of:
- every single payment (received/sent/relayed)
- every single network fee paid to the miners (funding, closing, and all commit/htlc transactions)
Note that network fees are considered paid when the corresponding tx has reached `min_depth`, it makes sense and allows us to compute the fee in one single place in the `CLOSING` handler. There is an exception for the funding tx, for which we consider the fee paid when the tx has successfully been published to the network. It simplifies the implementation and the tradeoff seems acceptable.
Three new functions have been added to the json-rpc api:
- `audit`: returns all individual payments, with optional filtering on timestamp.
- `networkfees`: returns every single fee paid to the miners, by type (`funding`, `mutual`, `revoked-commit`, etc.) and by channel, with optional filtering on timestamp.
- `channelstats`: maybe the most useful method; it returns a number of information per channel, including the `relayFee` (earned) and the `networkFee` (paid).
The `channels` method now returns details information about channels. It makes it far easier to compute aggregate information about channels using the command line.
Also added a new `ChannelFailed` event that allows e.g. the mobile app to know why a channel got closed.
* add api call to update channel relay fees
* fixed bug in GUI, as channel can had different fees in each direction!
* fire transitions on `TickRefreshChannelUpdate` (fixes#621)
* make router publish `channel_update`s on startup
* (gui) Channel info fees are now options and case where channels have no known fees data is now properly handled.
* add support for mainnet final pubkeyscript
* electrum: add addresses of electrumx servers on mainnet
* electrum: use chain hash to compute addresses and HD key paths
* store db files in a subdirectory of datadir
* add a 'catchall' around deserialization
* use chain-specific key derivation paths for channel keys
* fixed intermittently failing test (we were comparing timestamps...)
* parameters:
- set default `router-broadcast-interval` to 60s
- set default `mindepth-blocks` to 3 blocks
- removed deprecated setting `router-validate-interval`
- reduce fees: block target 1->2
- reduce `MIN_CLTV_EXPIRY` from 9 to 7
Default value in BOLT11 was indeed 9 blocks, but the absolute minimum
value computed in BOLT2 is 7 blocks.
- remove unused `default-feerate-per-kb`
- set default `max-htlc-value-in-flight-msat`=10mBTC
- set default `max-to-local-delay-blocks` to 2000 blocks
* Update README with instructions for mainnet
* Upgrade to bitcoin-lib 0.9.15 (#516)
* use fees from provider, not default ones
* Add api call to check invoice/paymentRequest and return details in json
* Adding in API call to allow user to check route before making send call
* Added a serializer for route response
* Update README to include new checkinvoice and findroute API calls
As soon as we receive a valid closing signature, we will publish the
resulting closing tx instead of our commitment tx if we need to
immediately close the channel (before end of negotiation, e.g. in case
of errors, or in case the counterparty goes OFFLINE).
When the closing signature didn't correspond to one we sent ourselves, we
weren't properly recognizing the publishing tx and went into
`ERR_INFORMATION_LEAK` state.
CMD_CLOSE is now split it to commands:
* `CMD_CLOSE`: this command will succeed only if the current
channel state is in NORMAL, or if the channel hasn't yet been created.
* `CMD_FORCECLOSE`: the channel will publish its current local
commitment (or its best signed closing tx if it has one).
Using `CMD_FORCECLOSE` is more expensive and it incurs a delay before
funds are spendable, but this can be useful in some situations, for
example when the counterparty isn't responding anymore.
Added a new `forceclose` method to the API and a force close button in
the GUI.
The Bitcoin Core `addwitnessaddress` rpc command is deprecated in version 0.16.0. Thus, the example `bitcoin.conf` needs to be extended to with the option `deprecatedrpc=addwitnessaddress`.
Without this change, Eclair will not be able to open new channels and the following error message will be written to eclair.log:
```
WARN akka.actor.OneForOneStrategy akka://default/user/$h/switchboard/peer-0390b6fbf04dd277955ce454c87b270d3c910fa96ef3d78afa4bf17d3b733fb877 - addwitnessaddress is deprecated and will be fully removed in v0.17. To use addwitnessaddress in v0.16, restart bitcoind with -deprecatedrpc=addwitnessaddress. Projects should transition to using the address_type argument of getnewaddress, or option -addresstype=[bech32|p2sh-segwit] instead. (code: -32)
```
* moved CoinUtils to eclair-core to expose conversion methods
Parsing and converting string amount to BtcAmount objects, and
displaying BtcAmount as well formatted string is a common use case.
Methods that were developed for the gui module should be exposed from
eclair-core so that they can be reused elsewhere.
* increased timeouts in integration tests
* have bitcoin wallet and watcher use a separate client
* return more information on channel opening
* removed `storedChannels` from `Peer` constructor
1) so that they can be garbage-collected
2) that was a memory leak because closed channel would stay forever
* OpenChannel now accepts a feerate param for the funding Tx
User should be able to override the fee rate of the funding tx that he
sends when he opens a channel. This custom fee is entered as a fee rate
in satoshi per byte, and converted to a fee rate per kw.
The default value of this fee rate does not change, and still aims for
the funding tx to be included in a block within the next 6 blocks,
estimation which is made through the FeeProvider class.
* reworked payment events
and made sure we send events even when `PaymentLifecycle` disappeared, e.g.
when we restart right after having sent a payment.
* (gui) Added funding tx id to channel panel
fixes#335
* (gui) close channel button hidden when CLOSING or CLOSED
fixes#332
* (gui) User can define a preferred display unit
On startup FxApp reads the configuration from `eclair.conf` and sets
a unit to which all the amounts in the UI must be converted.
This unit is a CoinUnit object. Only `sat`, `mbtc` and `btc` are accepted.
`msat` is not accepted because it's an internal accounting unit which
should be invisible to the user. Default unit is `btc`
The gui does not expose any ui feature to update this unit at runtime.
* (gui) Notification message wraps and can now be copied
Notification message should be readable especially when an error occurs.
The message can be copied for easier issue reporting.
* Payment attempts is limited to 1
* (gui) consistent order in status bar contextual actions
* (gui) fixed unit parsing
* (gui) added menu shortcuts
fixes#347
* Revert "Payment attempts is limited to 1"
This reverts commit 1f4168aacb.
* (gui) BtcAmount are convertible to raw big decimal without loss
Fix issue where the decimal part of a BtcAmount could be lost.
Millisatoshi is now accepted as a user preferred unit.
* Set a minimum htlc amount of 1 mSat by default
By default, the node configuration should be permissive and accept low
value payments. It is up to the node admin to set up a restrictive
configuration if he needs it.
* (gui) port in node uri is optional
* (gui) added reactive errors in open channel window
* (gui) format max pr amount to user preferred unit
* (gui) Payment Request QR should be uppercased
see #375
* (gui) added a conf for the gui module
* Added eclair.gui.unit option in README
* reworked eclair-cli
* API is disabled by default, disabled CORS and require basic auth password
* better error handling
* Fixed latest version in README
* Increased connection timeout to 15s in electrum client test
* Rgb in NodeAnnouncement is now a Color object
Makes the color field more practical to handle and enable finer
serialization with a more readable code.
allnodes command in api now exports a list with node announcements.
* Added api call to list all the channels updates
This call can also filter the channels for a given nodeId
This fixes#344.
Fixes#318, #317, #306
* README should recommend TESTNET
* (gui) ui can parse lightning: and lightning:// schemes
* (gui) Relaxed decimal amount regex: accepts no leading int
* (gui) Added a node info modal displaying node URI as QR code
* (gui) Logging throwable when gui can not start
* reworked peer management
- connection and channel opening are now separated, simplified
`switchboard`
- use a single authenticator for both incoming and outgoing connections
- `peers` api call now returns current state and channel count
* fixed last commit
* fixed last merge
* added inetsocketaddress serializer
Service pattern matching code visually separates each method and
params to improve the code readability and maintenance. Route completion
is handle on a case by case basis, for each call. This enables better error
management and useful feedback to the caller.
Added custom rejections to handle cases where the given rpc method or
params are not found or not correct.
HTTP code should now be consistent with the error returned.
