* Implement new 'routing sync' messages
* add a new feature bit for channel queries
when we receive their init message and check their features:
- if they set `initial_routing_sync` and `channel_range_queries` we do nothing, we should receive a
range query shorly
- if they support channel range queries we send a range query
* Modify query_short_channel_id to ask for a range of ids
And not just a single id
* use `SortedMap` to store channel announcements
* don't send prune channels with channel range queries
* update range queries type to match BOLT PR
* add timestamp-based filters for gossip messages
each peer can speficy a `timestamp range` to filter gossip messages against.
* don't preserve order in `decodeShortChannelIds`
It is not needed and allows us to return a `Set`, which is better suited
to how we use the result.
* channel range queries: handle multi-message responses
Handle case where there are too many short ids to fit in a single message.
* channel range queries: use zlib instead of gzip
but detect when a message was encoded with gzip and reply with gzip in that case.
* router: add more channel range queries logs
* Channel range queries: correctly set firstBlockNum and numberOfBlocks fields
* channel range queries: properly handle case where there is no data
we will just receive on byte (compression format)
* channel range queries: use their compression format to query channels
when we query channels with `query_short_channel_ids`, we now use the same compression
format as in their `repy_channel_range` message. So we should be able to communicate
with peers that have not implemented all compression formats.
* router: make sure that channel ids are sorted
For channel range queries to work properly, channel ids need to be sorted.
It is then much more efficient to use a sorted map in our router state.
* always use `keySet` instead of `keys`
`SortedMap`.`keySet` returns a `SortedSet`, whereas `SortedMap`.`keys`
returns an `Iterable`. This is a critical difference because channel
range queries has requirements on ordering of channel ids.
Using the former allows us to rely on type guarantees instead of on
assumptions that the `Iterable` is sorted in `ChannelRangeQueries`.
There is no cost difference as internally the `Iterator` is actually a
`SortedSet`.
Also, explicitely specified the type instead of relying on comments in
`Router`.
* publish channel update event on router startup
* channel range queries: use uint32 for 4-byte integers (and not int32)
* channel range queries: make sure we send at least one reply to `query_channel_range`
reply to `query_channel_range` messages for which we have no matching channel ids
with a single `reply_channel_range` that contains no channel ids.
* channel range queries: handle `query_channel_range` cleanly
add an explicit test when we have no matching channel ids and send back a reply with an
empty (uncompressed) channel ids payload
* channel range queries: rename GossipTimeRange to GossipTimestampFilter
* channel range queries: add gossip filtering test
* peer: forward all routing messages to the router
and not just channel range queries. this should not break anything and if
it does it would reveal a problem
* peer: add remote node id to messages sent to the router
this will improve logging and debugging and will help if we implement
banning strategies for mis-behaving peers
* router: filter messages with a wrong chainHash more cleanly
* channel range queries: set a "pass-all" timestamp filter
* router: remove useless pattern match
ChannelUpdates are wapped in a PeerRoutingMessage now
* Peer: fit typo in comment
* Peer: optimize our timestamp filter
* Router: use mdc to log remote node id when possible
* fix typos and improve log message formatting
* Peer: rephrase scala doc comment that breaks travis
* Peer: improve timestamp filtering + minor fixes
* Electrum tests: properly stop actor system at the end of the test
* Peer: filter out node announcements against our peer's timestamp
But we don't prune node annoucements for which we don't have a matching
channel in the same "rebroadcast" message
* relay htlcs to channels with the highest balance
In order to reduce unnecessary back-and-forth in case an outgoing
channel doesn't have enough capacity but another one has, the relayer
can now forward a payment to a different channel that the one specified
in the onion (to the same node of course).
If this preferred channel returns an error, then we will retry to the original
requested channel, this way if it fails again, the sender will always receive
an error for the channel she requested.
* improved logs on sig sent/received
* put 'sent announcements' log in debug
* added logging of IN/OUT wire messages
* added mdc support to IO classes
* reduced package length to 24 chars in logs
* add basic electrum wallet test
our wallet connects to a dockerized electrumx server
* electrum: clean up tests, and add watcher docker tests
* electrum wallet: fix balance computation issue
when different keys produced the exact same confirmed + unconfirmed balances, we
would compute an invalid balance because these duplicates would be pruned.
* electrum: rename wallet test
* electrum: add a specific test with identical outputs
* electrum: change scripthash balance logging level to debug
* electrum: make docker tests run on windows/mac
Our electrumx docker container needs to contains to bitcoind that is running on the host.
On linux we use the host network mode, which is not available on windows/osx
On windows/osx we use host.docker.internal, which is not available on linux. This
requires docker 18.03 or higher.
Our electrumx docker container needs to contains to bitcoind that
is running on the host.
On linux we use the host network mode, which is not available on windows/osx
On windows/osx we use host.docker.internal, which is not available on linux. This
requires docker 18.03 or higher.
electrum: change scripthash balance logging level to debug
electrum: add a specific test with identical outputs
electrum: rename wallet test
electrum wallet: fix balance computation issue
when different keys produced the exact same confirmed + unconfirmed balances, we
would compute an invalid balance because these duplicates would be pruned.
electrum: clean up tests, and add watcher docker tests
add basic electrum wallet test
our wallet connects to a dockerized electrumx server
* `ReceivePayment` now accepts additional routing info, which is useful for nodes that are not announced on the network but still want to receive funds.
* Fix scala.NotImplementedError when public-ips config parameter contains invalid values
* more comprehensive validations
* fix unit tests
This fixes#630
We should return a `FeeInsufficient` error when an incoming htlc doesn't
pay us what we require in our latest `channel_update`.
Note that the spec encourages us to being a bit more lax than that (BOLT
7):
> SHOULD accept HTLCs that pay an older fee, for some reasonable time
after sending channel_update.
> Note: this allows for any propagation delay.
* add api call to update channel relay fees
* fixed bug in GUI, as channel can had different fees in each direction!
* fire transitions on `TickRefreshChannelUpdate` (fixes#621)
* make router publish `channel_update`s on startup
* (gui) Channel info fees are now options and case where channels have no known fees data is now properly handled.
* rename InvalidDustLimit to DustLimitTooSmall
* make sure that our reserve is above our dust limit
* check that their accept message is valid
see BOLT 2:
- their channel reserve must be above their dust limit
- their channel reserve must be above our dust limit
- their dust limit must be below our reserve
* channel: check to_local and to_remote amounts againt channel_reserve_satoshis
see BOLT 2: The receiving node MUST fail the channel if both to_local and to_remote amounts for
the initial commitment transaction are less than or equal to channel_reserve_satoshis (see BOLT 3).
* channel: check that their open.max_accepted_htlcs is valid
* Added server address in ElectrumReady object
* Assigned remote address to variable to improve readability
* Checking that the master address exists in the addresses map
* set a minimum feerate-per-kw of 253 (fixes#602)
why 253 and not 250 since feerate-per-kw is feerate-per-kb / 250 and the minimum relay fee rate is 1000 satoshi/Kb ?
because bitcoin core uses neither the actual tx size in bytes or the tx weight to check fees, but a "virtual size" which is (3 * weight) / 4 ...
so we want :
fee > 1000 * virtual size
feerate-per-kw * weight > 1000 * (3 * weight / 4)
feerate_per-kw > 250 + 3000 / (4 * weight)
with a conservative minimum weight of 400, we get a minimum feerate_per-kw of 253
* set minimum fee rate to 2 satoshi/byte
users can still change it to 1 satoshi/byte
* use better weight estimations when computing fees
* test that tx fees are above min-relay-fee
* check that remote fee updates are above acceptable minimum
we need to check that their fee rate is always above our absolute minimum threshold
or we will end up with unrelayable txs
* fix ClaimHtlcSuccessTx weight computation
* channel tests: use actual minimum fee rate
test with our absolute minimum fee rate (253), it should be valid and anything below
sould be invalid and trigger and specific error
Because we keep sending them over and over.
Using `CacheBuilder.weakKeys` will cause the serialized messages to be
cleaned up when messages are garbage collected, hence there is no need
to set a maximum size.
* feerate: use satoshi/kb instead of satoshi/byte
* API fixup: convert input feerate from sat/bytes to sat/kw
* fixup: convert input feerate from sat/bytes to sat/kw
* add cleaner access to current feerate
implementation (blocks 2,6,12...) is not exposed, users will call getFeerate()
* fix feerate conversions
a kilobyte is 1000 bytes, not 1024 bytes (thanks @jimpo)
* revert commit 179dadc
keep this PR focused on 1 task only
* rename FeeratesPerKb to FeeratesPerKB
* reproduces bug decoding ipv4-mapped-as-ipv6 addresses
* handle IPv4-mapped addresses
`InetAddress.getByAddress` automatically converted IPv4-mapped addresses
into IPv4 addresses, which resulted in them being serialized as IPv4
addresses instead of IPv6 addresses, invalidating the signature.
We now use `Inet6Address.getByAddress` instead.
* added support for parsing tor2/tor3 addresses
Note that there are currently invalid announcements on mainnet due to
https://github.com/ElementsProject/lightning/pull/1175#pullrequestreview-111994441.
* support signing multiple identical htlcs
We previously resolved the commitment tx output based only on the
`pubkeyScript`, causing us to reuse the same output for multiple htlcs.
We now keep track of which commitment tx output has been used.
* resolve htlcs using the amount too
We may have htlcs with identical `payment_hash` + `cltv_expiry`, but
different amounts.
In that case we need to use the `amount` to differentiate and choose the
correct commitment output for each htlc.
In other cases, when there is no risk of confusion it is ok to only rely
on `pubkeyScript`.
When publishing delayed txes, we need to first watch the parent
transaction in order to know how many confirmations it has.
Electrum relies on hashes of `pubkeyScript`s in order to track
transactions.
In order to do this without having the parent transaction at hand,
we recompute the `pubkeyScript` from the child transaction witness data
and give it to Electrum.
But if the script was a `pay2wsh`, we were using the `redeemScript`
instead of computing the `pubkeyScript`.
This is the root cause of https://github.com/ACINQ/eclair-wallet/issues/17.
* Accept bech32 addresses
Our android wallet will be able to send funds to bech32 addresses
* improve parsing of base58/bech32 addresses
Return appropriate errors when a base58 address is parseable but on the wrong chain
* add test with invalid address (not parseable as base58 or bech32)
* fix invalid version test
* Enforce a minimum fee rate, with a default at 1 satoshi/byte
Same value as bitcoin core's minimum relay fee
* add `minFeeratePerByte` to `FallbackFeeProvider`
and require that `FeeratesPerByte` and `FeeratesPerKw` be always > 0
* add support for mainnet final pubkeyscript
* electrum: add addresses of electrumx servers on mainnet
* electrum: use chain hash to compute addresses and HD key paths
* store db files in a subdirectory of datadir
* add a 'catchall' around deserialization
* use chain-specific key derivation paths for channel keys
* fixed intermittently failing test (we were comparing timestamps...)
* parameters:
- set default `router-broadcast-interval` to 60s
- set default `mindepth-blocks` to 3 blocks
- removed deprecated setting `router-validate-interval`
- reduce fees: block target 1->2
- reduce `MIN_CLTV_EXPIRY` from 9 to 7
Default value in BOLT11 was indeed 9 blocks, but the absolute minimum
value computed in BOLT2 is 7 blocks.
- remove unused `default-feerate-per-kb`
- set default `max-htlc-value-in-flight-msat`=10mBTC
- set default `max-to-local-delay-blocks` to 2000 blocks
* Update README with instructions for mainnet
* Upgrade to bitcoin-lib 0.9.15 (#516)
* use fees from provider, not default ones
Previously we were only stealing the remote's main output when they publish a revoked commit, and were relying on a sufficiently high `channel_reserve` do deincentivize cheating.
In order to also steal the htlc outputs, we need to handle both cases:
- they only publish their revoked commit tx => we claim the htlc outputs directly from the commit tx
- they publish their revoked commit tx, and their 2nd-stage HTLCSuccessTx and HtlcTimeout txes => we claim the output of these htlcs tx
To do that, we need to be able to reconstruct htlc scripts (`htlcOffered` and `htlcReceived`), therefore we need to store `paymentHash` and `cltvExpiry` for each htlc we sign. Note that this won't be needed in the future when we have MAST.
* store `paymentHash` and `cltvExpiry` for each signed htlc
* spend htlc outputs with penalty txs
* added full integration tests on revoked scenario
add electrum router, which manages a set of client
primary client is the one on the longest chain, when a secondary client
has a chain that is longer than master + 2 it becomes the new master.
this will allow us to move away from "dead" electrum servers that are not
up to date with the blockchain.
electrum client: add 'get header' method
electrum: maintain a chain of headers for each client
and switch to the one with the longest chain
electrum: handle case when received header connects to our first header
it can happen when we connect to a server and ask for the last N headers
electrum: automatic reconnection
when we lose connection to our master client, we consider that we are
disconnected.
when we lose connection to a secondary client, we try to find an address
that is not used and connect to it. If there is no such address we wait
a bit and try to reconnect to the failing client
electrum: check that incoming header proof of work is valid
electrum router: migrate to Akka's FSM
electrum: fix match error for `ready` message
add missing copyright headers
move Blockchain to electrum package
electrum: simplify router
electrum client: fix handling of dead subscribers
electrum: move blockchain validation into electrum client
client will now send `ready` notifications and subscriptions messages only when it has a valid and
long enough blockchain
pass execution context as argument
reworked `ElectrumRouter` following
3dc1c2b61c82debeaed8d0d92238c0b5ee5c49c8
renamed `ElectrumRouter` -> `ElectrumMultiClient`
reformat
`var` -> `val`
electrum: rename multiclient -> pool + cleanup
* added a serializer for `UInt64` class
* added a serializer for `OutPoint` class
* use `txid` instead of `txhash` when serializing `OutPoint`
* added a simple serializer for class `InputInfo`
* set a configurable `maxPaymentFee` as safety
Sending a payment will not be attempted if the cheapest route found is
more expensive than this value. Default value is 3%.
This is meant as a protection mechanism, to protect against an
intermediate well-connected node to set outrageous fees.
* reduced default `fee-base-msat` to 1 sat
* (gui) using max fee from node params when sending payment from ui
Implementation should guarantee that in case of a
`BITCOIN_FUNDING_PUBLISH_FAILED` event, the funding tx will *never* be
published (see `commit` method in trait `EclairWallet`).
With that in mind, there is no need for a `ERR_FUNDING_PUBLISH_FAILED`
state, instead we can just permanently close the channel. Note that the
user will receive an error "couldn't publish funding tx".
As soon as we receive a valid closing signature, we will publish the
resulting closing tx instead of our commitment tx if we need to
immediately close the channel (before end of negotiation, e.g. in case
of errors, or in case the counterparty goes OFFLINE).
When the closing signature didn't correspond to one we sent ourselves, we
weren't properly recognizing the publishing tx and went into
`ERR_INFORMATION_LEAK` state.
CMD_CLOSE is now split it to commands:
* `CMD_CLOSE`: this command will succeed only if the current
channel state is in NORMAL, or if the channel hasn't yet been created.
* `CMD_FORCECLOSE`: the channel will publish its current local
commitment (or its best signed closing tx if it has one).
Using `CMD_FORCECLOSE` is more expensive and it incurs a delay before
funds are spendable, but this can be useful in some situations, for
example when the counterparty isn't responding anymore.
Added a new `forceclose` method to the API and a force close button in
the GUI.
(this fixes a regression caused by 24dadff625)
* restore the old node key path
use the same path as before the change in key management so node id will
remain the same after an upgrade
* add a non-regression test
we check that if the seed does not change, then the node id won't change
either, which could be a problem during a node upgrade.
* type-ified `ShortChannelId`
* removed unused `listUnspent` method and test
* updated sqlite version to 3.21.0.1
* ignore `ReadAck` in `PaymentLifecycle`
* generate all channel keys and secrets from the node key and channel number
* use a key manager
the key manager does not export private keys or secrets.
It exports public keys, and points, and provide methods to sign
transaction.
There is just one exception: it does export revocation secrets, since
we need to send them back when we receive a commitment signature.
* key management: cache private keys and public keys
* add key manager to node parameters
* create an interface for key manager
and an implementation which keeps private keys locally
* generate a new BIP32 key path for each new channel
When we create a new channel we generate a new random BIP32 key path
with 128 bits of entropy
* use a `DirectedWeightedPseudograph`
Because we want a directed graph with multiple weighted edges between two
vertices, and loops allowed.
See discussion in [1] for more details.
* take node fee into account when finding route
Fees vary depending on the amount we want to send, so as a simplification
we use a default `DEFAULT_AMOUNT_MSAT`=`10000000` to compute edges weight.
This fixes#310.
* reuse the same graph for all payments
In `findRoute`, if we have specific updates to add/remove, we don't mutate
the main graph and create a copy instead.
[1]
http://jgrapht-users.107614.n3.nabble.com/Difference-between-a-directed-multigraph-and-a-directed-pseudograph-td4024788.html
Make `Commitment` return `ExpiryTooSmall` and `ExpiryTooBig` when
appropriate, and don't do the check in the `Relayer`.
Be more restrictive when sending HTLCs, so that counterparty doesn't
close the channel when a block just appeared and there is temporarily a
1-block discrepancy between two peers.
Proper management of `FinalExpiryTooSoon` in the payment handler.
On top of that, added more tests and simplified some.
