* Electrum: don't ask for merkle proofs for unconfirmed txs
* Electrum: clear status when we get disconnected and still have pending history transactions
When we get disconnected and have script hashes for which we still have pending connections,
clear the script hash status. When we reconnect we will ask for its history again, this way we
won't miss anything. Since we rotate keys it should not result in heavy traffic (script hashes have
few history items).
* Electrum: represent and persist block heights as 32 bits signed ints
Int.MaxValue is about 40,000 years of block which should be enough, and it will fix the encoding
problem users on testnet when there's a reorg and one of their txs has a height of -1.
Side-note: changing the wallet codec can be done easily: if we cannot read and decode persisted data
we just start with an empty wallet and retrieve all wallet data from electrum servers, and once it's
ready it will be encoded with the new codec and saved.
* Electrum persistence: include a version number
It provides a clean way, when upgrading the app, of choosing whether to keep the same version and start from the
last persisted wallet (if the persistence format has not been changed or is compatible with the old one), or to
change the version and force starting from an empty wallet and downloading all wallet items from Electrum servers.
* ElectrumClient: remove useless buffer
The GUI took a very long time to startup because nodes and channels were
stored in javafx `ObervableList` which doesn't allow random access. We
can't easily use `ObservableMap` because `TableView` do not support
them.
As a workaround, created an `IndexedObservableList` which keeps track of
indices in `ObservableList` so that we can still have fast random
access.
Also, now grouping announcements in `NetworkEvent` instead of sending
them one by one.
On android there are some cases where we are not aware that a local public
channel as in fact been closed. If this happens we will tell the router to
forget about it.
If the channel is in fact still alive, we will get it again via network
announcements anyway.
* Enable searching for routes with size/CLTV/fee limits
* Expose the RouteParams in RouteRequest
* Expose the RouteParams in SendPayment
* Rename DEFAULT_ROUTE_MAX_LENGTH
* If we couldn't find a route on the first attempt, retry relaxing the restriction on the route size
* Avoid returning an empty path, collapse the route not found cases into one
* When retrying to search for a route, relax 'maxCltv'
* Move the default params for route searching in the conf, refactor together router params into a config class
* Remove max-payment-fee in favor of router.search-max-fee-pct
* Group search params configurations into a block
* Rename config keys for router path-finding
When we are disconnected and we don't have any channel with that peer,
we can stop it and remove its address from db.
We also need to handle the case where we are in `DISCONNECTED` or
`INITIALIZING` state and we are notified that the last channel we had
with that peer just got closed.
Note that this opens a race condition in the switchboard, if we receive
an incoming connection from that same peer right after we stopped it,
and before the switchboard received the `Terminated` event. If that
happens, then `Peer.Connect` or `Peer.OpenChannel` will timeout. We
could also have the switchboard listens to deadletter events, but that
seems a bit over the top.
Also, removed switchboard's map of peers. Instead, we use the actor's
children list, and access peers using the recommended `child()` method.
Now the 'peers request only returns an `Iterable` instead of a `Map`. This
removes the need to watch child actors, and thus removes the race
condition when peers were stopped. As a trade-off, peer lookup is now
in O(log(N)) instead of O(1) (with N being the number of peers), but this
seems acceptable.
Persist a partial view of our wallet to enable faster startup time.
Users will be able to create transactions as soon as we've checked that we're connected to a "good" electrum server. In the unlikely event where they were able to create and try to publish a transaction during the few seconds it took to finish syncing and their utxo was spent (because they were trying to use an unconfirmed utxo that got double spent, or if they spent their utxo themselves from another wallet initialized with the same seed while his one was offline), publishing the tx will fail.
* Electrum: persist wallet state
Wallet state is persisted and reloaded on startup.
* Electrum wallet: fix handling of headers chunk
When we receive a tx that is older than our checkpoints, we download and check
the header chunk that it's in, check it and connect it to our chain.
* Electrum: advertise wallet transactions
Send notifications for all wallet transactions once we're connected and synced
* Electrum: add timestamp to wallet events
Add an optional timestamp to TransactionReceived and TransactionConfidenceChanged, which is the timestamp of the block the tx was confirmed in (if any).
* Electrum client: use a Close message
This will fix concurrency issues where handlers are called when the actor
is already stopped.
Note that balance events are logged at most once every 30s, and only when
the balance actually changes (e.g. won't log if a payment is failed).
Also, only send `AvailableBalanceChanged` when needed.
We were sending this event everytime we sent a `commit_sig`, which is
incorrect because our balance doesn't change if, say, we are signing an
incoming htlc.
Note that we only send this event in `NORMAL` state, not in `SHUTDOWN`
state, because the balance is not really _available_ in the latter.
This includes support for hosting onion services, and connecting to them, which are two separate things:
- Opening an onion service implie interacting with the tor daemon controller, which requires authentication. We support both `SAFECOOKIE` and `HASHEDPASSWORD` authentication mechanisms, with a default to `SAFECOOKIE`. We support v2 and v3 services, with a default to v3 as recommended by the tor project.
- Connecting to onion services requires tunnelling through tor's local SOCKS5 proxy.
Incoming and outgoing tor connections are thus separate matters that needs to be configured independently. A specific documentation has been added to guide users through these steps.
Big thanks to @rorp for doing the heavy lifting on all this!
While it makes sense to exclude from the routing table channels for
which there is a spending tx in the mempool, we shouldn't blame our
peers for considering the channel still opened if the spending tx hasn't
yet been confirmed.
Also, reworked `ValidateResult` with better types. It appears that the
`NonexistingChannel` error wasn't really useful (a malicious peer would
probably point to an existing funding tx, so there is no real difference
between "txid not found" and "invalid script"), so it was replaced by
`InvalidAnnouncement` error which is a more serious offense (punished
by a disconnection, and probably a ban when we implement that sort of
things).
* Make route randomization optional (enabled by default), option is exposed in SendPayment/RouteRequest
* Fix non deterministic behavior in IntegrationTest
We previously assumed that the `gossip_timestamp_filter` only applied to
future messages, so we set `first_timestamp` to 0 in order to have a
pass-all filter.
But BOLT 7 actually says that the remote peer:
> SHOULD send all gossip messages whose timestamp is greater or equal to
first_timestamp, and less than first_timestamp plus timestamp_range
Which means that the way our filter was set, the remote peer would dump
the entire routing table on us.
By setting `first_timestamp` to the current time, we achieve what we
want. The synchronization of older messages is done by sending a
`query_channel_range` and then one or several `query_short_channel_ids`.
* relay to channel with lowest possible balance
Our current channel selection is very simplistic: we relay to the
channel with the largest balance. As time goes by, this leads to all
channels having the same balance.
A better strategy is to relay to the channel which has the smallest
balance but has enough to process the payment. This way we save larger
channels for larger payments, and also on average channels get depleted
one after the other.
* added tests...
...and found bugs!
Note that there is something fishy in BOLT 4, filed a PR:
https://github.com/lightningnetwork/lightning-rfc/pull/538
Also, first try of softwaremill's quicklens lib (in scope test for now)
* minor: fixed typo (h/t @btcontract)
This is a simple mechanism to test our direct peers by sending fake
small payments to them. A probe is considered successful if the peer
replies with an `UnknownPaymentHash` error.
Probing is configurable and disabled by default.
We use Yen's algorithm to find the k-shortest (loopless) paths in a graph, and dijkstra as search algo.
We then pick a random route up among the cheapest ones.
* Electrum: always stop clients when there's an error
We will automatically connect to another server, and it's much cleaner
that restarting right away
* Electrum: improve handling of server errors
* Electrum Pool: don't switch to our current master
In regtest (and probably testnet too) it may seem that our master server went
straight to current height + 2 when blocks are created very quickly, so
check first that the server is not already our master before we switch.
