docs: Use admonition markup for warnings and notes

doc/BACKUP.md

@@ -30,7 +30,9 @@ For example, if you are running `--mainnet`, it will be
 
 ## `hsm_secret`
 
-`/!\` WHO SHOULD DO THIS: Everyone.
+!!! note
+
+	WHO SHOULD DO THIS: Everyone.
 
 You need a copy of the `hsm_secret` file regardless of whatever backup
 strategy you use.
@@ -84,12 +86,14 @@ backup strategies below.
 
 ## SQLITE3 `--wallet=${main}:${backup}` And Remote NFS Mount
 
-`/!\` WHO SHOULD DO THIS: Casual users.
+!!! note
 
-`/!\` **CAUTION** `/!\` This technique is only supported after the version v0.10.2 (not included)
+	WHO SHOULD DO THIS: Casual users.
-or later.
+
-On earlier versions, the `:` character is not special and will be
+!!! warning
-considered part of the path of the database file.
+
+	This technique is only supported after the version v0.10.2 (not included) or later.
+	On earlier versions, the `:` character is not special and will be considered part of the path of the database file.
 
 When using the SQLITE3 backend (the default), you can specify a
 second database file to replicate to, by separating the second
@@ -100,11 +104,15 @@ For example, if the user running `lightningd` is named `user`, and
 you are on the Bitcoin mainnet with the default `${LIGHTNINGDIR}`, you
 can specify in your `config` file:
 
+```bash
 wallet=sqlite3:///home/user/.lightning/bitcoin/lightningd.sqlite3:/my/backup/lightningd.sqlite3
+```
 
 Or via command line:
 
+```bash
 lightningd --wallet=sqlite3:///home/user/.lightning/bitcoin/lightningd.sqlite3:/my/backup/lightningd.sqlite3
+```
 
 If the second database file does not exist but the directory that would
 contain it does exist, the file is created.
@@ -173,7 +181,9 @@ like fire or computer confiscation.
 
 ## `backup` Plugin And Remote NFS Mount
 
-`/!\` WHO SHOULD DO THIS: Casual users.
+!!! note
+
+	WHO SHOULD DO THIS: Casual users.
 
 You can find the full source for the `backup` plugin here:
 https://github.com/lightningd/plugins/tree/master/backup
@@ -221,8 +231,9 @@ like fire or computer confiscation.
 
 ## Filesystem Redundancy
 
-`/!\` WHO SHOULD DO THIS: Filesystem nerds, data hoarders, home labs,
+!!! note
-enterprise users.
+
+	WHO SHOULD DO THIS: Filesystem nerds, data hoarders, home labs, enterprise users.
 
 You can set up a RAID-1 with multiple storage devices, and point the
 `$LIGHTNINGDIR` to the RAID-1 setup.
@@ -336,7 +347,9 @@ of new storage devices to set up a new node.
 
 ## PostgreSQL Cluster
 
-`/!\` WHO SHOULD DO THIS: Enterprise users, whales.
+!!! note
+	
+	WHO SHOULD DO THIS: Enterprise users, whales.
 
 `lightningd` may also be compiled with PostgreSQL support.
 PostgreSQL is generally faster than SQLITE3, and also supports running a
@@ -420,9 +433,74 @@ This can be difficult to create remote replicas due to the latency.
 
 [pqsyncreplication]: https://www.postgresql.org/docs/13/warm-standby.html#SYNCHRONOUS-REPLICATION
 
+## SQLite Litestream Replication
+
+!!! warning
+
+	Previous versions of this document recommended this technique, but we no longer do so.
+	According to [issue 4857][], even with a 60-second timeout that we added
+	in 0.10.2, this leads to constant crashing of `lightningd` in some
+	situations.
+	This section will be removed completely six months after 0.10.3.
+	Consider using 
+	
+	```
+	--wallet=sqlite3://${main}:${backup}
+	```
+	
+	above, instead.
+
+[issue 4857]: https://github.com/ElementsProject/lightning/issues/4857
+
+One of the simpler things on any system is to use Litestream to replicate the SQLite database.
+It continuously streams SQLite changes to file or external storage - the cloud storage option
+should not be used.
+Backups/replication should not be on the same disk as the original SQLite DB.
+
+You need to enable WAL mode on your database.
+To do so, first stop `lightningd`, then:
+
+    $ sqlite3 lightningd.sqlite3
+    sqlite3> PRAGMA journal_mode = WAL;
+    sqlite3> .quit
+
+Then just restart `lightningd`.
+
+/etc/litestream.yml :
+
+    dbs:
+     - path: /home/bitcoin/.lightning/bitcoin/lightningd.sqlite3
+       replicas:
+         - path: /media/storage/lightning_backup
+
+ and start the service using systemctl:
+
+    $ sudo systemctl start litestream
+
+Restore:
+
+    $ litestream restore -o /media/storage/lightning_backup  /home/bitcoin/restore_lightningd.sqlite3
+
+Because Litestream only copies small changes and not the entire
+database (holding a read lock on the file while doing so), the
+60-second timeout on locking should not be reached unless
+something has made your backup medium very very slow.
+
+Litestream has its own timer, so there is a tiny (but
+non-negligible) probability that `lightningd` updates the
+database, then irrevocably commits to the update by sending
+revocation keys to the counterparty, and *then* your main
+storage media crashes before Litestream can replicate the
+update.
+Treat this as a superior version of "Database File Backups"
+section below and prefer recovering via other backup methods
+first.
+
 ## Database File Backups
 
-`/!\` WHO SHOULD DO THIS: Those who already have at least one of the
+!!! note
+
+	WHO SHOULD DO THIS: Those who already have at least one of the
 	other backup methods, those who are #reckless.
 
 This is the least desirable backup strategy, as it *can* lead to loss
@@ -528,3 +606,38 @@ still not assured with this backup strategy.
 `sqlite3` has `.dump` and `VACUUM INTO` commands, but note that
 those lock the main database for long time periods, which will
 negatively affect your `lightningd` instance.
+
+### `sqlite3` `.dump` or `VACUUM INTO` Commands
+
+!!! warning
+
+	Previous versions of this document recommended
+	this technique, but we no longer do so.
+	According to [issue 4857][issue 4857], even with a 60-second timeout that we added
+	in 0.10.2, this may lead to constant crashing of `lightningd` in some
+	situations; this technique uses substantially the same techniques as
+	`litestream`.
+	This section will be removed completely six months after 0.10.3.
+	Consider using `--wallet=sqlite3://${main}:${backup}` above, instead.
+
+Use the `sqlite3` command on the `lightningd.sqlite3` file, and
+feed it with `.dump "/path/to/backup.sqlite3"` or `VACUUM INTO
+"/path/to/backup.sqlite3";`.
+
+These create a snapshot copy that, unlike the previous technique,
+is assuredly uncorrupted (barring any corruption caused by your
+backup media).
+
+However, if the copying process takes a long time (approaching the
+timeout of 60 seconds) then you run the risk of `lightningd`
+attempting to grab a write lock, waiting up to 60 seconds, and
+then failing with a "database is locked" error.
+Your backup system could `.dump` to a fast `tmpfs` RAMDISK or
+local media, and *then* copy to the final backup media on a remote
+system accessed via slow network, for example, to reduce this
+risk.
+
+It is recommended that you use `.dump` instead of `VACUUM INTO`,
+as that is assuredly faster; you can just open the backup copy
+in a new `sqlite3` session and `VACUUM;` to reduce the size
+of the backup.