cln-plugins/backup
Seth For Privacy af8bc417ca Add referenced requirements.txt
This file is referenced in https://lightning.readthedocs.io/BACKUP.html#backup-plugin-and-remote-nfs-mount, but does not exist, making installation and usage of backup plugin much more tricky.

This commit simply adds the two required packages so that the docs are correct now and the plugin can be more easily installed.
2022-08-22 12:00:09 +02:00
..
tests backup: Add a test to reproduce #158 2020-12-11 10:52:23 +01:00
backend.py docs: Rename from c-lightning to core-lightning for local plugins 2022-05-13 18:07:13 +02:00
backends.py backup: Implement network backup 2021-02-04 13:06:41 +01:00
backup-cli docs: Rename from c-lightning to core-lightning for local plugins 2022-05-13 18:07:13 +02:00
backup.py docs: Rename from c-lightning to core-lightning for local plugins 2022-05-13 18:07:13 +02:00
filebackend.py docs: Rename from c-lightning to core-lightning for local plugins 2022-05-13 18:07:13 +02:00
poetry.lock gci: Do not nest virtualenvs in poetry 2022-04-28 12:57:06 +02:00
protocol.py backup: Implement network backup 2021-02-04 13:06:41 +01:00
pyproject.toml docs: Rename from c-lightning to core-lightning for local plugins 2022-05-13 18:07:13 +02:00
README.md backup: doc use poetry run also for backup-cli cmd 2022-07-21 19:06:34 +02:00
remote.md docs: Rename from c-lightning to core-lightning for local plugins 2022-05-13 18:07:13 +02:00
requirements.txt Add referenced requirements.txt 2022-08-22 12:00:09 +02:00
server.py backup: Improve logging configurability for socket server 2021-02-26 10:38:22 +01:00
socketbackend.py backup: Implement automatic reconnect in socket backend 2021-07-23 17:44:38 +09:30
test_backup.py backup: Add support for Tor 2021-02-26 10:35:48 +01:00

A simple and reliable backup plugin

This version only supports the default SQLite3 database

This plugin will maintain clean database backups to another location. It uses the db_write hook to make sure to always have a backup that is not missing any state updates and is not potentially harmful.

Related info about backup solutions: https://github.com/ElementsProject/lightning/blob/master/doc/BACKUP.md

Installation

There are some Python dependencies. You can install them using poetry:

poetry install

Setup

Before the backup plugin can be used it has to be initialized once. The following command will create /mnt/external/location/file.sql as backup file and reference it in backup.lock in the lightning directory that stores the internal state, and which makes sure no two instances are using the same backup. (Make sure to stop your Lightning node before running this command)

poetry run ./backup-cli init --lightning-dir ~/.lightning/bitcoin file:///mnt/external/location/file.bkp

Notes:

  • If you are not using the default lightning directory you'll need to change ~/.lightning/bitcoin in the command line to point to that directory instead.
  • You should use some non-local SSH or NFS mount as destination, otherwise any failure of the disk may result in both the original as well as the backup being corrupted.
  • Currently only the file:/// URL scheme is supported.

IMPORTANT note about hsm_secret

You need to secure ~/.lightning/bitcoin/hsm_secret once! This file will not change, but without this file, the database backup will be unusable!

Make sure it has user read only permissions, otherwise lightningd will refuse to work: chmod 0400 hsm_secret

Running

In order to tell lightningd to use the plugin you either need to tell it via the startup option --plugin /path/to/backup.py or by placing it (or a symlink to it) in the lightning plugin directory (~/.lightning/plugins) or by adding it to the lightningd configuration (important-plugin=/path/to/backup.py).

On daemon startup the plugin will check the integrity of the existing backup and complain if there is a version mismatch.

Performing backup compaction

A backup compaction incorporates incremental updates into a single snapshot. This will reduce the size of the backup file and reduce the time needed to restore the backup. This can be done through the plugin command backup-compact:

lightning-cli backup-compact

Be aware that this can take a long time depending on the size of the backup and I/O speeds, during which the daemon will not be reachable.

Restoring a backup

If things really messed up and you need to reinstall clightning, you can restore the database backup by using the backup-cli utility:

./backup-cli restore file:///mnt/external/location ~/.lightning/bitcoin/lightningd.sqlite3