The kvdb-to-SQL data migration polls lnd's ListMacaroonIDs RPC, which
only becomes available once lnd reaches its "RPC active" state. On
nodes with a large channel/graph state, lnd can take well over a
minute to get there after the wallet is unlocked, which exceeded the
previous fixed 60-second (120 x 500ms) poll budget and caused the
migration - and therefore litd startup - to fail permanently,
requiring a manual restart.
Replace the fixed attempt cap with a wait bounded by the new
--lndreadytimeout config option, defaulting to a generous 10 minutes,
while still aborting early if the daemon is shutting down. The wait
happens inside the migration's SQL write transaction, so it is kept
bounded rather than unbounded as a safety backstop.
Avoid creating legacy bbolt database files when the kvdb-to-SQL
programmatic migration runs on a node that never had LiT kvdb state.
Note that we only skip the creating the database file if there is no
single bbolt database file. If for example there's only 1 existing
bbolt database file, we will still create the new bbolt database file
for the other 2 bbolt databases, as we for example need to pass the
accounts & session databases to the firewalldb migration, and we'd want
to avoid having the migrations become more complex by having to handle
the case where the other databases are missing.
Due to that certain wallet setups, such as `lnd.noseedbackup`, can take
a bit of time to open the `lnd` wallet, we need to increase the time we
allow for the RPC call which lists the macaroon IDs.
We therefore allow up to 60 seconds instead of 5 seconds for that to
happen, to ensure that the kvdb->SQL migration doesn't fail in such
setups.
The kvdb accounts db is specifically set to be created in the same
folder as the macaroon path, while the sessions and rules db are created
in the network folder.
That means that users which have set a custom macaroon path, can have
an accounts db in a separate folder than the sessions and rules db.
In order to ensure that the migration from kvdb to SQL uses the correct
db folders, we need to make sure that the migration code uses the same
logic as the creation of the db logic.
Remove the KVDB-to-SQL migration from the dev SQL migration set now
that this migration is now part of the production migration set.
As the migrations_dev folder no longer contains any migrations, we also
update the db logic and the gen_sqlc_docker.sh script to not error when
the folder(s) are empty.
Move the programmatic KVDB-to-SQL migration out of the dev-only
migration stream and into the production SQL migration set.
This also requires that we add a new migration file for the prod
migration, which will trigger the kvdb to sql programmatic migration to
run in production.
With the latest version of the `sqldb/v2` & `migrate` packages, Post
Migration Callbacks were renamed to Programmatic Migrations.
Our previous codebase still had some mentions of Post Migration
Callbacks, and therefore we rename those to use the word Programmatic
Migrations instead.
Mark the legacy kvdb stores as deprecated once the kvdb -> SQL
migration commits successfully. This prevents normal bbolt startup
from reopening accounts.db, session.db, or rules.db after their data
has already been migrated.
Add explicit deprecation checks to the three kvdb store open paths and
provide migration-only constructors that can still reopen deprecated
files when the SQL database is deleted or downgraded and the migration
must be rerun.
Use store-specific tombstones for the deprecation markers and add
tests that verify deprecated stores are rejected while migration
reruns continue to work.
