Initialize the migrated session's MacaroonRecipe before applying the
nil-perms and nil-caveats normalization used during migration
validation.
This is needed because the SQL store can represent a session with no
macaroon permission or caveat rows as a nil MacaroonRecipe, while the
KV store may still have a non-nil recipe wrapper. Without this guard,
overrideMacaroonRecipe can dereference a nil migrated recipe and panic
instead of letting the migration validation complete normally.
Using an empty MacaroonRecipe preserves the intended normalization
behavior for the nil/empty recipe cases while still allowing genuine
mismatches to fail through DeepEqual.
Store explicit positions for session macaroon caveats and permissions
in the SQL schema and read them back in position order. Also remove the
migration-time sorting workaround in session comparison, so migration
validation now checks the actual persisted order instead of masking it.
This is needed because session caveat order is not just presentation
data. LiT adds caveats to the baked macaroon in slice order, and the
macaroon library updates the signature hash chain for each added caveat.
Reordering caveats can therefore change the resulting macaroon bytes and
signature.
The previous schema split caveats and permissions into child tables
without any position column, and the SQL reads had no ORDER BY. The KV
store preserves slice order, but SQL had no explicit way to reproduce
that order after migration or on later reads. The migration code’s old
sorting step was only making validation deterministic; it did not
preserve the original recipe order.
Permissions are canonicalized by lnd when baking, so their order is less
semantically important for the final macaroon. They still get positions
here so the stored recipe remains faithful to the original session data
and both child tables behave consistently.
Why it was needed:
- caveats needed explicit order preservation because they are appended
and signed in order.
- The old SQL schema did not store order, and the read queries did not
request one.
- Adding position makes the SQL representation faithful to the KV/TLV
recipe instead of relying on incidental row order.
- Adding it to permissions too keeps the stored recipe lossless and
consistent, even though lnd. canonicalizes permissions before baking.
NOTE: This commit explicitly edits the previous migration instead of
adding a new one. This is ok as SQL dbs are not yet supported in
production, so there are no live deployments to worry about.
Add migration coverage for nested empty caveat verification IDs
and normalize the migrated SQL representation so empty byte
slices compare consistently with the KV store.
Add clearer progress logging across the KVDB to SQL migration paths
for accounts, sessions, and firewalldb.
Log the total number of items to migrate where it is cheap to
determine up front, and emit progress updates every 100 migrated
items.
Mark the legacy kvdb stores as deprecated once the kvdb -> SQL
migration commits successfully. This prevents normal bbolt startup
from reopening accounts.db, session.db, or rules.db after their data
has already been migrated.
Add explicit deprecation checks to the three kvdb store open paths and
provide migration-only constructors that can still reopen deprecated
files when the SQL database is deleted or downgraded and the migration
must be rerun.
Use store-specific tombstones for the deprecation markers and add
tests that verify deprecated stores are rejected while migration
reruns continue to work.
In the bbolt store, an empty config for a feature is represented as an
empty array, while in for the SQL store, the same config is represented
as nil. Therefore, in the scenario where a specific feature has an empty
config, we override the SQL FeatureConfig for that feature to also be
set to an empty array. This is needed to ensure that the deep equals
check in the migration validation does not fail in this scenario.
If a user creates a session linked to an account, but later deletes that
account using the `litcli accounts remove` command, the KVDB session
will still contain the account ID of the removed account.
During the KVDB to SQL migration, the accounts migration runs before the
sessions migration. Since the deleted account no longer exists, it
cannot be migrated. As a result, the migrated SQL session cannot link to
the now non-existent account in the SQL store.
In such cases, we will migrate the session and not link to any account
in the SQL session.
The logic to avoid duplicate session IDs was first added with the
session linking linking functionality. That means that there may be old
legacy sessions that were added prior to that which may have duplicate
IDs. Before this commit, such sessions would cause the kvdb to SQL
migration to error, as there is a uniqueness constraint on the legacy
session alias in the SQL sessions table.
We want to keep that constraint, so in such rare cases, we update the
kvdb to SQL migration logic to drop all but the session with the
latest CreatedAt time if for sessions that share the same ID.
That follows the logic in c8b78bd10d as
closely as possible, as that migration ensured that all sessions but the
latest one was revoked if multiple sessions shared the same ID.
Similar to the previous commit, we also sort the
`MacaroonRecipe.Permissions` slice to ensure it can be compared in a
deterministic manner during migrations.
In the kvdb to sql migration, if there have been caveats set for the
MacaroonRecipe, the order of the postgres db caveats will in very rare
cases differ from the kv store caveats. Therefore, we sort both the kv
and sql caveats by their ID, so that we can compare them in a
deterministic way.
This commit introduces the migration logic for transitioning the
sessions store from kvdb to SQL.
Note that as of this commit, the migration is not yet triggered by any
production code, i.e. only tests execute the migration logic.
