Migrate KV firewall actions by traversing the global actions-index
instead of iterating per-session buckets, so SQL action IDs follow the
same global creation order as the legacy KV store.
Also stabilize SQL action listing by ordering on created_at and id,
using id as a deterministic tie-breaker when multiple actions share the
same timestamp.
This is needed because KV actions have a real global sequence in
actions-index, while the old migration assigned SQL IDs based on bucket
traversal order. That could reorder legacy actions during migration.
Separately, ordering by created_at alone was not stable for equal
timestamps, so action queries could return different orders for the
same data.
NOTE: This commit explicitly edits the previous migration instead of
adding a new one. This is ok as SQL dbs are not yet supported in
production, so there are no live deployments to worry about.
Normalize action timestamp handling to UTC during KV and SQL
marshalling so migration validation behaves consistently on
Postgres backends outside UTC.
This also allows the action time comparison helper to use the
same timezone normalization approach as the other migration
paths.
Add clearer progress logging across the KVDB to SQL migration paths
for accounts, sessions, and firewalldb.
Log the total number of items to migrate where it is cheap to
determine up front, and emit progress updates every 100 migrated
items.
Mark the legacy kvdb stores as deprecated once the kvdb -> SQL
migration commits successfully. This prevents normal bbolt startup
from reopening accounts.db, session.db, or rules.db after their data
has already been migrated.
Add explicit deprecation checks to the three kvdb store open paths and
provide migration-only constructors that can still reopen deprecated
files when the SQL database is deleted or downgraded and the migration
must be rerun.
Use store-specific tombstones for the deprecation markers and add
tests that verify deprecated stores are rejected while migration
reruns continue to work.
- Replace occurrences of `// nolint:lll` with `// nolint:ll` across
files for consistency.
- Reformat multiline strings, comments, and function parameters to
improve clarity and adhere to style guidelines.
- Add `// nolint:ll` comments where necessary to prevent linter
warnings.
If there are no RPCParamsJson set for an action, the value is
represented differently in KVDB vs SQL.
In the SQL DB, empty RPCParamsJson are represented as nil, while they
are represented as an empty array in the KVDB version. Therefore, we
need to override the RPCParamsJson in that scenario, so that they are
set to the same representation when a KVDB and an SQL action is
compared.
If the user has deleted their session.db file, but kept their
rules.db file, there can exist privacy mapper pairs that point to a now
deleted session ID. Such pairs should be ignored during the migration,
as they are cannot be used anymore.
This commit updates the migration to handle this case.
If the user has deleted their session.db file, but kept their
rules.db file, there can exist kv entry values that point to a now
deleted session ID. Such kv entries should be ignored during the
migration, as they are cannot be used anymore.
This commit updates the migration to handle this case.
In the upcoming commits, we will update the kv stores and the privacy
mapper migration to not migrate entries if their linked session has been
deleted. As those checks will need to query the SQL db to see if the
session still exists, we move the session alias to session map to not
only be used in the actions migration, but throughout the migration
when ever we need to query a session by its alias. This is done to
avoid multiple queries to the SQL db for the same session alias, to
improve the performance of the migration.
This commit introduces the migration logic for transitioning the
actions store from kvdb to SQL.
Note that as of this commit, the migration is not yet triggered by any
production code, i.e. only tests execute the migration logic.
This commit introduces the migration logic for transitioning the
privacy mapper store from kvdb to SQL.
Note that as of this commit, the migration is not yet triggered by any
production code, i.e. only tests execute the migration logic.
This commit removes speculative comments from the firewalldb migration
docs that forecast future implementations. Such comments can create
confusion for developers looking at the current code base without
knowing our plans how the migration will be further developed.
This commit introduces the migration logic for transitioning the
kvstores store from kvdb to SQL.
Note that as of this commit, the migration is not yet triggered by any
production code, i.e. only tests execute the migration logic.
