SEQN Auth Key Rotation Runbook

This runbook covers rotation for SEQN Auth credentials while keeping email delivery and payment-provider flows out of scope.

Credential inventory

| Credential | Prefix or env | Owner | Rotation impact | | --- | --- | --- | --- | | Publishable project key | pk_live_ | App/admin console | Browser-safe; rotate only by creating or recreating projects. | | Project secret key | sk_live_ | App backend | Backend integrations fail until consumers receive the new key. | | Webhook signing secret | whsec_ | Webhook receiver | Receivers fail signature validation until they accept the new secret. | | OIDC client secret | OIDC_CLIENT_SECRET | Silver Auth API/Authentik | New login callbacks can fail if API and Authentik disagree. | | Authentik API token | AUTHENTIK_API_TOKEN | Silver Auth API/Authentik | Project provisioning is blocked while invalid. | | Session secret | SESSION_SECRET | Silver Auth API | Existing app sessions are invalidated if single-secret rotation is used. | | Database passwords | AUTHENTIK_PG_PASS, SILVER_POSTGRES_PASSWORD | VPS ops | Requires coordinated database and service restart. | | Backup encryption key | OFFSITE_BACKUP_AGE_RECIPIENT or OFFSITE_BACKUP_GPG_RECIPIENT | VPS ops | Old backups require the old private key until retention expires. |

Standard safety rules

  • Take a fresh local backup before high-blast-radius rotations.
  • Keep a dated incident or maintenance note with actor, reason, scope, and rollback plan.
  • Never paste raw secrets into docs, alerts, screenshots, support tickets, or chat.
  • Prefer secret stores and one-time terminals over local files.
  • Rotate canaries first when possible.

Project secret key rotation

Use for exposed sk_live_ keys, environment moves, or scheduled hygiene.

1. Identify the project and backend consumers. 2. Prepare the receiving app secret store for the new value. 3. In the hosted console, rotate the project secret. 4. Copy the one-time sk_live_ value directly into the backend secret store. 5. Restart or redeploy the backend consumers. 6. Run GET /v1/backend/application from the backend environment. 7. Confirm no new invalid_secret_key errors.

Current rotation replaces the valid secret immediately, so schedule production rotations when consumers can be updated at the same time.

Webhook signing secret rotation

Use for exposed whsec_ values or scheduled receiver hygiene.

1. Confirm the receiver can accept a new secret. 2. If the receiver supports dual secrets, add a pending new secret slot. 3. Rotate the webhook endpoint secret in the hosted console. 4. Copy the one-time whsec_ value into the receiver secret store. 5. Send a test webhook event. 6. Confirm delivery logs show successful signatures. 7. Remove the old receiver secret after the grace window.

If the receiver cannot accept dual secrets, schedule a short maintenance window because outgoing signatures switch immediately.

OIDC client secret rotation

1. Take a fresh local backup. 2. Create or rotate the client secret on the Authentik OAuth2 provider for silver-auth. 3. Update the VPS .env value for OIDC_CLIENT_SECRET. 4. Recreate only the silver-auth-api container. 5. Start a fresh login from an incognito browser. 6. Confirm /auth/callback completes and /v1/me returns the signed-in user.

Active sessions should continue unless SESSION_SECRET also changes.

Authentik API token rotation

1. Create a replacement Authentik API token with the same minimum provisioning permissions. 2. Update AUTHENTIK_API_TOKEN in the VPS .env. 3. Recreate only the silver-auth-api container. 4. Create or update a canary project. 5. Confirm provisioning status is synced. 6. Revoke the old Authentik token.

Session secret rotation

SESSION_SECRET protects opaque session token hashes and cookies. With the current single-secret model, rotation signs users out.

1. Announce a maintenance window. 2. Take a fresh local backup. 3. Replace SESSION_SECRET with at least 32 random characters. 4. Recreate silver-auth-api. 5. Confirm anonymous users redirect to login and new sessions work. 6. Monitor support channels for forced sign-in reports.

Database password rotation

1. Take a fresh local and offsite backup. 2. Stop write traffic if possible. 3. Rotate the Postgres role password inside the target database. 4. Update the matching .env variable. 5. Recreate dependent containers. 6. Confirm /healthz, /v1/config, login, and admin console access.

Backup encryption key rotation

1. Add the new public recipient to .env. 2. Run backup-offsite.sh. 3. Confirm the new archive can be decrypted by the new private key. 4. Keep the old private key until all old encrypted backups pass retention. 5. Remove the old private key only after restore coverage is no longer needed.