SEQN Auth Operations

Environments

Production base URL:

https://auth.seqn.in

Local service default:

http://127.0.0.1:4080

Use local HTTP only for development. Non-localhost targets should use HTTPS.

Operations map

Smoke check

Run the consolidated public MVP readiness report from the VPS first:

cd /srv/silver-auth
npm run readiness:public-mvp

The report separates PASS, BLOCKED, and FAIL items. BLOCKED means the code/deploy path is ready but needs credentials, session state, or an external target; FAIL means something regressed and needs engineering attention before launch.

Run the hosted Croox email/OTP canary from a trusted VPS shell:

cd /srv/silver-auth
SEQN_AUTH_CROOX_CANARY_TO=ops@example.com npm run smoke:croox-email

The canary reads the deployed .env, sends one non-secret direct email and one OTP email through https://comms.seqn.in, and redacts the ck_... key. It proves Croox API/template/provider acceptance; the operator still needs to confirm inbox placement from the mailbox or Croox dashboard.

The service package includes a public, unauthenticated MVP launch smoke:

cd D:\sil
npm.cmd --prefix services\silver-auth-api run smoke:public-mvp

The check validates public health/config, OIDC discovery, SDK CORS preflight, landing/sign-in/sign-up pages, keyless setup, anonymous guard responses, login handoff, SDK tarball availability, docs readiness, docs health, Authentik root redirect, and optional canary publishable/backend key health when SEQN_AUTH_PUBLISHABLE_KEY and SEQN_AUTH_SECRET_KEY are set.

The deployed service package also includes a secret-blind operations audit for VPS-local evidence:

cd /srv/silver-auth
npm run audit:ops

The audit checks production env key presence, dedicated secret-domain env, Croox hosted email config, migration files, backup scripts, docs-site artifacts, local backup freshness, and encrypted offsite backup proof. It prints key names and artifact names only; it must not print secret values. Treat any FAIL line as launch-blocking until intentionally waived.

Run the non-destructive restore drill after a fresh backup and before launch:

cd /srv/silver-auth
npm run drill:restore
npm run audit:ops

The drill restores the latest backup into temporary clean Postgres containers, verifies MANIFEST.sha256, validates the Authentik files archive or skip marker, checks the Silver Auth migration table, writes RESTORE_DRILL.json, and removes its temporary containers.

The SDK package includes a non-mutating smoke check:

cd D:\sil\packages\seqn-auth-js
$env:SEQN_AUTH_BASE_URL="https://auth.seqn.in"
$env:SEQN_AUTH_PUBLISHABLE_KEY="pk_live_..."
$env:SEQN_AUTH_SECRET_KEY="sk_live_..."
npm run smoke

The check validates:

  • The publishable key can load public client config from /v1/client/config.
  • The secret key can verify backend application health from /v1/backend/application.
  • Returned application records are active.

The smoke check does not deploy, create applications, rotate secrets, or write service data.

Authenticated production smoke is session-bound and should run only from a trusted shell:

cd /srv/silver-auth
SEQN_AUTH_SESSION_COOKIE='silver_session=...' npm run smoke:authenticated

By default it verifies /v1/me, organizations, account page, hosted console, app list, audit-log access, webhook event catalog, and the platform Authentik proxy boundary. For a SEQN-A platform admin session, set SEQN_AUTH_EXPECT_PLATFORM_ADMIN=true; for a customer/non-platform org-admin session, leave it false and expect the platform proxy to be denied.

The mutating canary path is opt-in:

SEQN_AUTH_SESSION_COOKIE='silver_session=...' \
SEQN_AUTH_MUTATING_SMOKE=true \
SEQN_AUTH_INVITE_EMAIL=canary@example.com \
npm run smoke:authenticated

It creates a canary application, checks publishable-key config, checks backend secret-key health, rotates the canary secret once, creates a canary invitation, queues a webhook test event, and reads audit logs without printing raw sk_live_, whsec_, invite-token, or session-cookie values.

Daily operations loop

  • Confirm external /healthz and /v1/config checks are green.
  • Confirm CORS preflight for browser SDK calls allows x-seqn-publishable-key.
  • Confirm /auth/console redirects anonymous users into hosted login instead of exposing admin data.
  • Confirm https://docs.seqn.in/ loads if the public docs route is enabled.
  • Confirm the Caddy container has /srv/silver-auth/docs-site/public mounted read-only before recreating Caddy.
  • Check backup freshness and that the newest backup contains MANIFEST.sha256.
  • Run npm run drill:restore for the latest backup after backup or database changes.
  • Check offsite backup age if backup-offsite.sh is enabled.
  • Run npm run audit:ops on the VPS after env, docs, or backup changes.
  • Run npm run smoke:croox-email after Croox API-key, template, DNS, or sender-policy changes.
  • Run npm run smoke:authenticated with a trusted canary session after auth/session/proxy/console changes.
  • Review 429 rate_limited, 401, 402, and 5xx trends before changing limits or plans.
  • Review webhook delivery retries and failed deliveries from the hosted console.
  • Review admin audit logs for unexpected project, user, role, subscription, or secret-rotation events.

Key handling

  • pk_live_ keys can appear in browser code and frontend configuration.
  • sk_live_ keys must stay server-side or in short-lived operations shells.
  • whsec_ webhook signing secrets must stay server-side in webhook consumers.
  • Authentik API tokens and OIDC client secrets must stay in deployment secrets, never browser storage.
  • Store secret keys in the deployment secret store, not in repo files or docs.
  • Rotate keys after accidental disclosure or when moving an application between environments.

Rollout order

  1. Deploy the Silver Auth API endpoint implementation for /v1/client/config and /v1/backend/application.
  2. Run the SDK unit tests locally.
  3. Run the smoke check against staging with staging keys.
  4. Run the smoke check against production with production keys.
  5. Wire app frontend config to loadPublicConfig().
  6. Wire backend boot checks to verifyBackendKeyHealth().
  7. Enable monitoring checks and non-email alerts before public onboarding.
  8. Enable local backups, then offsite encrypted backups.
  9. Publish @seqn/auth-js to npm when package ownership and auth are ready; keep the hosted tarball as fallback until the npm smoke passes.

Failure triage

  • invalid_base_url: the SDK rejected the configured endpoint before making a request.
  • seqn_auth_http_error: the API returned a non-2xx response. Check status, code, and message.
  • seqn_auth_network_error: the request failed before the API response was available. Check DNS, TLS, and proxy routes.
  • application_disabled: the key is valid but the app should not accept traffic.
  • rate_limited: the app limiter rejected the request. Check traffic source, route, and Retry-After.
  • subscription_inactive: the org subscription state blocks mutations.
  • subscription_limit_exceeded: the org reached a plan limit for projects, users, or webhooks.