SEQN Auth Support and Admin Runbook

This runbook is for support/admin operations across auth, projects, webhooks, subscriptions, account recovery handoff, and invitation email delivery. Email verification workflows and payment-provider workflows remain outside the current MVP.

Intake checklist

  • Capture the customer org slug, project slug, user email, request ID or approximate timestamp, and affected route.
  • Confirm whether the issue is authentication, authorization, project config, subscription state, webhook delivery, or platform availability.
  • Never request raw sk_live_, whsec_, OIDC client secrets, Authentik API tokens, database passwords, or session cookies.
  • Ask for secret prefixes only when needed, for example sk_live_abcd1234.
  • Link the incident or support ticket to the relevant audit-log entries.

Account cannot sign in

Symptoms:

  • Login redirects loop.
  • Callback returns an error.
  • User lands in the wrong workspace.
  • User cannot complete password recovery.

Actions:

  1. Check public health and Authentik OIDC discovery.
  2. Confirm the user's Authentik identity exists and has the expected email.
  3. Confirm platform admins are in SEQN-A.
  4. For non-platform customers, confirm public signup created a tenant workspace.
  5. For password recovery issues, confirm /auth/recovery redirects to the configured Authentik recovery flow and that hosted Croox direct email/OTP smoke is healthy for SEQN-owned transactional mail.
  6. Ask the user to retry in a clean browser session.
  7. If callback errors continue, follow the OIDC client secret rotation runbook only when Authentik/API secret mismatch is confirmed.

Admin access missing

Symptoms:

  • /auth/console opens but admin actions are unavailable.
  • Admin API returns 403.

Actions:

  1. Confirm the user is signed in at /v1/me.
  2. Confirm the org membership role is admin for the target workspace.
  3. For platform Authentik resource proxy routes, confirm the user is in SEQN-A.
  4. Use an existing admin to update membership role when appropriate.
  5. Record the role change in the support ticket.

Revoke a user session

Use when a device is lost, a token is suspected exposed, or employment/access changes.

  1. Open the hosted console sessions view for the organization.
  2. Locate the user and active session.
  3. Revoke the session.
  4. Confirm the user must sign in again.
  5. Review audit logs for suspicious actions before and after the revocation.

Project config issue

Common causes:

  • Missing redirect URI.
  • Missing JavaScript origin.
  • Disabled application.
  • Backend using an old sk_live_ after rotation.

Actions:

  1. Load the project in the hosted console.
  2. Confirm status is active.
  3. Confirm redirect URLs and browser origins match the app environment exactly.
  4. Run the public config check with the project pk_live_.
  5. Run backend key health from the backend environment.
  6. Rotate the project secret only if exposure or stale backend config is confirmed.

Provisioning blocked

Symptoms:

  • Project create/update succeeds locally but Authentik provisioning status is blocked or failed.
  • Hosted app config exists, but Authentik provider/application was not created or updated.

Actions:

  1. Confirm AUTHENTIK_API_TOKEN is present on the VPS without printing it.
  2. Confirm Authentik API health.
  3. Retry a canary project update.
  4. If the token is invalid, rotate the Authentik API token.
  5. If Authentik rejects the payload, capture sanitized status, resource IDs, and timestamps.

Subscription blocked

Symptoms:

  • Mutation returns 402 subscription_inactive.
  • Mutation returns 402 subscription_limit_exceeded.

Actions:

  1. Check the organization subscription state in the hosted console.
  2. Confirm plan key, seat limit, current period dates, and status.
  3. If a limit is reached, compare current usage with the plan limits in Pricing.
  4. If the customer should be allowed through, update the manual subscription state or plan.
  5. Do not create checkout, invoice, or payment-provider workarounds in this MVP.

Webhook delivery issue

Symptoms:

  • Receiver reports invalid signature.
  • Delivery logs show retrying or failed events.
  • Test webhook does not arrive.

Actions:

  1. Confirm the endpoint URL is HTTPS and reachable.
  2. Confirm the endpoint is enabled and subscribed to the expected event types.
  3. Send a test webhook event.
  4. Check delivery response status and retry state.
  5. Rotate the webhook secret if the receiver secret is stale or exposed.
  6. Ask the receiver to respect retries and avoid side effects without idempotency.

Rate limited customer

Symptoms:

  • Customer receives 429 rate_limited.
  • Logs show repeated startup checks, tight retry loops, or high setup-agent traffic.

Actions:

  1. Confirm the route and whether the caller respects Retry-After.
  2. Check whether the customer is over a subscription limit instead.
  3. If legitimate traffic is blocked, tune the edge limiter for the specific route or source.
  4. Do not disable the app limiter globally during active abuse.
  5. Rotate secrets if traffic follows a single leaked key prefix.

Incident communication

  • Use non-email channels for MVP ops: Slack, Discord, status page, PagerDuty/Opsgenie, ntfy, or customer support tracker.
  • State impact, affected routes, start time, next update time, and mitigation.
  • Do not include raw secrets, customer PII beyond necessary account identifiers, or internal bearer tokens.
  • Close with root cause, customer impact, prevention item, and owner.