T10.9: rollback-aware deploy runbook for the Okta cutover
New DEPLOY-runbook-2026-09-03.md, separate from the 2026-08-04 runbook. Names the five new OKTA_* env vars, treats the pre-deploy backup as the only way back once 1d60a608bb51 (drop_local_password) commits since its downgrade() restores the column but not the data, and splits Rollback into the fixable case (Okta app integration misconfigured, fix and redeploy api, no data at risk) versus the severe case (abandoning Okta for local-password code, which only a destructive backup restore can reach). States D16's no-break-glass posture plainly. D17 records the decision and why: staged-deploy-sequence docs and backlog-only were both considered and declined in favor of the runbook. Logged to backlog.md rather than fixed here: okta_auth.describe() has no caller (BL-028), users.failed_attempts/locked_until are vestigial (BL-029), DEPLOY-login-portal.md is fully stale (BL-030).
This commit is contained in:
@@ -75,3 +75,48 @@ of scripting a login form. No live Okta tenant needed, and T10.4 no longer depen
|
||||
T10.7's timing.
|
||||
|
||||
`wave-10.md`'s T10.4 bullet is updated to reflect this full scope.
|
||||
|
||||
## D17: A dedicated rollback-aware runbook for the Okta cutover deploy
|
||||
|
||||
Raised while preparing to close out wave 10: T10.4's migration
|
||||
(`server/alembic/versions/1d60a608bb51_drop_local_password.py`) drops the
|
||||
`password_hash` column, and its `downgrade()` re-adds the column with
|
||||
`server_default=''`. That restores the schema, not the data — the real bcrypt hashes
|
||||
are destroyed the moment `upgrade()`'s `op.drop_column` commits, and no amount of
|
||||
`alembic downgrade` brings them back. `DEPLOY-runbook-2026-08-04.md`, the existing
|
||||
precedent for how a deploy of this repo is handed to IT, has no equivalent case in its
|
||||
own Rollback section — its migrations are additive or reversible, so nothing there
|
||||
warns an operator that this one is different.
|
||||
|
||||
Three options were on the table: write this runbook now; also document a staged
|
||||
deploy sequence (ship T10.5's Okta-live-alongside-password state first, verify real
|
||||
sign-ins, then ship T10.4's column drop as a separate follow-up deploy); or log the
|
||||
hazard to `backlog.md` and stop there. Decided: write the runbook only. Staged
|
||||
sequencing is real risk reduction but is a second deploy plan on top of a wave that is
|
||||
otherwise a single cutover (D15's "full replacement," not a toggle) — worth
|
||||
proposing on its own if IT wants it, not worth building unasked. Backlog-only was
|
||||
rejected because the hazard is concrete and dated (this wave, this migration), not a
|
||||
someday item.
|
||||
|
||||
### What the runbook has to do differently from the 2026-08-04 precedent
|
||||
|
||||
- Name the five `OKTA_*` environment variables as newly required for this deploy —
|
||||
the precedent's own "no new environment variables" note does not apply here and
|
||||
restating it unchanged would be actively wrong.
|
||||
- Treat the pre-deploy backup (`docker exec wp_db_backup /scripts/db-backup.sh`) as
|
||||
the only way back once `1d60a608bb51` commits, not as routine due diligence.
|
||||
- Separate two failure modes that look similar but are not: an Okta app integration
|
||||
that is misconfigured after a clean deploy (redirect URI, client secret, an
|
||||
unassigned test account) is fixable in place — fix the config, redeploy the `api`
|
||||
service, no data at risk, migration already applied and stays applied. Deciding to
|
||||
abandon Okta and restore local-password code is the severe case — the empty
|
||||
`password_hash` column means the old code has nothing to check a password against,
|
||||
so the only way back is the destructive backup restore (`DEPLOY-runbook-2026-08-04.md`
|
||||
Case C's own procedure, reused here). Conflating these two would send an operator
|
||||
straight to a destructive restore for a problem that a config fix would have solved.
|
||||
- Fold in D16's no-break-glass posture: if Okta is down or misconfigured, the app is
|
||||
down for everyone, including admins, by design — not a defect to roll back from.
|
||||
|
||||
Filed as **T10.9** in `wave-10.md` rather than folded into T10.6 (already merged and
|
||||
verified) or T10.8 (UI/test verification, a different kind of check). New scope found
|
||||
after the task that raised it closed gets a new ID, per `CLAUDE.md`.
|
||||
|
||||
Reference in New Issue
Block a user