T10.9: rollback-aware deploy runbook for the Okta cutover

New DEPLOY-runbook-2026-09-03.md, separate from the 2026-08-04 runbook.
Names the five new OKTA_* env vars, treats the pre-deploy backup as the
only way back once 1d60a608bb51 (drop_local_password) commits since its
downgrade() restores the column but not the data, and splits Rollback
into the fixable case (Okta app integration misconfigured, fix and
redeploy api, no data at risk) versus the severe case (abandoning Okta
for local-password code, which only a destructive backup restore can
reach). States D16's no-break-glass posture plainly.

D17 records the decision and why: staged-deploy-sequence docs and
backlog-only were both considered and declined in favor of the runbook.

Logged to backlog.md rather than fixed here: okta_auth.describe() has
no caller (BL-028), users.failed_attempts/locked_until are vestigial
(BL-029), DEPLOY-login-portal.md is fully stale (BL-030).

docs/waves/backlog.md

@@ -594,3 +594,49 @@ deliberately deferred.
   and pinned; the systematic guard is its own small task, not a hotfix rider.
 - **Suggested wave or follow-up:** next housekeeping pass, paired with BL-026
   (both are "deploys should be boring" work).
+
+### BL-028 — `okta_auth.describe()` is never called
+
+- **Found during:** T10.9 (writing the deploy runbook's live-verification step)
+- **Where:** `server/okta_auth.py` (`describe()`), `server/app.py` (no caller anywhere)
+- **What:** `describe()` exists specifically to shout `"*** FAKE OKTA PROVIDER
+  ACTIVE..."` or report missing config at a glance, and both
+  `server/.env.example` and `server/README.md` tell an operator to "check the
+  startup log line" for it. Nothing prints it. `app.py` never imports or calls
+  `describe()` at process start, so that log line does not exist and an operator
+  following the docs will not find it.
+- **Why not now:** a runbook is documentation, not server code; wiring a
+  startup log call is a real (if small) change to `app.py` and wants its own
+  diff and its own verification, not a rider on T10.9.
+- **Suggested wave or follow-up:** next housekeeping pass. One call
+  (`logger.info(okta_auth.describe())` near startup) plus updating
+  `DEPLOY-runbook-2026-09-03.md` Step 4 to check the log line once it exists.
+
+### BL-029 — `users.failed_attempts` / `users.locked_until` are vestigial
+
+- **Found during:** T10.6
+- **Where:** `server/models.py` (`User.failed_attempts`, `User.locked_until`)
+- **What:** both columns exist to support local-password lockout, which T10.4
+  removed. They are still reset to `0`/`None` on every Okta sign-in but nothing
+  increments them anymore — dead columns, not a bug, but schema drift from the
+  D15 cutover.
+- **Why not now:** T10.6 is documentation scope; dropping columns is a migration
+  and belongs with the rest of the local-password cleanup, not folded into a
+  docs task.
+- **Suggested wave or follow-up:** next housekeeping pass, alongside any other
+  post-cutover schema tidy-up.
+
+### BL-030 — `DEPLOY-login-portal.md` is fully stale
+
+- **Found during:** T10.9
+- **Where:** `DEPLOY-login-portal.md` (repo root)
+- **What:** the original username/password login rollout doc. References
+  `bcrypt`, `create-admin --password`, `AUTH_SECRET_KEY` as the only secret,
+  and a login form — none of which describe the app since T10.4/T10.5. Someone
+  handed this to IT today would be told to do things that no longer work.
+- **Why not now:** no task currently owns deploy-doc cleanup as a category;
+  deleting or archiving a doc is a product/records call (`CLAUDE.md`'s "removed
+  fields are hidden, not deleted" spirit likely applies to docs too, but that is
+  worth confirming rather than assuming).
+- **Suggested wave or follow-up:** next housekeeping pass — needs Nick on
+  whether to delete, archive, or rewrite it as historical record.

docs/waves/decisions-2026-09-03.md

@@ -75,3 +75,48 @@ of scripting a login form. No live Okta tenant needed, and T10.4 no longer depen
 T10.7's timing.
 
 `wave-10.md`'s T10.4 bullet is updated to reflect this full scope.
+
+## D17: A dedicated rollback-aware runbook for the Okta cutover deploy
+
+Raised while preparing to close out wave 10: T10.4's migration
+(`server/alembic/versions/1d60a608bb51_drop_local_password.py`) drops the
+`password_hash` column, and its `downgrade()` re-adds the column with
+`server_default=''`. That restores the schema, not the data — the real bcrypt hashes
+are destroyed the moment `upgrade()`'s `op.drop_column` commits, and no amount of
+`alembic downgrade` brings them back. `DEPLOY-runbook-2026-08-04.md`, the existing
+precedent for how a deploy of this repo is handed to IT, has no equivalent case in its
+own Rollback section — its migrations are additive or reversible, so nothing there
+warns an operator that this one is different.
+
+Three options were on the table: write this runbook now; also document a staged
+deploy sequence (ship T10.5's Okta-live-alongside-password state first, verify real
+sign-ins, then ship T10.4's column drop as a separate follow-up deploy); or log the
+hazard to `backlog.md` and stop there. Decided: write the runbook only. Staged
+sequencing is real risk reduction but is a second deploy plan on top of a wave that is
+otherwise a single cutover (D15's "full replacement," not a toggle) — worth
+proposing on its own if IT wants it, not worth building unasked. Backlog-only was
+rejected because the hazard is concrete and dated (this wave, this migration), not a
+someday item.
+
+### What the runbook has to do differently from the 2026-08-04 precedent
+
+- Name the five `OKTA_*` environment variables as newly required for this deploy —
+  the precedent's own "no new environment variables" note does not apply here and
+  restating it unchanged would be actively wrong.
+- Treat the pre-deploy backup (`docker exec wp_db_backup /scripts/db-backup.sh`) as
+  the only way back once `1d60a608bb51` commits, not as routine due diligence.
+- Separate two failure modes that look similar but are not: an Okta app integration
+  that is misconfigured after a clean deploy (redirect URI, client secret, an
+  unassigned test account) is fixable in place — fix the config, redeploy the `api`
+  service, no data at risk, migration already applied and stays applied. Deciding to
+  abandon Okta and restore local-password code is the severe case — the empty
+  `password_hash` column means the old code has nothing to check a password against,
+  so the only way back is the destructive backup restore (`DEPLOY-runbook-2026-08-04.md`
+  Case C's own procedure, reused here). Conflating these two would send an operator
+  straight to a destructive restore for a problem that a config fix would have solved.
+- Fold in D16's no-break-glass posture: if Okta is down or misconfigured, the app is
+  down for everyone, including admins, by design — not a defect to roll back from.
+
+Filed as **T10.9** in `wave-10.md` rather than folded into T10.6 (already merged and
+verified) or T10.8 (UI/test verification, a different kind of check). New scope found
+after the task that raised it closed gets a new ID, per `CLAUDE.md`.

docs/waves/wave-10.md

@@ -120,6 +120,31 @@ Depends only on `main` as it stands after `D15`. Not sequenced behind any other
 - **T10.8 — Verification.** 390px and 1440px, full suite, done-when checks per task,
   matching the rigor D13 was held to.
 
+- **T10.9 — Rollback-aware deploy runbook.** Raised after hazard review found
+  `DEPLOY-runbook-2026-08-04.md`'s Rollback section has no case for a migration whose
+  `downgrade()` cannot restore the data it drops — see `D17`. `T10.4`'s
+  `1d60a608bb51_drop_local_password` is exactly that: the schema comes back, the
+  bcrypt hashes do not.
+
+  Built: `DEPLOY-runbook-2026-09-03.md`, following the 2026-08-04 runbook's structure
+  (fill-in table, numbered deploy steps, case-by-case Rollback section, Notes). Names
+  the five `OKTA_*` vars as newly required (the precedent's "no new environment
+  variables" note does not carry over), treats the pre-deploy backup as the only way
+  back once the migration commits, and splits Rollback into the fixable case (Okta app
+  integration misconfigured — fix and redeploy `api`, no data at risk, migration stays
+  applied) versus the severe case (abandoning Okta for local-password code — only the
+  destructive backup restore gets there, reusing the 2026-08-04 runbook's own Case C
+  procedure). D16's no-break-glass posture is stated plainly rather than left implicit.
+
+  Logged, not fixed here (out of scope): `okta_auth.describe()`'s startup log line
+  (referenced by `DEPLOYMENT.md`/`server/README.md`) has no caller anywhere in
+  `server/app.py` — nothing actually prints it at process start. The runbook's Step 4
+  therefore verifies via a live Okta sign-in rather than a log line, and this gap is
+  flagged in `docs/waves/backlog.md` as a candidate fix (wiring `describe()` into
+  startup) since it directly bears on deploy verifiability. `DEPLOY-login-portal.md`
+  is now fully stale (bcrypt, `create-admin --password`, none of which still exist) —
+  not touched, no task claims it.
+
 ## Still open
 
 - The OIDC claim mapping (`T10.3`).