From 75ac930d0c88f1e3d99358ab823dc5cf628ad3f6 Mon Sep 17 00:00:00 2001 From: Matt Mabrey Date: Fri, 18 Sep 2026 09:21:29 -0700 Subject: [PATCH 01/20] waves 11-13: planning docs for CR-019, CR-020, D18 CR-019 - usage/activity metrics (admin console), wave 11 CR-020 - bulk editing of users, wave 12 D18 - Okta/AD deprovisioning detection and auto-disable, wave 13 Raised by Matt Mabrey 2026-09-17. Decision record and task breakdowns only in this commit - no feature code yet. --- docs/waves/decisions-2026-09-17.md | 327 +++++++++++++++++++++++++++++ docs/waves/wave-11.md | 223 ++++++++++++++++++++ docs/waves/wave-12.md | 193 +++++++++++++++++ docs/waves/wave-13.md | 200 ++++++++++++++++++ 4 files changed, 943 insertions(+) create mode 100644 docs/waves/decisions-2026-09-17.md create mode 100644 docs/waves/wave-11.md create mode 100644 docs/waves/wave-12.md create mode 100644 docs/waves/wave-13.md diff --git a/docs/waves/decisions-2026-09-17.md b/docs/waves/decisions-2026-09-17.md new file mode 100644 index 0000000..6f5aaeb --- /dev/null +++ b/docs/waves/decisions-2026-09-17.md @@ -0,0 +1,327 @@ +# Decisions — September 17, 2026 + +Three items. Like `D11`, `D15`, `D16` and `D17`, these are new scope raised after +R2 closed out (see `docs/reference/completion.md`, T9.7), not a reopening of +anything already decided there. + +`CR-019` and `CR-020` get `CR` ids because they are field/product-facing feature +requests — the same kind of thing `CR-001`-`CR-018` were — not internal +engineering calls made mid-build. `D18` gets a `D` id because it is exactly that: +an internal security/architecture call, the same category as `D15` (Okta vs. +LDAPS), not a feature a user asked for. + +Requested/raised by Matt Mabrey, 2026-09-17. + +--- + +## CR-019 — Usage and activity metrics (admin console) + +- **Area:** Admin Console / Monitoring +- **Priority:** proposed High — the current admin console has no reliable way to + answer "who is using this and how much," which is the same visibility gap the + original UX review found in `B4` (numbers that look authoritative but are not). +- **Source:** Matt Mabrey, 2026-09-17. + +### Why this is not already built + +`D5`/`T7.10` already shipped a usage-analytics feature — `html/wp-usage.js`, with +a report at `admin.js:666-699` — but it reads `localStorage`, which is scoped to +one browser. The report's own empty state says exactly this: *"No usage recorded +in this browser yet."* It cannot show who across the team is active or what they +use, because each person's activity exists only on their own machine. This is the +same class of defect `B4` named for the pipeline strip — a number that looks +authoritative but is not — just never generalized to this feature. `CR-019` is +therefore new server-side work, not a UI addition on top of what exists. + +Two things already in the schema are relevant and reused rather than duplicated: +`User.last_login_at` (one timestamp, no history) and `AuditLog` (append-only, +already records business mutations — WP created, status changed, role changed — +per user, with a timestamp). Neither captures navigation or feature-open events, +which is the actual gap. + +### Intent + +Give admins a real, server-backed picture of who is using the suite, what parts +of it they use, and how active they are — replacing the per-browser report as +the thing anyone actually looks at. + +### Decisions, made 2026-09-17 + +1. **Retention: indefinite.** No automatic purge of usage-event data. (Separate + from `AuditLog`'s own retention, which this does not change.) +2. **Scope: suite-wide, filterable.** Not per-project by default; the console + provides filters (date range, project, user, tool/page) rather than scoping + the data itself. +3. **Export: raw view shows real identities; export supports sanitization.** + The admin console's own tables and the CSV export both default to real + usernames — this is an internal audit tool, not a public one. But the export + also offers a "sanitize" toggle that replaces the actor with a **stable + pseudonymous id** (a per-user hash, consistent across rows and across + export runs) rather than dropping the identity field outright — so an + external system (Power BI or similar) can still group and trend "by user" + without ever receiving a real name. Flagged here as the recommended + approach rather than the only one Matt confirmed in so many words: if a + fully-anonymous (no stable id at all) export turns out to be what's + actually wanted, that is a one-line change to the same feature, raise it + in the PR rather than treating it as blocking. +4. **The old per-browser report is retired**, not kept alongside the new one. + Once `CR-019` ships, `admin.js`'s existing `usage-admin` panel (reading + `WPUsage.load(...)` per browser) is removed rather than left next to the + real report, where it would show a smaller, misleading number for whoever + happens to have it open. `html/wp-usage.js` and its two call sites + (`html/work-package-suite-app.js`'s wizard dwell-tracking, and the + creator's equivalent) are a separate question — the *recording* code can + stay or go independent of the *admin report* being retired, since dwell + events were never reliably tied to a real identity anyway. Default to + removing both unless a task finds a reason to keep the recorder; log that + reason rather than deciding it here. + +### Acceptance criteria + +- A new admin-only tab in `admin.html` (same role gate as the User Directory) + shows: active users over a selectable date range (day/week/month), each + user's last-active timestamp, and a breakdown of which tools/pages get + opened and how often. +- Server-side event capture, keyed to the authenticated session (real identity, + not a browser-local guess) — a new table, not an extension of `AuditLog`, + since page-open/navigation events are not business mutations and mixing them + in would make `AuditLog` noisy for its existing, narrower purpose. +- The console's filters cover date range, project, user, and tool/page, and + combine (e.g., "user X, last 30 days, field view only"). +- CSV export from the console, in both raw (real usernames) and sanitized + (stable pseudonymous id per user) modes. +- Retention is indefinite; nothing in this item purges data. +- View-only, refresh-on-load. No alerting — matches how the rest of the admin + console works today; if that changes later it is new scope, not a rider on + this item. +- The old per-browser "Usage" report and its admin-console panel are removed + in the same wave, not left running alongside the new one. +- Accessible per `CLAUDE.md`'s standing `C1` rules (this is a new component, + not a legacy one — it ships accessible or it is not done, same as + everything else built since wave 7). + +### Frontend/backend boundary + +This needs server work, the same way `CR-004`/`CR-018`/`B4` did: a real table, +a capture endpoint, an aggregation endpoint, and an export endpoint. If a task +under this item is being built by writing to `localStorage`, it is rebuilding +the exact defect this item exists to replace — stop and say so, per +`CLAUDE.md`. + +### Scheduling + +New wave. Wave 10 (Okta) is merged, so this does not wait on anything. +Task breakdown: `docs/waves/wave-11.md`. + +--- + +## CR-020 — Bulk editing of users in the admin console + +- **Area:** Admin Console / User Directory +- **Priority:** proposed High — three of the four actions below touch access + control (role, project assignment, active/disabled) and the fourth is a hard + delete; getting the guardrails right matters more than getting it built fast. +- **Source:** Matt Mabrey, 2026-09-17. + +### Why this is not already built + +Every user-editing action in `html/users.js` today is one row, one action: +a role `` controls, keyboard-reachable, `aria-live` on any +count that updates without a page reload, focus visible throughout. + +**Done when:** + +- [ ] the tab is reachable only by an admin (verify: a non-admin session gets + no tab and the route itself refuses the request server-side, not just a + hidden tab) +- [ ] every filter is a real form control, keyboard-operable +- [ ] both export buttons produce the files T11.4 defines +- [ ] works at 390px and 1440px + +--- + +### T11.6 — CR-019: retire the per-browser Usage report + +- **Items:** `CR-019` +- **Depends on:** T11.5 +- **Blocks:** T11.7 +- **Surface:** `html/` +- **Files:** `html/admin.js` (the `usage-admin` panel, `admin.js:666-699`), + `html/wp-usage.js` and its two call sites + +**Do:** Remove the old per-browser `usage-admin` panel from `admin.js` now that +the real one exists, per the 2026-09-17 decision. Decide what happens to +`wp-usage.js`'s recording calls (wizard dwell-tracking, the creator's +equivalent): they were never reliably tied to a real identity, so they are not +a data source the new report can adopt. Default to removing the recorder too +unless it is still doing something useful on its own (re-read what it actually +records before deciding — do not assume from this file alone). + +**Do not:** leave the old panel in place "just in case." Two activity reports +showing two different numbers is worse than one. + +**Done when:** + +- [ ] the old `usage-admin` panel and its markup are gone from `admin.html`/ + `admin.js` +- [ ] a decision on `wp-usage.js` itself is recorded (removed, or kept with a + stated reason) — not left ambiguous +- [ ] nothing else in the app references the removed code; grep confirms + +--- + +### T11.7 — CR-019: verification + +- **Items:** `CR-019` +- **Depends on:** T11.6 +- **Blocks:** nothing +- **Surface:** `html/` + `server/` +- **Files:** as touched above + +**Do:** Full verification per `CLAUDE.md`: run the app locally, exercise the +new tab at 390px and 1440px, before/after screenshots, run the existing smoke +test and `seed_demo.py`, run the full suite. + +**Done when:** + +- [ ] all `CR-019` acceptance criteria in `decisions-2026-09-17.md` are met or + a failure is stated with a reason +- [ ] screenshots committed +- [ ] smoke test and `seed_demo.py` both still pass +- [ ] full test suite passes + +--- + +## Wave 11 exit criteria + +- [ ] real, server-side activity data exists per user, indefinitely retained +- [ ] the admin console shows it, filterable by date/project/user/tool +- [ ] export works in both raw and sanitized form +- [ ] the old per-browser report is gone, not duplicated +- [ ] `CR-019` fully accounted for, no open acceptance criteria diff --git a/docs/waves/wave-12.md b/docs/waves/wave-12.md new file mode 100644 index 0000000..d796392 --- /dev/null +++ b/docs/waves/wave-12.md @@ -0,0 +1,193 @@ +# Wave 12 — Bulk editing of users + +**Items:** `CR-020` +**Depends on:** wave 10 merged (it is). Independent of wave 11 (`CR-019`) — no +shared files, may build in either order or in parallel. +**Decision record:** `docs/waves/decisions-2026-09-17.md` + +Six tasks, in build order. + +--- + +### T12.1 — CR-020: bulk endpoints + +- **Items:** `CR-020` +- **Depends on:** nothing (first task) +- **Blocks:** T12.2, T12.4 +- **Surface:** `server/` +- **Files:** `server/app.py` + +**Do:** Add bulk variants of the four existing single-user actions — role +change, active/disabled, project assignment (add/remove + project-role), and +delete. Each takes a list of `user_id`s plus the action's parameters and +applies `require_see_user`/`require_manage_user`/`grantable_roles` **per row**, +exactly as the single-user endpoint does today — a super user's bulk request +cannot reach further than their existing single-user requests can. Do not skip +the self-action guard: an actor's own account is rejected out of any batch +that would disable, demote, or delete it, same as today. + +Each successful row writes its own `AuditLog` entry via `log_event`, same +action names the single endpoints already use. A row that fails is reported in +the response (user id, reason) and does not stop the rest of the batch from +being attempted. + +**Do not:** invent a single opaque "bulk_action" audit entry in place of the +per-row entries. Do not build a soft-delete path for bulk delete that doesn't +exist for the single case — bulk delete stays a hard delete, matching +`delete_user` today. + +**Done when:** + +- [ ] each of the four bulk actions is callable with a list of user ids and a + single set of parameters +- [ ] a batch containing the actor's own account rejects only that row, not + the whole batch — verified for disable, demote, and delete +- [ ] a super user's batch that includes a user/project outside what they + manage rejects only that row, with a stated reason +- [ ] every successful row produces its own `AuditLog` entry, identical in + shape to what the single-user endpoint would have written +- [ ] a batch with some failing rows still applies to the rows that succeed, + and the response lists exactly which rows failed and why + +--- + +### T12.2 — CR-020: row selection in the User Directory table + +- **Items:** `CR-020` +- **Depends on:** T12.1 +- **Blocks:** T12.3 +- **Surface:** `html/` +- **Files:** `html/users.js`, `html/users.html` + +**Do:** Add a checkbox per row and a select-all control, respecting whatever +filter (`role`, `active`/`disabled`, project) is currently applied — select-all +selects the filtered set, not every user in the system regardless of what's +shown. A bulk-action toolbar appears once at least one row is checked and +disappears at zero. + +**Done when:** + +- [ ] select-all selects exactly the rows currently visible under the active + filter, not the full unfiltered table +- [ ] changing the filter while rows are selected does something sane and + visible (either clears the selection or keeps it explicit which rows are + still selected) — pick one and state it, don't leave it undefined +- [ ] the toolbar is keyboard-reachable and only present when >=1 row is + selected + +--- + +### T12.3 — CR-020: bulk-action toolbar + +- **Items:** `CR-020` +- **Depends on:** T12.2 +- **Blocks:** T12.5 +- **Surface:** `html/` +- **Files:** `html/users.js` + +**Do:** Wire the toolbar to T12.1's endpoints for role change, activate/ +disable, and project assignment (add to / remove from a project + project- +role). Confirmation before applying uses the `wp-dialog` kit (`T7.9`) — +`wpConfirmDialog`, not a native `confirm()` — naming exactly how many users are +affected. On completion, report per-row results if anything failed (T12.1 +already returns this) rather than a single success/failure toast that hides a +partial failure. + +**Delete is built separately, in T12.5** — do not wire delete here. + +**Done when:** + +- [ ] role change, activate/disable, and project assignment each work end to + end against a multi-row selection +- [ ] the confirmation dialog names the exact affected count before anything is + sent +- [ ] a batch with a partial failure shows which rows failed, not just an + undifferentiated error +- [ ] `aria-live` announces the outcome + +--- + +### T12.4 — CR-020: CSV upload path + +- **Items:** `CR-020` +- **Depends on:** T12.1 +- **Blocks:** T12.6 +- **Surface:** `html/` + `server/` +- **Files:** `html/users.js`, `server/app.py` + +**Do:** An upload accepting a list of usernames plus the action to apply, +following the validate-and-report pattern `CR-005` established: reject and +report bad rows (username not found, actor lacks permission over that user) +rather than silently dropping them. This is a second entry point onto the same +T12.1 endpoints, not a third implementation of the bulk logic. + +**Done when:** + +- [ ] a CSV with a mix of valid and invalid usernames applies to the valid + rows and reports the invalid ones by row, with a reason +- [ ] the same permission/self-action guards from T12.1 apply here — a CSV + cannot reach a user a checkbox-driven batch couldn't +- [ ] duplicate usernames in one CSV are handled without double-applying or + erroring confusingly + +--- + +### T12.5 — CR-020: bulk delete confirmation + +- **Items:** `CR-020` +- **Depends on:** T12.3 +- **Blocks:** T12.6 +- **Surface:** `html/` +- **Files:** `html/users.js` + +**Do:** Wire delete into the toolbar with a heavier confirmation than the other +three actions, per the decision record's recommendation: list the affected +usernames and require typing a confirmation phrase (e.g. `DELETE`) before the +request is sent, regardless of how many rows are selected. This is flagged in +the decision record as a recommendation Matt has not explicitly signed off on +— if the PR reviewer wants a lighter or heavier mechanism, that's the moment to +change it, not a reason to skip building a real confirmation now. + +**Done when:** + +- [ ] the affected usernames are listed in the confirmation dialog before + delete is sent +- [ ] the request is not sent until the confirmation phrase is typed correctly +- [ ] the actor's own account, if somehow selected, is rejected with a clear + reason rather than silently included or silently dropped + +--- + +### T12.6 — CR-020: verification + +- **Items:** `CR-020` +- **Depends on:** T12.4, T12.5 +- **Blocks:** nothing +- **Surface:** `html/` + `server/` +- **Files:** as touched above + +**Do:** Full verification per `CLAUDE.md`: run locally, exercise bulk role +change, activate/disable, project assignment, CSV upload, and bulk delete at +390px and 1440px, before/after screenshots, smoke test, `seed_demo.py`, full +suite. + +**Done when:** + +- [ ] all `CR-020` acceptance criteria in `decisions-2026-09-17.md` are met or + a failure is stated with a reason +- [ ] screenshots committed +- [ ] smoke test and `seed_demo.py` both still pass +- [ ] full test suite passes + +--- + +## Wave 12 exit criteria + +- [ ] all four bulk actions work against both a checkbox selection and a CSV + upload +- [ ] every existing single-user guardrail (self-action, scope, grantable + roles) holds under bulk use +- [ ] bulk delete requires typed confirmation and lists affected usernames +- [ ] partial failures are always reported, never hidden behind a blanket + success +- [ ] `CR-020` fully accounted for, no open acceptance criteria diff --git a/docs/waves/wave-13.md b/docs/waves/wave-13.md new file mode 100644 index 0000000..47cb05e --- /dev/null +++ b/docs/waves/wave-13.md @@ -0,0 +1,200 @@ +# Wave 13 — Okta/AD deprovisioning sync + +**Items:** `D18` +**Depends on:** wave 10 merged (it is). Not blocked by wave 11 or wave 12, but +shares `server/app.py` account-state surface with wave 12 (`CR-020`) — sequence +commits to avoid an avoidable conflict. +**Decision record:** `docs/waves/decisions-2026-09-17.md` + +Six tasks, in build order. **T13.3's fail-closed behavior is the most +important done-when list in this wave — do not relax it to ship faster.** + +--- + +### T13.1 — D18: shrink `AUTH_SESSION_HOURS` + +- **Items:** `D18` +- **Depends on:** nothing (first task, independent of the rest) +- **Blocks:** nothing +- **Surface:** `server/` +- **Files:** `server/auth.py`, `server/.env.example`, `DEPLOYMENT.md` + +**Do:** Change the default from 12 to 2 hours. Document the change and the +reasoning (defense in depth against an already-live session outliving a +deprovisioning event) in `server/.env.example` next to `AUTH_SESSION_HOURS`, +and in `DEPLOYMENT.md`'s auth section. Note in the PR that this is proposed, +not confirmed against the tenant's actual Okta SSO session policy — flag it +for Nick/IT rather than treating 2 as unquestionable. + +**Done when:** + +- [ ] default is 2 hours; still overridable via env, unchanged mechanism +- [ ] `.env.example` and `DEPLOYMENT.md` explain why +- [ ] existing session/auth tests updated for the new default where they + assumed 12 + +--- + +### T13.2 — D18: Okta Management API client + +- **Items:** `D18` +- **Depends on:** nothing (independent of T13.1) +- **Blocks:** T13.3 +- **Surface:** `server/` +- **Files:** new `server/okta_sync.py` (or extend `server/okta_auth.py` — + task's call), `server/.env.example` + +**Do:** A small client for Okta's user-list endpoint, authenticated with a new, +separate credential (e.g. `OKTA_API_TOKEN`) — not the OIDC client secret used +for sign-in. Fetch the full user list (paginated per Okta's API) rather than +one-by-one lookups per local user; this app has ~20-odd accounts today, and a +list-and-diff is simpler and cheaper than N calls. Return each Okta user's +identity-claim value (matching `OKTA_IDENTITY_CLAIM`, already confirmed live +per wave 10) and status. + +Follow the existing pattern for external credentials in this repo (`MICRON_DB_URL`, +`SMTP_PASSWORD`): env-only, never logged, never returned to the browser in an +error message. + +**Do not:** reuse `OKTA_CLIENT_ID`/`OKTA_CLIENT_SECRET` for this. Sign-in and +the management API are different trust boundaries with different scopes; +conflating them means a compromise or rotation of one affects the other +unnecessarily. + +**Done when:** + +- [ ] the client authenticates with its own credential, distinct from the OIDC + client +- [ ] it fetches the complete Okta user list, handling pagination +- [ ] an auth failure or malformed response raises a clear, specific error + rather than returning an empty list indistinguishable from "everyone was + deprovisioned" — this distinction is what T13.3 depends on +- [ ] the credential is never logged or surfaced in any API response + +--- + +### T13.3 — D18: the sync job + +- **Items:** `D18` +- **Depends on:** T13.2 +- **Blocks:** T13.4 +- **Surface:** `server/` +- **Files:** `server/okta_sync.py`, `server/app.py` (or wherever `log_event` + lives) + +**Do:** Compare the Okta user list (T13.2) against local `users` rows. For any +local `is_active=True` user whose Okta identity is missing from the list, or +present with a non-active status, set `is_active=False` and write an +`AuditLog` row (`actor="system:okta_sync"`, action e.g. +`user_deprovisioned_by_sync`, detail naming the Okta status found). Never +touch a user already `is_active=False`. Never re-enable anyone. + +**The fail-closed rule, non-negotiable:** if T13.2's client raises an error of +any kind (network, auth, malformed response, timeout), this task takes **no +action on any account** for that run and logs the failure clearly (server log, +at minimum). An error is never treated as "Okta returned zero active users." +Write a test that asserts this directly: feed the sync a failing client and +assert zero rows changed and zero `AuditLog` entries written. + +**Done when:** + +- [ ] a user missing from Okta's list, or present but not active, is disabled + with a correctly-detailed audit row +- [ ] a user already disabled is left alone (no duplicate audit row each run) +- [ ] an active Okta user already active locally produces no audit row (only + changes are logged, not a clean bill of health every cycle) +- [ ] **a simulated Okta API failure results in zero account changes and zero + audit rows** — this is the one check that must never be skipped or + weakened +- [ ] the job never re-enables an account under any input + +--- + +### T13.4 — D18: run it on a schedule + +- **Items:** `D18` +- **Depends on:** T13.3 +- **Blocks:** T13.6 +- **Surface:** `server/`, `docker-compose.yml` +- **Files:** `server/app.py` (startup hook) or a new sidecar per the `backup` + container's pattern — task's call, per the decision record's noted + alternative + +**Do:** Wire T13.3 to run on an interval (proposed 15 minutes, env-overridable +— e.g. `OKTA_SYNC_INTERVAL_SECONDS`, matching `BACKUP_INTERVAL_SECONDS`'s +naming). Default choice is an in-process background task in the `api` +container; if a separate container is chosen instead, follow the `backup` +service's shape (own Dockerfile or script, `internal` network plus whatever +egress reaching Okta requires — check whether `outbound` as currently defined +is sufficient or Okta needs a distinct allowance). + +**Done when:** + +- [ ] the job runs automatically on the configured interval without manual + invocation +- [ ] interval is env-configurable with a sane default +- [ ] a container restart does not produce a duplicate/overlapping run, and a + slow cycle does not stack with the next one + +--- + +### T13.5 — D18: admin visibility + +- **Items:** `D18` +- **Depends on:** T13.3 +- **Blocks:** T13.6 +- **Surface:** `html/` +- **Files:** `html/users.js` / `html/admin.js` (wherever audit history is + already surfaced) + +**Do:** Confirm an auto-disable reads clearly wherever admins already look at +account history — the actor string (`system:okta_sync`) should be +self-explanatory in context, not require reading server logs to understand. +Do not build new UI beyond making sure the existing audit surface renders this +actor sensibly. Email/notification to admins on auto-disable is a noted +fast-follow (the decision record flags it as open, not required here) — do not +build it in this task; log it instead if it's tempting to add. + +**Done when:** + +- [ ] an auto-disabled account's audit entry is visible and legible in the + existing admin UI without special-casing +- [ ] nothing here silently assumes `CR-019`'s activity view exists yet — this + must work standalone + +--- + +### T13.6 — D18: verification + +- **Items:** `D18` +- **Depends on:** T13.4, T13.5 +- **Blocks:** nothing +- **Surface:** `server/` +- **Files:** as touched above + +**Do:** Full verification per `CLAUDE.md`. Beyond the usual run: specifically +re-run T13.3's fail-closed test in isolation and confirm it still passes after +T13.4's scheduling wrapper is in place — the scheduling layer must not +introduce a path that swallows the client's error and proceeds anyway. + +**Done when:** + +- [ ] all `D18` acceptance criteria in `decisions-2026-09-17.md` are met or a + failure is stated with a reason +- [ ] the fail-closed behavior is verified end to end through the scheduled + wrapper, not just the bare sync function +- [ ] smoke test and `seed_demo.py` both still pass +- [ ] full test suite passes + +--- + +## Wave 13 exit criteria + +- [ ] `AUTH_SESSION_HOURS` defaults to 2, documented +- [ ] the sync job runs on a schedule and correctly disables accounts Okta no + longer shows as active +- [ ] every auto-disable is individually audited with a clearly non-human actor +- [ ] an Okta API failure of any kind changes zero accounts — verified, not + assumed +- [ ] the sync never re-enables an account +- [ ] `D18` fully accounted for, no open acceptance criteria From 0652fa732dfe1f18227fca977297eef4da235bb8 Mon Sep 17 00:00:00 2001 From: Matt Mabrey Date: Wed, 23 Sep 2026 11:23:14 -0700 Subject: [PATCH 02/20] T11.1: UsageEvent model + migration (CR-019) New usage_events table, separate from audit_log - see the model docstring for why. Verified: applies and downgrades cleanly on SQLite, and the postgresql-dialect --sql render has no risky defaults (the BL-027 class of defect). No app wiring yet - that's T11.2. --- .../versions/8e2cb3003f8a_usage_events.py | 46 +++++++++++++++++++ server/models.py | 39 ++++++++++++++++ 2 files changed, 85 insertions(+) create mode 100644 server/alembic/versions/8e2cb3003f8a_usage_events.py diff --git a/server/alembic/versions/8e2cb3003f8a_usage_events.py b/server/alembic/versions/8e2cb3003f8a_usage_events.py new file mode 100644 index 0000000..88f407f --- /dev/null +++ b/server/alembic/versions/8e2cb3003f8a_usage_events.py @@ -0,0 +1,46 @@ +"""usage events (CR-019, wave 11) + +Append-only navigation/session activity, separate from audit_log on purpose — +see the UsageEvent docstring in server/models.py. Retention is indefinite by +decision (docs/waves/decisions-2026-09-17.md); nothing here schedules a purge. + +Revision ID: 8e2cb3003f8a +Revises: 1d60a608bb51 +Create Date: 2026-09-23 00:00:00.000000 +""" +from alembic import op +import sqlalchemy as sa + + +# revision identifiers, used by Alembic. +revision = '8e2cb3003f8a' +down_revision = '1d60a608bb51' +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.create_table('usage_events', + sa.Column('id', sa.String(length=40), nullable=False), + sa.Column('at', sa.DateTime(timezone=True), nullable=False), + sa.Column('username', sa.String(length=200), nullable=False), + sa.Column('project_id', sa.String(length=40), nullable=True), + sa.Column('tool', sa.String(length=40), nullable=False), + sa.Column('event', sa.String(length=40), nullable=False), + sa.Column('detail', sa.JSON(), nullable=False), + sa.PrimaryKeyConstraint('id') + ) + op.create_index(op.f('ix_usage_events_at'), 'usage_events', ['at'], unique=False) + op.create_index(op.f('ix_usage_events_username'), 'usage_events', ['username'], unique=False) + op.create_index(op.f('ix_usage_events_project_id'), 'usage_events', ['project_id'], unique=False) + op.create_index(op.f('ix_usage_events_tool'), 'usage_events', ['tool'], unique=False) + op.create_index(op.f('ix_usage_events_event'), 'usage_events', ['event'], unique=False) + + +def downgrade() -> None: + op.drop_index(op.f('ix_usage_events_event'), table_name='usage_events') + op.drop_index(op.f('ix_usage_events_tool'), table_name='usage_events') + op.drop_index(op.f('ix_usage_events_project_id'), table_name='usage_events') + op.drop_index(op.f('ix_usage_events_username'), table_name='usage_events') + op.drop_index(op.f('ix_usage_events_at'), table_name='usage_events') + op.drop_table('usage_events') diff --git a/server/models.py b/server/models.py index 05d1b19..6e02226 100644 --- a/server/models.py +++ b/server/models.py @@ -337,6 +337,45 @@ class AuditLog(Base): } +class UsageEvent(Base): + """CR-019: append-only record of who used the suite, when, and which tool — + navigation/session activity, not business mutations. Deliberately a SEPARATE + table from AuditLog rather than a new `action` value there: AuditLog answers + "who changed what" and is read by people auditing a specific record's + history; mixing in a `page_open` row for every authenticated page load + would make that trail noisy for its existing purpose. This table answers a + different question — "who is active, and on what" — and CR-019's admin + console reads from here, not from AuditLog. + + Not a ForeignKey to `users`, matching AuditLog's own reasoning: a user who + is later removed should still show up in historical activity rather than + silently vanishing from it, and `D18`'s deprovisioning sync only ever sets + `is_active=False` — it never deletes a row — so this is defensive symmetry + rather than a live concern today. + + Retention is indefinite (decided 2026-09-17, `decisions-2026-09-17.md`) — + nothing purges rows written here; that is a deliberate product decision, + not an oversight to fix later.""" + __tablename__ = "usage_events" + + id: Mapped[str] = mapped_column(String(40), primary_key=True) + at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True) + username: Mapped[str] = mapped_column(String(200), default="", index=True) + project_id: Mapped[Optional[str]] = mapped_column(String(40), nullable=True, index=True) + # creator | wizard | field_view | dashboard | admin | directory | ... + tool: Mapped[str] = mapped_column(String(40), default="", index=True) + # page_open | login + event: Mapped[str] = mapped_column(String(40), default="", index=True) + detail: Mapped[dict] = mapped_column(JSON, default=dict) + + def to_dict(self) -> dict: + return { + "id": self.id, "at": _iso(self.at), "username": self.username, + "project_id": self.project_id, "tool": self.tool, "event": self.event, + "detail": self.detail or {}, + } + + class AppSetting(Base): """Admin-editable application settings (feature flags, SMTP config, …) stored as key -> JSON value. Read/written via /api/settings (admin only). Secrets like From 850b78972b22cbfbf16b98bf09ba494c6150ebcf Mon Sep 17 00:00:00 2001 From: Matt Mabrey Date: Wed, 23 Sep 2026 11:24:49 -0700 Subject: [PATCH 03/20] T11.2: capture usage events (CR-019) POST /api/usage/ping writes one page_open UsageEvent per authenticated page load, identity from the session (get_current_user), never from the client. Wired from exactly one place - auth-guard.js's proceed(), after wp-auth-ready - so this can't drift into six separate per-page copies. okta_callback now also writes one login event per sign-in. Verified locally: unauthenticated ping -> 401; a real fake-Okta sign-in writes exactly one login row and one page_open row, no duplicates. --- html/auth-guard.js | 28 ++++++++++++++++++++++++++++ server/app.py | 28 ++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/html/auth-guard.js b/html/auth-guard.js index 6d55683..1fb3f9f 100644 --- a/html/auth-guard.js +++ b/html/auth-guard.js @@ -125,12 +125,40 @@ // Nothing replaces it. Every signed-in page mounts the drawer, so there is no page // left that would need a floating fallback pill. + // ── CR-019: usage ping ─────────────────────────────────────────────────── + // One page_open event per authenticated load, sent from exactly ONE place + // (here) rather than from each page's own script - the shared-chrome lesson + // S4 and the token-drift lesson S5 both taught this codebase the hard way. + // Fire-and-forget: never blocks reveal(), never retries, never surfaces an + // error to the person using the app - a missed usage ping is not something + // anyone here should notice happening. + var TOOL_BY_PAGE = { + 'index.html': 'launcher', + 'work-package-suite.html': 'wizard', + 'wp-creation-index.html': 'creator', + 'field.html': 'field_view', + 'admin.html': 'admin', + 'users.html': 'directory' + }; + function pingUsage() { + var page = (location.pathname.split('/').pop() || 'index.html'); + var tool = TOOL_BY_PAGE[page] || page.replace(/\.html$/, ''); + try { + fetch('/api/usage/ping', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ tool: tool }) + }).catch(function () {}); + } catch (e) {} + } + function proceed(user) { clearTimeout(safety); window.WP_USER = user; reveal(); if (window.WP_USER) { window.wpFlags(); // start the feature-flag fetch; pages await it as needed + pingUsage(); try { document.dispatchEvent(new CustomEvent('wp-auth-ready', { detail: window.WP_USER })); } catch (e) {} } } diff --git a/server/app.py b/server/app.py index 3df79e3..24f18e8 100644 --- a/server/app.py +++ b/server/app.py @@ -756,6 +756,11 @@ async def okta_callback(request: Request, db: Session = Depends(get_db)): user.failed_attempts = 0 user.locked_until = None user.last_login_at = models.utcnow() + # CR-019: one login event per Okta sign-in, written here rather than + # inferred from session creation elsewhere, so there is exactly one + # source of truth for "did this person sign in" - not one per page load + # afterward (T11.2 covers that separately, as page_open events). + db.add(models.UsageEvent(id=gen_id("uev"), username=user.username, tool="", event="login")) db.commit() db.refresh(user) @@ -3374,6 +3379,29 @@ def create_feedback(body: CommentIn, user: models.User = Depends(auth.get_curren return _save_comment(body, db, user) +# ── CR-019: usage/activity metrics ────────────────────────────────────────── +class UsageIn(BaseModel): + tool: str = "" + project_id: Optional[str] = None + + +@app.post("/api/usage/ping") +def usage_ping(body: UsageIn, user: models.User = Depends(auth.get_current_user), db: Session = Depends(get_db)): + """One usage_events row per authenticated page load. Called exactly once, + from auth-guard.js after wp-auth-ready fires (T11.2) - never duplicated + per page's own script, the same lesson S4's per-page nav already taught + this codebase. `user` comes from the session via get_current_user, never + from anything the client claims - identity here is a server-enforced + fact, matching every other write in this file, not a client-reported one.""" + tool = (body.tool or "").strip()[:40] + db.add(models.UsageEvent( + id=gen_id("uev"), username=user.username, project_id=body.project_id, + tool=tool, event="page_open", + )) + db.commit() + return {"ok": True} + + @app.get("/api/comments") def list_comments( source: Optional[str] = Query(None), From 358469531c16973d6363cc5bbe0ac35881cd4c2e Mon Sep 17 00:00:00 2001 From: Matt Mabrey Date: Wed, 23 Sep 2026 11:38:37 -0700 Subject: [PATCH 04/20] D18: revise T13.1 to idle timeout + absolute ceiling Matt asked whether idle time would be a better fit than a flat session length. It is, but idle-alone weakens D18's own purpose - a continuously active session would never force a fresh Okta recheck on its own. Decided: both. AUTH_IDLE_MINUTES (new, default 30) slides the session on activity; AUTH_SESSION_HOURS (existing var, meaning changes to an absolute ceiling, default 12 -> proposed 8) caps it regardless of activity. Docs only in this commit - implementation is T13.1, next. --- docs/waves/decisions-2026-09-17.md | 31 ++++++++++---- docs/waves/wave-13.md | 66 ++++++++++++++++++++++++------ 2 files changed, 76 insertions(+), 21 deletions(-) diff --git a/docs/waves/decisions-2026-09-17.md b/docs/waves/decisions-2026-09-17.md index 6f5aaeb..459ca7e 100644 --- a/docs/waves/decisions-2026-09-17.md +++ b/docs/waves/decisions-2026-09-17.md @@ -259,15 +259,28 @@ the actual code: signature-verification surface. Traded deliberately: this is poll-interval-late rather than real-time, which is judged acceptable for an HR/offboarding-driven event, not a to-the-second requirement. -2. **Also shrink `AUTH_SESSION_HOURS`** as defense in depth, independent of the - sync job, so the already-live-session window itself is smaller regardless - of how fast the sync runs. Proposed default: **2 hours**, down from 12 — - flagged as a recommendation, not confirmed: Okta's own SSO session (set on - the Okta side, separate from this app's cookie) may make re-authentication - silent rather than a real re-login if it outlives this app's shorter - session, in which case 2 hours costs little. Confirm against the tenant's - actual Okta session policy before treating 2 as final; if Okta's SSO - session is itself long-lived, a shorter number here doesn't hurt, either. +2. **Revised 2026-09-23, in response to Matt's question about idle time + instead of a flat session length: sessions now slide on activity, with a + hard ceiling underneath.** A flat `AUTH_SESSION_HOURS` forces a re-check + with Okta on a fixed schedule regardless of activity; a pure idle timer + with no ceiling does the opposite — a continuously-active session would + never force a fresh Okta check on its own, which is a worse fit for the + exact threat this item exists to address (someone still clicking around + after being deprovisioned). Decided: **both**. + - `AUTH_IDLE_MINUTES` (new, default **30**): a session with no request + for this long stops being valid. Implemented as a sliding JWT expiry — + the token is reissued with a fresh `exp` on activity, throttled so the + cookie isn't rewritten on literally every request. + - `AUTH_SESSION_HOURS` (existing var, meaning changes to an **absolute + ceiling**): no session survives past this many hours from the original + sign-in, no matter how continuously active it is. Default changing from + 12 to a proposed **8** — flagged as a recommendation, not confirmed. + - Both defaults, and the mechanism itself, should be sanity-checked + against the tenant's actual Okta SSO session policy — if Okta's own + session silently outlives either number, re-authentication here is + likely a fast redirect, not a real re-login screen, so these numbers + cost less than they look like they do. Confirm before treating either + as final. 3. **The sync only ever disables an account — it never re-enables one.** A rehire showing active in Okta again does not automatically restore access; an admin re-enabling the account is a deliberate act, consistent with diff --git a/docs/waves/wave-13.md b/docs/waves/wave-13.md index 47cb05e..fcf4f4b 100644 --- a/docs/waves/wave-13.md +++ b/docs/waves/wave-13.md @@ -11,27 +11,69 @@ important done-when list in this wave — do not relax it to ship faster.** --- -### T13.1 — D18: shrink `AUTH_SESSION_HOURS` +### T13.1 — D18: idle timeout with an absolute ceiling - **Items:** `D18` - **Depends on:** nothing (first task, independent of the rest) - **Blocks:** nothing - **Surface:** `server/` -- **Files:** `server/auth.py`, `server/.env.example`, `DEPLOYMENT.md` +- **Files:** `server/auth.py`, `server/app.py` (`auth_gate` middleware), + `server/.env.example`, `DEPLOYMENT.md` -**Do:** Change the default from 12 to 2 hours. Document the change and the -reasoning (defense in depth against an already-live session outliving a -deprovisioning event) in `server/.env.example` next to `AUTH_SESSION_HOURS`, -and in `DEPLOYMENT.md`'s auth section. Note in the PR that this is proposed, -not confirmed against the tenant's actual Okta SSO session policy — flag it -for Nick/IT rather than treating 2 as unquestionable. +**Revised 2026-09-23** — originally just "shrink `AUTH_SESSION_HOURS`". Matt +asked whether an idle timeout would be a better fit than a flat session +length. It is, but not by itself — see the decision record's reasoning on why +idle-with-no-ceiling is actually a worse fit for this item's own threat model +than a flat expiry would have been. Build both. + +**Do:** + +- Add `login_at` to the JWT payload in `create_token()` — the original + sign-in time, distinct from `iat`, which becomes "when THIS token was + issued" once tokens start getting reissued. `login_at` never changes across + reissues; it's what the absolute ceiling is measured from. +- Add `AUTH_IDLE_MINUTES` (default 30). `AUTH_SESSION_HOURS` stays the name + for the absolute ceiling, default changing from 12 to a proposed 8 — update + its docstring/comment in `auth.py` and `.env.example`, since its MEANING is + changing (session length -> hard ceiling on top of a sliding idle window), + not just its value. +- In `auth_gate` (`server/app.py`), after confirming a request is + authenticated: if `now < login_at + AUTH_SESSION_HOURS` (the ceiling hasn't + passed), compute `new_exp = min(now + AUTH_IDLE_MINUTES, login_at + + AUTH_SESSION_HOURS)`. If `new_exp` is meaningfully later than the current + token's `exp` (throttle this — do not reissue on every single request, only + when enough time has passed to be worth a new cookie; a few minutes of + slack is fine), mint a refreshed token carrying forward `sub`/`username`/ + `role`/`ver`/`login_at` unchanged, and set it on the response. +- If the ceiling HAS passed, do not refresh — let the existing token expire + on its own terms (it may already be invalid, or may tick over within the + idle window; either way, no new one is issued past the ceiling). +- The middleware should not need a DB round trip to do this — everything + needed (`sub`, `username`, `role`, `ver`, `login_at`) is already in the + validated claims. `get_current_user`'s existing per-request `is_active`/ + `token_version` check is unaffected and still runs separately. + +**Do not:** reissue the cookie on every request unconditionally — that's a +`Set-Cookie` header on every API call for no benefit over a coarser refresh. +Do not let the ceiling check silently vanish for tokens issued before this +change ships — a token with no `login_at` claim should fall back to treating +its own `iat` as `login_at`, not bypass the ceiling entirely. **Done when:** -- [ ] default is 2 hours; still overridable via env, unchanged mechanism -- [ ] `.env.example` and `DEPLOYMENT.md` explain why -- [ ] existing session/auth tests updated for the new default where they - assumed 12 +- [ ] a session with continuous activity stays alive past 30 minutes but is + cut off at the `AUTH_SESSION_HOURS` ceiling regardless +- [ ] a session with no activity for 30+ minutes is rejected on its next + request +- [ ] the cookie is not rewritten on every single request — verify the + refresh is throttled, not unconditional +- [ ] a pre-existing token with no `login_at` claim (simulating a session + issued before this shipped) still gets a hard ceiling, via the `iat` + fallback +- [ ] `.env.example` and `DEPLOYMENT.md` explain both variables and flag both + defaults as proposed, not confirmed against Okta's own session policy +- [ ] existing session/auth tests updated for the new mechanism, not just a + new number --- From 6cde6e3f60ec9b37310f41ac1deb96636cadf409 Mon Sep 17 00:00:00 2001 From: Matt Mabrey Date: Wed, 23 Sep 2026 11:40:59 -0700 Subject: [PATCH 05/20] T13.1: idle timeout + absolute ceiling (D18) Sessions now slide on activity (AUTH_IDLE_MINUTES, default 30) capped by a hard ceiling from original sign-in (AUTH_SESSION_HOURS, meaning changed, default 12 -> proposed 8). login_at carried across reissues so the ceiling survives refreshes; pre-D18 tokens with no login_at fall back to iat. Refresh is throttled (~IDLE_MINUTES/3) so the cookie isn't rewritten on every request. Wired into auth_gate (server/app.py) - no DB hit, reads only the already-validated claims. Verified: 7 unit-level checks (fresh-token expiry, past-ceiling refusal, throttling, mid-session extension, legacy-token fallback both live and expired, idle cutoff itself) all pass, plus the full 27-check smoke suite still passes end to end through the new middleware path. --- DEPLOYMENT.md | 9 +++++ server/.env.example | 14 ++++++- server/app.py | 15 +++++++- server/auth.py | 89 +++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 120 insertions(+), 7 deletions(-) diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md index 72db5ba..9d411ac 100644 --- a/DEPLOYMENT.md +++ b/DEPLOYMENT.md @@ -58,6 +58,15 @@ POSTGRES_PASSWORD= # openssl rand -base64 48 AUTH_SECRET_KEY= +# OPTIONAL — D18 (2026-09-23): a session slides on activity (AUTH_IDLE_MINUTES, +# default 30) capped by a hard ceiling from original sign-in regardless of +# activity (AUTH_SESSION_HOURS, default 8). Both are proposed defaults, not +# confirmed against this tenant's Okta SSO session policy — if Okta's own +# session outlives either one, re-auth here is likely a fast silent redirect +# rather than a real login screen. Full explanation in server/.env.example. +# AUTH_IDLE_MINUTES=30 +# AUTH_SESSION_HOURS=8 + # REQUIRED (in spirit — see the note below) — Okta OIDC is the only sign-in # path (D15/D16). There is no local password anywhere in this app to fall back # to, so without these nobody can sign in at all. Get them from the Okta app diff --git a/server/.env.example b/server/.env.example index ddbd99d..93721ed 100644 --- a/server/.env.example +++ b/server/.env.example @@ -27,8 +27,18 @@ DATABASE_URL=postgresql+psycopg://wpsuite:CHANGE_ME@localhost:5432/wpsuite # python -c "import secrets; print(secrets.token_urlsafe(48))" AUTH_SECRET_KEY=CHANGE_ME_run_the_command_above -# How long a login lasts before re-authentication (hours). Default 12. -# AUTH_SESSION_HOURS=12 +# D18 (2026-09-23): a session slides on activity, capped by a hard ceiling +# underneath - not one flat lifetime. Both defaults are proposed, not +# confirmed against this tenant's actual Okta SSO session policy - if Okta's +# own session outlives either number, re-auth here is likely a fast redirect +# rather than a real login screen, so these cost less than they look like. +# +# No request for this many minutes ends the session outright. +# AUTH_IDLE_MINUTES=30 +# +# The absolute ceiling from original sign-in, regardless of activity - no +# session outlives this no matter how continuously active it is. Default 8. +# AUTH_SESSION_HOURS=8 # ── Okta OIDC (required — this is the only sign-in path) ─────────────────────── # The Okta *authorization server* issuer, e.g. https://yourorg.okta.com/oauth2/default diff --git a/server/app.py b/server/app.py index 24f18e8..ae50578 100644 --- a/server/app.py +++ b/server/app.py @@ -103,12 +103,23 @@ def _csrf_ok(request: Request) -> bool: async def auth_gate(request: Request, call_next): path = request.url.path method = request.method + claims = None if method != "OPTIONS" and auth._needs_auth(path): - if not auth.is_request_authenticated(request): + claims = auth.is_request_authenticated(request) + if not claims: return JSONResponse(status_code=401, content={"detail": "Not authenticated"}) if method in _UNSAFE_METHODS and path.startswith("/api/") and not _csrf_ok(request): return JSONResponse(status_code=403, content={"detail": "Cross-origin request rejected"}) - return await call_next(request) + response = await call_next(request) + # D18: slide the session forward on activity, capped by its absolute + # ceiling. Reads only the already-validated claims - no DB hit here, and + # the separate is_active/token_version check in get_current_user still + # runs on its own for every request regardless of whether this refreshes. + if claims is not None: + refreshed = auth.maybe_refresh_token(claims) + if refreshed: + auth.set_session_cookie(response, request, refreshed) + return response def gen_id(prefix: str) -> str: diff --git a/server/auth.py b/server/auth.py index 17b9bd2..af58dbb 100644 --- a/server/auth.py +++ b/server/auth.py @@ -59,8 +59,26 @@ log = logging.getLogger("wpsuite.auth") COOKIE_NAME = "wp_session" JWT_ALG = "HS256" -# How long a session lasts before the person must sign in again. -SESSION_HOURS = int(os.getenv("AUTH_SESSION_HOURS", "12")) +# D18 (2026-09-23): a session now slides on activity, capped by a hard ceiling +# underneath - not a single flat lifetime. A pure idle timer with no ceiling +# would let a continuously-active session never force a fresh Okta recheck, +# which is a worse fit for this item's own purpose (catching someone still +# active after being deprovisioned) than a flat expiry would have been. Both +# numbers are proposed defaults, not confirmed against the tenant's actual +# Okta SSO session policy - see docs/waves/decisions-2026-09-17.md. +# +# No request for this long invalidates the session outright. +IDLE_MINUTES = int(os.getenv("AUTH_IDLE_MINUTES", "30")) +# The absolute ceiling from the ORIGINAL sign-in, regardless of activity. Same +# env var name as the old flat-lifetime design; the meaning changed, the name +# didn't, because it still answers "how long can this session possibly live." +SESSION_HOURS = int(os.getenv("AUTH_SESSION_HOURS", "8")) +# How much later a refreshed exp must be before it's worth rewriting the +# cookie. Without this, an active session gets a new Set-Cookie on literally +# every request - correct, but wasteful, and it makes the cookie a busier +# target than it needs to be. A third of the idle window is a reasonable +# balance: refreshed a few times within any idle window, never every request. +_REFRESH_SLACK = timedelta(minutes=max(1, IDLE_MINUTES // 3)) # ── permissions roles ───────────────────────────────────────────────────────── ROLE_ADMIN = "admin" @@ -160,6 +178,30 @@ SECRET_KEY = _load_secret() # ── tokens ────────────────────────────────────────────────────────────────── +def _parse_claim_dt(v) -> Optional[datetime]: + """`login_at` is a custom claim, so unlike `exp`/`iat` (which PyJWT + special-cases for a datetime -> POSIX-timestamp conversion on encode) it + is stored and read back as a plain numeric timestamp. Returns None for + anything unparseable rather than raising - a malformed or legacy token + should fail closed into "no refresh", not 500.""" + if v is None: + return None + try: + return datetime.fromtimestamp(float(v), tz=timezone.utc) + except (TypeError, ValueError, OSError): + return None + + +def _next_exp(login_at: datetime, now: datetime) -> datetime: + """Whichever comes first: another IDLE_MINUTES of quiet from now, or the + absolute SESSION_HOURS ceiling measured from the session's original + sign-in. Shared by create_token and maybe_refresh_token so the two can't + drift apart.""" + ceiling = login_at + timedelta(hours=SESSION_HOURS) + idle_edge = now + timedelta(minutes=IDLE_MINUTES) + return min(ceiling, idle_edge) + + def create_token(user: "models.User") -> str: now = datetime.now(timezone.utc) payload = { @@ -168,7 +210,48 @@ def create_token(user: "models.User") -> str: "role": user.role, "ver": user.token_version or 0, "iat": now, - "exp": now + timedelta(hours=SESSION_HOURS), + "login_at": now.timestamp(), + "exp": _next_exp(now, now), + } + return jwt.encode(payload, SECRET_KEY, algorithm=JWT_ALG) + + +def maybe_refresh_token(claims: dict) -> Optional[str]: + """Given a validated token's claims, return a reissued token if the + session is worth extending, or None if nothing should change. Called from + `auth_gate` on every authenticated request (D18) - deliberately reads only + the already-validated claims, never the database, so it costs nothing + beyond the JWT encode itself. The separate is_active/token_version check + in get_current_user is unaffected either way. + + Three ways this returns None: past the absolute ceiling (session is done, + full re-login required - never extended, not even by a second); the + claims can't be parsed (fails closed into no-refresh rather than guessing); + or a refresh happened recently enough that a new cookie isn't worth + writing yet (_REFRESH_SLACK).""" + now = datetime.now(timezone.utc) + login_at = _parse_claim_dt(claims.get("login_at")) + if login_at is None: + # Pre-D18 token (no login_at claim) - fall back to iat so it still + # gets a real ceiling instead of riding on the old flat exp forever. + login_at = _parse_claim_dt(claims.get("iat")) + if login_at is None: + return None + ceiling = login_at + timedelta(hours=SESSION_HOURS) + if now >= ceiling: + return None + new_exp = _next_exp(login_at, now) + current_exp = _parse_claim_dt(claims.get("exp")) + if current_exp is not None and (new_exp - current_exp) < _REFRESH_SLACK: + return None + payload = { + "sub": claims.get("sub"), + "username": claims.get("username"), + "role": claims.get("role"), + "ver": claims.get("ver", 0), + "iat": now, + "login_at": login_at.timestamp(), + "exp": new_exp, } return jwt.encode(payload, SECRET_KEY, algorithm=JWT_ALG) From 8e863ae7d0703bc4a2b1042fc87575cc7065bc33 Mon Sep 17 00:00:00 2001 From: Matt Mabrey Date: Wed, 23 Sep 2026 12:19:14 -0700 Subject: [PATCH 06/20] T11.3: usage aggregation endpoint with filters (CR-019) GET /api/usage/summary: active-user counts by day/week/month, per-user last-active, per-tool breakdown. Filters (from/to/project_id/username/ tool) combine. Gated by require_user_manager - same boundary as the User Directory. A project_super_user is scoped to events tied to projects they manage plus their own activity (managed_project_ids), never another user's suite-wide activity outside that; an app admin sees everything. _usage_query() factored out so T11.4's export can never disagree with what this endpoint counted - same filtered row set, not two derivations. Verified: admin sees all seeded events; a project_super_user scoped to one of two projects correctly sees only that project's events plus their own account-wide activity, and specifically does NOT see the admin's other-project or no-project activity; date/tool/project filters each narrow results correctly and combine. --- server/app.py | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 1 deletion(-) diff --git a/server/app.py b/server/app.py index ae50578..60b1353 100644 --- a/server/app.py +++ b/server/app.py @@ -13,6 +13,7 @@ import logging import os import re import uuid +from datetime import datetime, timedelta, timezone from typing import Any, Optional from urllib.parse import urlparse @@ -22,7 +23,7 @@ from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import JSONResponse, RedirectResponse from fastapi.staticfiles import StaticFiles from pydantic import BaseModel, ConfigDict, Field -from sqlalchemy import select, delete, func +from sqlalchemy import select, delete, func, or_ from sqlalchemy.orm import Session from starlette.middleware.sessions import SessionMiddleware @@ -3413,6 +3414,101 @@ def usage_ping(body: UsageIn, user: models.User = Depends(auth.get_current_user) return {"ok": True} +def _parse_date_q(v: str, end: bool = False) -> Optional[datetime]: + """Accepts a plain YYYY-MM-DD (what a sends) or a full + ISO datetime. A date-only `to` means "through the end of that day", not + midnight at its start - otherwise a range of "today" would match nothing + from today at all.""" + if not v: + return None + try: + d = datetime.fromisoformat(v) + except ValueError: + return None + if d.tzinfo is None: + d = d.replace(tzinfo=timezone.utc) + if end and len(v) <= 10: # date-only + d = d + timedelta(days=1) - timedelta(microseconds=1) + return d + + +def _usage_query(db: Session, caller: "models.User", date_from, date_to, project_id, username, tool): + """Shared by the summary and export endpoints so the two can never + disagree about which rows a filter set matches - the export is a raw + dump of exactly what the summary counted, not a separately-derived view.""" + stmt = select(models.UsageEvent) + managed = managed_project_ids(db, caller) + if managed is not None: + # A project_super_user (never an app admin, who gets managed=None) is + # scoped to events tied to a project they administer, plus their OWN + # suite-wide activity (admin console opens etc. carry no project_id) - + # never another user's activity outside what they manage. + if managed: + stmt = stmt.where(or_( + models.UsageEvent.project_id.in_(managed), + models.UsageEvent.username == caller.username, + )) + else: + stmt = stmt.where(models.UsageEvent.username == caller.username) + df = _parse_date_q(date_from) + dt = _parse_date_q(date_to, end=True) + if df: + stmt = stmt.where(models.UsageEvent.at >= df) + if dt: + stmt = stmt.where(models.UsageEvent.at <= dt) + if project_id: + stmt = stmt.where(models.UsageEvent.project_id == project_id) + if username: + stmt = stmt.where(models.UsageEvent.username == username) + if tool: + stmt = stmt.where(models.UsageEvent.tool == tool) + return stmt.order_by(models.UsageEvent.at) + + +@app.get("/api/usage/summary") +def usage_summary( + date_from: Optional[str] = Query(None, alias="from"), + date_to: Optional[str] = Query(None, alias="to"), + project_id: Optional[str] = Query(None), + username: Optional[str] = Query(None), + tool: Optional[str] = Query(None), + caller: models.User = Depends(require_user_manager), + db: Session = Depends(get_db), +): + """CR-019. Same gate as the User Directory (require_user_manager): an app + admin or a project_super_user with at least one managed project. Filters + combine. Aggregated in Python over the filtered row set rather than a SQL + GROUP BY - correct and simple at today's scale; if usage_events grows + into the millions (plausible, given retention is indefinite by decision), + the day/week/month bucketing here is the first thing to move server-side + into SQL. Not done now because nothing currently requires it.""" + rows = db.scalars(_usage_query(db, caller, date_from, date_to, project_id, username, tool)).all() + + by_day: dict[str, set] = {} + by_week: dict[str, set] = {} + by_month: dict[str, set] = {} + per_user_last: dict[str, datetime] = {} + per_tool: dict[str, int] = {} + for e in rows: + by_day.setdefault(e.at.date().isoformat(), set()).add(e.username) + by_week.setdefault(e.at.strftime("%G-W%V"), set()).add(e.username) + by_month.setdefault(e.at.strftime("%Y-%m"), set()).add(e.username) + if e.username not in per_user_last or e.at > per_user_last[e.username]: + per_user_last[e.username] = e.at + per_tool[e.tool] = per_tool.get(e.tool, 0) + 1 + + return { + "active_users": { + "by_day": {k: len(v) for k, v in sorted(by_day.items())}, + "by_week": {k: len(v) for k, v in sorted(by_week.items())}, + "by_month": {k: len(v) for k, v in sorted(by_month.items())}, + }, + "per_user_last_active": {u: models._iso(t) for u, t in sorted(per_user_last.items())}, + "by_tool": dict(sorted(per_tool.items(), key=lambda kv: -kv[1])), + "event_count": len(rows), + } + + @app.get("/api/comments") def list_comments( source: Optional[str] = Query(None), From 2de76d52e60a71540d3dbf78e60023108ad67126 Mon Sep 17 00:00:00 2001 From: Matt Mabrey Date: Wed, 23 Sep 2026 12:25:05 -0700 Subject: [PATCH 07/20] T11.4: usage export endpoint (raw + sanitized CSV) CR-019 / wave 11. Adds GET /api/usage/export, reusing _usage_query()'s scoped filters (from/to/project_id/username/tool) and the same require_user_manager gate as /api/usage/summary. Two modes: - raw (default): real usernames, for internal admin use. - sanitize=true: usernames replaced with an HMAC-SHA256 pseudonym (keyed with auth.SECRET_KEY, 16 hex chars, 'u_' prefix) so the file can be fed into PowerBI or another external reporting tool without carrying real identities. HMAC chosen over a plain hash since the username space is small enough to brute-force a bare digest. Both modes emit at, username, project_id, tool, event as columns and deliberately omit the detail JSON column in both modes to avoid an identity leak riding along inside free-form detail data. Response is returned with a Content-Disposition: attachment header and a filename that encodes mode + date. Verified locally against a throwaway SQLite DB with two seeded users and four seeded UsageEvent rows: - raw export contains the real usernames and matches the summary endpoint's event_count for the same session state - sanitized export contains no real username or email anywhere in the file body, across two independently-issued export calls - the same real user maps to the same pseudonym both within one export and across the two separate export calls - raw and sanitized rows line up 1:1 on at/tool/event for the same filter set - the tool= filter narrows the export the same way it narrows the summary - a plain project_user is refused with 403; an unauthenticated request is refused with 401 - full smoke test (27/27) and seed_demo.py both still pass --- server/app.py | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/server/app.py b/server/app.py index 60b1353..31a6f99 100644 --- a/server/app.py +++ b/server/app.py @@ -9,6 +9,10 @@ Run (prod): gunicorn -k uvicorn.workers.UvicornWorker -b 127.0.0.1:8000 serve Interactive docs: http:///api/docs """ import base64 +import csv +import hashlib +import hmac +import io import logging import os import re @@ -3509,6 +3513,59 @@ def usage_summary( } +def _pseudonym(username: str) -> str: + """A stable per-user id for the sanitized export — the SAME input always + produces the SAME output, within one export and across separate export + runs, so an external system (Power BI or similar) can still group and + trend "by user" without ever receiving a real name. HMAC rather than a + plain hash: a plain sha256(username) is trivially reversed against a + wordlist of the handful of usernames this app actually has; keying it + with AUTH_SECRET_KEY (already a real secret, already required in + production — see auth.py) means recovering a username from its + pseudonym requires the signing key, not just guessing.""" + digest = hmac.new(auth.SECRET_KEY.encode(), username.encode(), hashlib.sha256).hexdigest() + return "u_" + digest[:16] + + +@app.get("/api/usage/export") +def usage_export( + date_from: Optional[str] = Query(None, alias="from"), + date_to: Optional[str] = Query(None, alias="to"), + project_id: Optional[str] = Query(None), + username: Optional[str] = Query(None), + tool: Optional[str] = Query(None), + sanitize: bool = Query(False), + caller: models.User = Depends(require_user_manager), + db: Session = Depends(get_db), +): + """CR-019. Same gate, same filters, same underlying row set as + usage_summary() (_usage_query) — the export can never show a different + slice of data than what the console counted for the same filters. + + sanitize=true replaces `username` with a stable pseudonym (_pseudonym) + and — deliberately — the `detail` column is not exported in EITHER mode. + Every event this app writes today (login, page_open) leaves `detail` + empty, so this costs nothing now, but it also means a future event type + that DOES populate `detail` can't accidentally leak a real name into a + sanitized file through a column nobody thought to scrub. If `detail` + is ever needed in the export, it has to be sanitized explicitly, not + assumed safe because the rest of the row was.""" + rows = db.scalars(_usage_query(db, caller, date_from, date_to, project_id, username, tool)).all() + buf = io.StringIO() + w = csv.writer(buf) + w.writerow(["at", "username", "project_id", "tool", "event"]) + for e in rows: + who = _pseudonym(e.username) if sanitize else e.username + w.writerow([models._iso(e.at), who, e.project_id or "", e.tool, e.event]) + filename = "usage_export_%s_%s.csv" % ( + "sanitized" if sanitize else "raw", datetime.now(timezone.utc).strftime("%Y%m%d"), + ) + return Response( + content=buf.getvalue(), media_type="text/csv", + headers={"Content-Disposition": f'attachment; filename="{filename}"'}, + ) + + @app.get("/api/comments") def list_comments( source: Optional[str] = Query(None), From a6c3fbfe504aa02a73c321dbdd882854330b69bd Mon Sep 17 00:00:00 2001 From: Matt Mabrey Date: Wed, 23 Sep 2026 12:31:55 -0700 Subject: [PATCH 08/20] T11.5: admin console Activity & usage card (CR-019) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New card in admin.html/admin.js, above the old per-browser Usage logs card (which T11.6 retires next). Filters (date range, project, user, tool) drive GET /api/usage/summary; two export buttons call GET /api/usage/export (raw / sanitized) and save the CSV via a Blob, same download pattern wp-usage.js already uses. Access: the card lives inside admin.html, already gated admin-only client-side by gateByRole() (unchanged) - a non-admin never sees the card. The underlying API is gated server-side by require_user_manager regardless (admin or project_super_user on >=1 project), independent of and stricter than the client-side gate, so a non-admin request is refused even if someone reached the endpoint directly. Accessibility (C1): every control is a real / + + + + + + + + +
Loading…
+
+ + +
+ + + +

Usage logs

Engagement recorded by both tools — the work package creator and the SOP wizard — diff --git a/html/admin.js b/html/admin.js index f339bd1..e1a5934 100644 --- a/html/admin.js +++ b/html/admin.js @@ -24,6 +24,7 @@ function reveal(){ loadNotifications(); loadComments(); loadAudit(); + loadActivity(); loadUsage(); } function showDenied(){ @@ -189,6 +190,19 @@ async function loadProjects(){ banner.style.display='none'; _adminProjects = json; renderProjects(); + populateActivityProjectFilter(); +} + +// The activity project filter reuses the same project list the Projects card +// already fetched — no second /api/projects call just to fill a