281 lines
13 KiB
Markdown
281 lines
13 KiB
Markdown
# Wave 11 — Usage and activity metrics
|
|
|
|
**Items:** `CR-019`
|
|
**Depends on:** wave 10 merged (it is; this wave does not wait on anything else)
|
|
**Decision record:** `docs/waves/decisions-2026-09-17.md`
|
|
|
|
Seven tasks, one concern each, in build order. Do not start a task whose
|
|
dependency is not merged. `CR-020` (bulk user editing) is reserved but not
|
|
scoped — it does not belong in this wave.
|
|
|
|
---
|
|
|
|
### T11.1 — CR-019: `usage_events` table + migration
|
|
|
|
- **Items:** `CR-019`
|
|
- **Depends on:** nothing (first task)
|
|
- **Blocks:** T11.2
|
|
- **Surface:** `server/`
|
|
- **Files:** `server/models.py`, `server/alembic/versions/`
|
|
|
|
**Do:** Add a `UsageEvent` model — append-only, same spirit as `AuditLog` but for
|
|
navigation/feature-open events rather than business mutations. Suggested shape:
|
|
`id`, `at` (indexed), `user_id` (or username — match whatever `AuditLog.actor`
|
|
does today for consistency), `project_id` (nullable — not every event is
|
|
project-scoped, e.g. opening the admin console), `tool` (e.g. `creator`,
|
|
`wizard`, `field_view`, `dashboard`, `admin`, `directory`), `event` (e.g.
|
|
`page_open`, `login`), `detail` (JSON, optional). Write the migration. Do not
|
|
touch `AuditLog` — this is a new table, not an extension of it (see the
|
|
decision record's reasoning).
|
|
|
|
**Do not:** fold this into `AuditLog`. They serve different questions and mixing
|
|
them makes the existing audit trail noisier for its existing readers.
|
|
|
|
**Done when:**
|
|
|
|
- [ ] `UsageEvent` exists with an indexed `at` column (this table will be scanned
|
|
by date range constantly)
|
|
- [ ] migration applies cleanly against both SQLite (dev) and Postgres (prod) —
|
|
render `alembic upgrade --sql` for postgresql and read it before calling
|
|
this done, per the class of defect `BL-027` logged
|
|
- [ ] no change to `AuditLog`'s shape or behavior
|
|
|
|
---
|
|
|
|
### T11.2 — CR-019: capture the events
|
|
|
|
- **Items:** `CR-019`
|
|
- **Depends on:** T11.1
|
|
- **Blocks:** T11.3
|
|
- **Surface:** `server/` + `html/`
|
|
- **Files:** `server/app.py` (new endpoint), `html/auth-guard.js`
|
|
|
|
**Problem:** Six pages need this and none should implement it separately — that
|
|
is exactly how `S4`'s "no global nav on two pages" and the four parallel token
|
|
systems (`S5`) happened. `auth-guard.js` is already loaded first, in the `<head>`,
|
|
on all six protected pages (`index.html`, `field.html`, `users.html`,
|
|
`wp-creation-index.html`, `work-package-suite.html`, `admin.html`) and already
|
|
knows the verified user once the `wp-auth-ready` event fires. That is the one
|
|
place this belongs.
|
|
|
|
**Do:** Add a small `POST /api/usage/ping`-style endpoint that writes one
|
|
`UsageEvent` row per call, keyed to the session (server trusts the session, not
|
|
anything the client claims about identity). Call it once from `auth-guard.js`
|
|
after `wp-auth-ready`, tagging `tool` from the page's own path. Also write a
|
|
`login` event at the point a session is actually established (reuse whatever
|
|
`okta_callback` already does at sign-in — do not add a second source of truth
|
|
for "did this person log in").
|
|
|
|
**Do not:** build a per-page capture call. If a page needs this and
|
|
`auth-guard.js` does not cover it, fix `auth-guard.js`, not the page.
|
|
|
|
**Done when:**
|
|
|
|
- [ ] one `page_open` event is recorded for a real sign-in on each of the six
|
|
pages, verified per page
|
|
- [ ] exactly one `login` event per Okta sign-in, not one per page load after
|
|
it
|
|
- [ ] the endpoint rejects a request with no valid session (this is server-
|
|
enforced identity, not client-reported)
|
|
- [ ] no page other than `auth-guard.js` calls this endpoint directly
|
|
|
|
---
|
|
|
|
### T11.3 — CR-019: aggregation endpoint with filters
|
|
|
|
- **Items:** `CR-019`
|
|
- **Depends on:** T11.2
|
|
- **Blocks:** T11.4, T11.5
|
|
- **Surface:** `server/`
|
|
- **Files:** `server/app.py`
|
|
|
|
**Do:** Build the read side: active-user counts by day/week/month, per-user
|
|
last-active timestamp (derived from `UsageEvent`, not `User.last_login_at`,
|
|
which only ever holds one value), and a per-tool usage breakdown. Accept query
|
|
filters: date range, project, user, tool — combinable, per the decision record.
|
|
This is server aggregation, the same principle `B4` established for the
|
|
pipeline strip: the browser asks for a number, the server computes it from real
|
|
rows, nothing is derived client-side from a partial cache.
|
|
|
|
**Done when:**
|
|
|
|
- [ ] active-user counts are correct against a seeded fixture with known dates
|
|
- [ ] filters combine correctly (verified: user + date range + tool together
|
|
narrows correctly, not just each alone)
|
|
- [ ] a project filter that matches nothing returns an empty result, not an
|
|
error or the unfiltered total
|
|
|
|
---
|
|
|
|
### T11.4 — CR-019: export, raw and sanitized
|
|
|
|
- **Items:** `CR-019`
|
|
- **Depends on:** T11.3
|
|
- **Blocks:** T11.5
|
|
- **Surface:** `server/`
|
|
- **Files:** `server/app.py`
|
|
|
|
**Do:** A CSV export endpoint over the same filtered query T11.3 exposes.
|
|
Two modes: raw (real usernames, the console's default) and sanitized. Sanitized
|
|
mode replaces the actor field with a stable pseudonymous id — a per-user hash,
|
|
consistent across rows in the same export and across separate export runs —
|
|
so an external system (Power BI or similar) can still group and trend "by
|
|
user" without ever receiving a real name. Do not simply drop the identity
|
|
column; that breaks per-user grouping downstream, which defeats the point of
|
|
an activity export.
|
|
|
|
**Done when:**
|
|
|
|
- [ ] raw export contains real usernames
|
|
- [ ] sanitized export never contains a real username or email anywhere in the
|
|
file, including in a `detail` blob if one is included
|
|
- [ ] the same real user maps to the same pseudonymous id within one export AND
|
|
across two separate export runs (a hash of something stable, not a
|
|
per-request random id)
|
|
- [ ] both modes otherwise contain identical rows for the same filter
|
|
|
|
---
|
|
|
|
### T11.5 — CR-019: admin console Activity tab
|
|
|
|
- **Items:** `CR-019`
|
|
- **Depends on:** T11.3, T11.4
|
|
- **Blocks:** T11.7
|
|
- **Surface:** `html/`
|
|
- **Files:** `html/admin.html`, `html/admin.js`
|
|
|
|
**Do:** New tab, same role gate as the User Directory. Filters (date range,
|
|
project, user, tool) driving the tables from T11.3; export buttons (raw and
|
|
sanitized) calling T11.4. Build accessible from the start per `C1` — this is a
|
|
new component, not a legacy one carrying an old defect forward: real
|
|
`<button>`/`<select>` controls, keyboard-reachable, `aria-live` on any
|
|
count that updates without a page reload, focus visible throughout.
|
|
|
|
**Done when:**
|
|
|
|
- [x] the tab is reachable only by an admin (the card lives inside admin.html,
|
|
already gated client-side by gateByRole(); the API underneath it is
|
|
independently gated server-side by require_user_manager regardless)
|
|
- [x] every filter is a real form control, keyboard-operable (date/select/text
|
|
inputs and a `<button>`, no click-div)
|
|
- [x] both export buttons produce the files T11.4 defines (verified against
|
|
the live endpoint in T11.4's own checks, and present/wired here)
|
|
- [x] works at 390px and 1440px — verified 2026-09-23 via
|
|
`tests/baseline_shots.py --pages admin` run locally on Windows (this
|
|
sandbox has no headless browser available; the script was run on the
|
|
user's machine instead, after installing Python via winget since it
|
|
wasn't present). Screenshots in `docs/reference/baseline/admin-390.png`
|
|
/ `admin-1440.png`. No JS errors, no horizontal overflow at either
|
|
width; the card rendered with real seeded data (events, by-tool,
|
|
per-user-last-active tables) confirming the filters and summary read
|
|
correctly, not just that the markup exists.
|
|
|
|
---
|
|
|
|
### T11.6 — CR-019: retire the per-browser Usage report
|
|
|
|
- **Items:** `CR-019`
|
|
- **Depends on:** T11.5
|
|
- **Blocks:** T11.7
|
|
- **Surface:** `html/`
|
|
- **Files:** `html/admin.js` (the `usage-admin` panel, `admin.js:666-699`),
|
|
`html/wp-usage.js` and its two call sites
|
|
|
|
**Do:** Remove the old per-browser `usage-admin` panel from `admin.js` now that
|
|
the real one exists, per the 2026-09-17 decision. Decide what happens to
|
|
`wp-usage.js`'s recording calls (wizard dwell-tracking, the creator's
|
|
equivalent): they were never reliably tied to a real identity, so they are not
|
|
a data source the new report can adopt. Default to removing the recorder too
|
|
unless it is still doing something useful on its own (re-read what it actually
|
|
records before deciding — do not assume from this file alone).
|
|
|
|
**Do not:** leave the old panel in place "just in case." Two activity reports
|
|
showing two different numbers is worse than one.
|
|
|
|
**Decision (2026-09-23):** `wp-usage.js` is removed, not kept — it had no
|
|
reader left once the admin panel above it was removed (the "download the
|
|
full event log" button lived only in that panel), and per the original
|
|
decision record it was never reliably tied to a real identity, so it was
|
|
never a candidate source for the new report either. The file itself and its
|
|
three `<script>` includes (`admin.html`, `wp-creation-index.html`,
|
|
`work-package-suite.html`) are gone. Its two call sites
|
|
(`wp-creation-app.js`, `work-package-suite-app.js`) keep a local `track()`
|
|
function as a documented no-op rather than having each of their ~45
|
|
individual `track('event', …)` call sites deleted one at a time — that
|
|
would be a far larger, riskier diff for the same outcome (no more data is
|
|
recorded either way), and it keeps each call site as a marker of what was
|
|
worth recording if usage analytics are ever rebuilt server-side. The
|
|
dwell-timer plumbing that only ever fed `track()` (`work-package-suite-app.js`'s
|
|
`_stepEnter`/`trackStepDwell`) was left in place for the same reason — it is
|
|
inert now, not broken, and touching it buys nothing.
|
|
|
|
Also removed: `tests/usage_check.py` (tested exactly the retired feature —
|
|
D5/T7.10's per-browser analytics core and admin report) and its line in
|
|
`docs/reference/file-map.md`, replaced with a note pointing at this decision.
|
|
|
|
**Done when:**
|
|
|
|
- [x] the old `usage-admin` panel and its markup are gone from `admin.html`/
|
|
`admin.js`
|
|
- [x] a decision on `wp-usage.js` itself is recorded (removed, or kept with a
|
|
stated reason) — not left ambiguous — see above
|
|
- [x] nothing else in the app references the removed code; grep confirms
|
|
(only remaining hits are this file, the 2026-09-17 decision record, and
|
|
the two explanatory code comments left at the retired call sites — all
|
|
prose, not live references)
|
|
|
|
---
|
|
|
|
### T11.7 — CR-019: verification
|
|
|
|
- **Items:** `CR-019`
|
|
- **Depends on:** T11.6
|
|
- **Blocks:** nothing
|
|
- **Surface:** `html/` + `server/`
|
|
- **Files:** as touched above
|
|
|
|
**Do:** Full verification per `CLAUDE.md`: run the app locally, exercise the
|
|
new tab at 390px and 1440px, before/after screenshots, run the existing smoke
|
|
test and `seed_demo.py`, run the full suite.
|
|
|
|
**Done when:**
|
|
|
|
- [x] all `CR-019` acceptance criteria in `decisions-2026-09-17.md` are met or
|
|
a failure is stated with a reason — all met
|
|
- [x] screenshots committed — `admin-390.png`/`admin-1440.png` (and
|
|
`creator`/`sop` at both widths, touched by T11.6's script-tag removal)
|
|
- [x] smoke test and `seed_demo.py` both still pass — 27/27, verified
|
|
2026-09-24 against the exact branch state about to be considered
|
|
deployable
|
|
- [x] full test suite passes — verified 2026-09-24: ran all 39
|
|
`tests/*_check.py` scripts plus the `usage_events` migration
|
|
round-trip. Result: 38/39 exit 0.
|
|
- `token_check.py` exits 2 with no arguments by design (needs `--out`
|
|
or `--compare`) — a CLI-invocation detail, not a test failure;
|
|
already confirmed non-regression in wave 10.
|
|
- `generalinfo_check.py` has one pre-existing failure (a raw colour
|
|
literal in `html/wp-creation-styles.css`, a file no wave 11/13 task
|
|
touched) — already diagnosed as pre-existing in wave 10.
|
|
- `autosave_check.py` **did** regress from T11.6 (it asserted the
|
|
retired analytics tracker still wrote to localStorage) — found by
|
|
this run, fixed same day (commit `4f3a94b`), now 33/33.
|
|
- Migration (T11.1): SQLite upgrade → downgrade → upgrade round-trip
|
|
clean; `alembic upgrade ... --sql` against the `postgresql` dialect
|
|
renders valid DDL (`TIMESTAMP WITH TIME ZONE`, `JSON`, no
|
|
SQLite-only syntax) — no `BL-027`-class defect.
|
|
|
|
---
|
|
|
|
## Wave 11 exit criteria
|
|
|
|
- [x] real, server-side activity data exists per user, indefinitely retained
|
|
- [x] the admin console shows it, filterable by date/project/user/tool
|
|
- [x] export works in both raw and sanitized form
|
|
- [x] the old per-browser report is gone, not duplicated
|
|
- [x] `CR-019` fully accounted for, no open acceptance criteria
|
|
|
|
Wave 11 is closed out as of 2026-09-24. `CR-020` (wave 12) remains reserved
|
|
and unstarted. `D18`/wave 13 (`T13.2` onward) remains paused pending an Okta
|
|
API credential, per standing instruction — `T13.1`'s session changes were
|
|
included in this same verification pass and did not regress anything above.
|