Release: wave 11 + T13.2-T13.5 (D18 Okta sync), T13.1 held back until T14.1; mobile roadmap docs (BL-062..BL-069)

This commit is contained in:
2026-10-16 14:03:33 -07:00
37 changed files with 3311 additions and 347 deletions

View File

@@ -404,6 +404,72 @@ Directory page — no shell access needed.
unreachable for everyone, including admins, until Okta is restored — the same
posture the abandoned LDAPS design took, carried forward deliberately (D16).
### Accounts the Okta sync disabled (D18)
**Who this is for:** app admins, and project super users who manage accounts.
When `OKTA_API_TOKEN` is set, the `api` container asks Okta every 15 minutes
(`OKTA_SYNC_INTERVAL_SECONDS`) who is still active. It **disables** a local
account when the person is missing from Okta's list, or when Okta shows the
status SUSPENDED or DEPROVISIONED. A password expiry, a lockout, or a recovery
in progress does not disable anyone. With no token set, the sync is off and the
startup log says `Okta sync DISABLED`.
The sync only ever disables. **It never re-enables an account.** A person comes
back only when an admin enables the account by hand.
**How to see what the sync did**
1. Open the **Admin Console** (`admin.html`). Find the **Activity log** card.
2. Click **Refresh**. Search for `okta_sync`, or set the type to **User accounts**.
3. A row from the sync reads: Who `system:okta_sync`, Action `user deprovisioned
by sync`, Item the username, Detail `okta_status: SUSPENDED` (or `MISSING` if
the person was not in Okta's list at all).
Only app admins see these rows. They belong to no project.
**How to re-enable a person**
1. **Fix the person in Okta first.** Ask the Okta admin to make the person ACTIVE
again (or to restore them, for a rehire). If you skip this step, the next sync
cycle (up to 15 minutes later) disables the account again.
2. Open the **User Directory** (`users.html`).
3. Find the person. A disabled account shows the **Enable** button on its row.
Click it.
4. The person can sign in on their next attempt. The app writes a `user enabled`
row to the Activity log under your name.
**Who may do step 3**
* An **app admin** may enable any account.
* A **project super user** may enable an account only if every project the person
belongs to is a project that super user administers. If the person is also on a
project that the super user does not administer, the row is read-only and shows
the reason. An app admin must make the change.
* Nobody can change their own account.
**If no admin can sign in** (for example, a bad sync run disabled every admin),
use the shell on the host. Replace `alice` with the username:
```bash
docker compose exec api python -m server.manage_users list
docker compose exec api python -m server.manage_users enable alice
```
**If the sync disabled the wrong people**
1. Turn the sync off. Remove `OKTA_API_TOKEN` from `.env` and restart the api
(`docker compose up -d api`). The sync starts only when the token is set.
2. Find the cause before you turn it on again. The usual cause is that the Okta
profile attribute does not match `users.username`. See
`OKTA_SYNC_PROFILE_ATTRIBUTE` in `server/.env.example`.
3. Enable the people who were disabled by mistake (steps above).
4. Set the token again and restart.
If the sync cannot reach Okta, or Okta gives an answer the sync cannot trust, the
sync changes **no** account in that cycle and writes a line to the api log:
`docker compose logs api | grep "Okta sync"`.
## Permissions roles
`User.role` is the **permissions** role; `User.project_role` is the person's **job

View File

@@ -49,6 +49,18 @@ services:
# here instead, so an operator who leaves .env's copy commented out gets
# the real default, not a broken one.
OKTA_IDENTITY_CLAIM: ${OKTA_IDENTITY_CLAIM:-preferred_username}
# Okta deprovisioning sync (D18, wave 13). OFF until OKTA_API_TOKEN is set; the
# startup log says which. The token is a read-only Okta API token, NOT the OIDC
# client secret above. Empty values here mean "unset" to the code, so the
# ${VAR:-} form is safe for all four (unlike OKTA_IDENTITY_CLAIM above, whose
# code default only applies when the variable is absent).
# Reaching Okta needs outbound https. The `outbound` network the api service
# already joins is an ordinary bridge with a gateway, so no new network is
# needed; the host firewall must allow https to the Okta org.
OKTA_API_TOKEN: ${OKTA_API_TOKEN:-}
OKTA_ORG_URL: ${OKTA_ORG_URL:-}
OKTA_SYNC_PROFILE_ATTRIBUTE: ${OKTA_SYNC_PROFILE_ATTRIBUTE:-}
OKTA_SYNC_INTERVAL_SECONDS: ${OKTA_SYNC_INTERVAL_SECONDS:-}
# Optional — SMTP password for WP-assignment emails. Email is off by
# default and enabled from the Admin console; this is the only email
# secret and it is never stored in the DB. Leave unset until configured.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 382 KiB

After

Width:  |  Height:  |  Size: 307 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 315 KiB

After

Width:  |  Height:  |  Size: 260 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 193 KiB

After

Width:  |  Height:  |  Size: 169 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 135 KiB

After

Width:  |  Height:  |  Size: 125 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 41 KiB

After

Width:  |  Height:  |  Size: 34 KiB

View File

@@ -283,7 +283,9 @@ python tests/triage_check.py # A6 - the sidebar answers the stand-up
python tests/qa_gate_check.py # CR-014/D2/D9/D10 - QA gate + capture sink 41 checks
python tests/files_check.py # CR-007/D8 - drawings upload + real offline 36 checks
python tests/sticky_bar_check.py # B6 - save reachable on every wizard step 12 checks
python tests/usage_check.py # D5 - one analytics core, admin report 15 checks
# tests/usage_check.py (D5) removed at T11.6 - it tested the per-browser
# analytics core and admin report, both retired in favour of CR-019's
# server-side Activity & usage card (see docs/waves/decisions-2026-09-17.md).
python tests/creator_dialogs_check.py # S1 creator - 0 natives, errors at fields 20 checks
```

View File

@@ -696,3 +696,124 @@ deliberately deferred.
- **Suggested wave or follow-up:** wave 14, alongside T14.1's badge changes:
do not show the badge on pages with nothing to sync, and keep it clear of
page controls.
<<<<<<< ours
### BL-069 - The sync has no cap on one run and no last-admin guard
- **Found during:** T13.3 (D18)
- **Where:** `server/okta_sync.py` `run_sync()`
- **What:** a run disables every account the rules select, however many. If the
identity attribute is partly wrong (it matches some accounts and not others),
one run can disable most of the company, including every admin. The
zero-match guard only stops the case where nothing matches. Disabled accounts
cannot sign in, so re-enabling needs an admin who is still active, or the
`manage_users.py` command line on the host.
- **Why not now:** D18 does not ask for it, and the right limit is a product
call: a count, a share of accounts, a rule that the last active admin never
disables, or all three.
- **Suggested wave or follow-up:** before the first live run, or in T13.6 if
Matt decides. Needs Matt.
=======
## Future build: mobile app and scaling
Logged 2026-10-12 from the meeting with Nick. The answers are in
`docs/waves/decisions-2026-10-12.md` (Q2 to Q14). The path from here to a
mobile version outside the VPN, across projects, is in
`docs/waves/roadmap-mobile-scaling.md`. These are features for a future
build. They have no wave and no `CR-`/`D-` id yet. Give each one an id when a
wave is planned. Items marked "to confirm" have an open question listed in the
decision file ("Still open"). Do not build them until it is answered.
Follow-up answers from Matt on 2026-10-12 are folded into the entries below.
Server work is needed for BL-062 to BL-066 (CLAUDE.md, "Frontend and backend
boundary"). Do not use `localStorage` as the store of record.
### BL-062 - Offline Field View for crews
- **From:** Q2, Q3
- **What:** Field View opens offline for up to **7 days** after the last sign-in
(today: 12 hours). Only crews in Field View work offline. The creator has no
offline mode. When the app opens online again and the account is disabled,
wipe that account's local data.
- **Why not now:** new scope. It needs an id, and it follows the outbox work in
wave 14 (T14.1 to T14.3).
- **Suggested wave or follow-up:** future mobile wave. Check it against the D18
session rules (idle timeout, absolute ceiling): a 7 day offline window and a
session ceiling must agree on what "signed in" means.
### BL-063 - Offline edit conflicts: merge, notify, log
- **From:** Q4
- **What:** When an offline edit meets a newer server version, the app merges
the two, shows a notification to the crew, writes a log entry, and updates the
owner. If both sides changed the same field, **the field value wins**, and the
app keeps a **history of changes**. This replaces the earlier default (keep
both, crew chooses).
- **To confirm:** who "the owner" is, how long the history is kept, and who can
read it. See the decision file, "Still open" item 2.
- **Suggested wave or follow-up:** same wave as BL-062.
### BL-064 - Field photo limits
- **From:** Q5
- **What:** a limit of **5 MB per photo**. This equals the D8 limit that the
server already enforces for drawings (`FILE_MAX_BYTES`), so photos can use the
same upload path and the same D19 download headers.
- **To confirm:** the count per package, and whether the app resizes a photo
before upload (phone photos are often 3 MB to 8 MB). Decision file, "Still
open" item 1.
- **Suggested wave or follow-up:** same wave as BL-062.
### BL-065 - Batch print and merged PDF
- **From:** Q6, Q7, Q8, Q9
- **What:** print many packages in one PDF.
- Limit: **25 packages on phones**. On desktop the user can scale the filter
up to 50 or 100. Open: who sets the larger value (each user, or an admin
setting). Decision file, "Still open" item 5.
- Drawings go **after the form**, in **upload order**, **scaled to fit**.
- SVG drawings are **left out**. A page names each file that was left out.
- Every page shows the **package number, the print date, and page X of Y**.
- **Related:** BL-000d (merge or list attachments on export) asked the same
question for CR-008. This entry answers the layout part of it.
- **Suggested wave or follow-up:** future print wave.
### BL-066 - Keep drawings when a package is deleted
- **From:** Q13
- **What:** deleting a package no longer deletes its drawings. Keep them for
data retention. **The storage quota is freed** when the package is deleted.
Storage will move to cloud storage in a future deployment. This replaces the
earlier default (delete with the package).
- **Today:** drawing bytes live in the Postgres `wp_files` table (CR-007, D8).
The 2 GB per project limit counts them. Kept files must not count toward that
limit, so the count has to change.
- **To confirm:** retention period, who can delete for good, who can recover.
See the decision file, "Still open" item 3.
- **Suggested wave or follow-up:** roadmap Phase 3 (object storage). It affects
D19 (safe download) paths too: recovered files must keep the attachment and
sandbox headers.
### BL-067 - Ready for QA progress weight is 0.85
- **From:** Q12
- **What:** change the progress weight for Ready for QA from 0 to **0.85**.
Today a package falls from 75% to 0% when it reaches Ready for QA.
- **Why not now:** it changes a shipped number on dashboards and roll-ups. Use
its own item id and re-run the roll-up checks.
- **Suggested wave or follow-up:** future wave. Small task.
### BL-068 - One cost-code list, kept in the SOP
- **From:** Q14
- **What:** the creator has 38 cost codes, the SOP has 39, and 13 differ. Keep
**one list, in the SOP**, and make the creator read it.
- **Blocked on:** the correct codes. The **PM** supplies them (decided
2026-10-12). The date is not set. This is a data question and not a code
question.
- **Related:** BL-019 (closed) made a stored code that has left the list stay
as an option. Keep that behavior when the list changes.
- **Suggested wave or follow-up:** future wave, after the codes arrive.
>>>>>>> theirs

View File

@@ -0,0 +1,97 @@
# Context handoff: Work Package Suite
> Update 2026-10-16: waves 11 and 13 (T13.2 to T13.5) are on the release branch `release/waves-11-13-okta-sync`, with T13.1 held back. The branch and wave status tables below describe 2026-10-12. See `docs/waves/wave-13.md` for the current state.
Date: 2026-10-12
Purpose: Give a new chat the facts about waves 11, 12, 13, and 16.
This file is a handoff note. It is not a spec. The spec files stay the source of truth (`IMPLEMENTATION.md`, `docs/waves/*`, `CLAUDE.md`).
It was checked against `git` on 2026-10-12. Earlier chat summaries about this repo are out of date. Use this file.
## 1. Branch state
| Branch | State |
|---|---|
| `main` | At `d42ae0c` (2026-10-06). Contains wave 12 (CR-020, bulk user editing) and wave 16 (D19, D20, D21), both merged by PR. |
| `feat/waves-11-13` | Local and `origin` are the same commit, `da1325c`. It is pushed. It is not merged. It is 13 commits ahead of `main` and 19 behind. |
| `notes/cr019-usage-metrics-d18-deprovisioning` | Holds the same wave 11 work plus the session notes. |
Correction to an earlier statement: the Gitea push is no longer blocked. The branch is on `origin`.
Merge rule from `wave-12.md`: T13.1 (idle timeout) must not reach `main` before T14.1. Do not merge `feat/waves-11-13` into `main` as a whole without a decision from the user. Wave 11 and T13.1 are on the same branch.
Working tree on `main` (2026-10-12) has two untracked files: `WorkPackageSuite_Consolidated_Change_Request_R2.docx` and `docs/waves/decisions-2026-08-25.md`. Do not delete them.
## 2. What is on `feat/waves-11-13`
| Commit | Task | Item |
|---|---|---|
| `75ac930` | Planning docs for CR-019, CR-020, D18 | CR-019, CR-020, D18 |
| `0652fa7` | T11.1 UsageEvent model and migration | CR-019 |
| `850b789` | T11.2 capture usage events | CR-019 |
| `3584695`, `6cde6e3` | T13.1 idle timeout with an absolute ceiling | D18 |
| `8e863ae` | T11.3 aggregation endpoint with filters | CR-019 |
| `2de76d5` | T11.4 usage export (raw and sanitized CSV) | CR-019 |
| `a6c3fbf`, `0b5ab59` | T11.5 admin console "Activity & usage" card, checked at 390px and 1440px | CR-019 |
| `b2a083a` | T11.6 retire the per-browser Usage report | CR-019 |
| `2842ec9` | Session notes for 2026-09-23 | none |
| `4f3a94b` | T11.6 follow-up: fix `tests/autosave_check.py` | CR-019 |
| `da1325c` | T11.7 full verification pass. Wave 11 closed on 2026-09-24. | CR-019 |
Notes:
- `html/wp-usage.js` and `tests/usage_check.py` were deleted in T11.6. The `track()` functions in `wp-creation-app.js` and `work-package-suite-app.js` are now documented no-ops. About 45 call sites stay.
- T11.7 found a real regression that T11.6 caused. `tests/autosave_check.py` still expected the retired tracker to write to `localStorage`. The test was fixed. It now passes 33 of 33. The cause was that T11.6 ran only `tests/baseline_shots.py`, not the full suite. Run the full suite for any frontend task.
## 3. T11.7 verification results (2026-09-24)
- Alembic: upgrade, downgrade, upgrade on SQLite passed. The SQL render for Postgres was clean (this catches SQLite-only DDL, the "BL-027 class" of defect).
- Test scripts: 38 of 39 passed after the autosave fix.
- Two known failures that are not regressions:
- `token_check.py` exits 2. This is a CLI usage detail.
- `generalinfo_check.py` finds an old color-literal gap in `wp-creation-styles.css`.
## 4. Wave status
| Wave | Items | Status |
|---|---|---|
| 10 | Okta OIDC sign-in (D15, D16) | Done. In production. |
| 11 | CR-019 usage and activity metrics | Done on `feat/waves-11-13`. Not on `main`. |
| 12 | CR-020 bulk user editing | Done. Merged to `main` on 2026-10-06 (T12.1 to T12.6). |
| 13 | D18 Okta/AD deprovisioning sync | T13.1 done on `feat/waves-11-13`. T13.2 and later are not started. T13.3 must fail closed. Do not relax that. |
| 14, 15 | Not checked in this handoff | T14.1 must land before T13.1 reaches `main`. Read the wave files before you plan. |
| 16 | D19 safe downloads, D20 one SOP per project, D21 session decode fails closed | Done. Merged to `main` on 2026-10-06. |
Wave 16 decisions (`docs/waves/decisions-2026-10-02.md`): SEC-06 (session revocation) waits for the AWS work. Other SEC findings stay as backlog drafts in `docs/reference/security-review/backlog-drafts.md`.
## 5. Rules that apply here (`CLAUDE.md`)
- Every change traces to an item ID (`CR-`, `F`, `S`, `A`, `B`, `C`, `D`, task `T<wave>.<n>`). New scope gets a new ID.
- One task per PR. Put the task ID and item IDs in the commit message and PR title.
- Do not fix other problems that you see. Log them in `docs/waves/backlog.md`.
- Do not reorder waves. Do not start a wave before the previous wave is merged, unless the task says it is independent.
- Server work is needed for CR-004, CR-018, CR-007, CR-011, CR-014, CR-013, and B4. Do not use `localStorage` for these.
- Do not change: Actual Hours in Closeout (CR-017), localization (A7), uppercase card headers in `console.css` (A5), the logged-override path `confirmEarlyRelease()` (A1), or hidden-not-deleted fields (CR-002, CR-016).
- Token rule: one place defines color, spacing, and type. Do not add raw hex values to page stylesheets.
- Accessibility (C1): use real `<button>`, `<a>`, or input elements. Add `aria-live` for status changes. Keep focus visible. Text contrast is 4.5:1 (3:1 for large text).
- Frontend checks: run `uvicorn server.app:app` on a throwaway SQLite database. Check 390px and 1440px. Capture before and after screenshots. Run the smoke test.
- If a done-when check cannot be run, do not mark the task complete. Say which check failed and why.
## 6. Environment on the user's PC
- Python was installed with `winget`. Tests use the repo `.venv`.
- Browser tests (`tests/cdp.py` and about 40 `*_check.py` scripts) need Edge or Chrome. `cdp.py` starts from an elevated shell (wave 16 note). The Linux sandbox cannot run them.
- See `C:\Users\m.mabrey\GIT\Field Value Driver\sde-meeting-toolkit\docs\context\environment-notes.md` for tool problems (PowerShell `$` removal, long git paths, the 180 second command limit).
## 7. Open items
1. Decide how `feat/waves-11-13` reaches `main`. The T13.1 and T14.1 order is the constraint.
2. Plan the next wave. T13.2 and later need an explicit instruction.
3. Future build list added 2026-10-12: `BL-062` to `BL-068` in `docs/waves/backlog.md` (mobile app, offline Field View, batch print, and related items). Answers are in `docs/waves/decisions-2026-10-12.md` (follow-ups from Matt included). The path to a mobile version outside the VPN is in `docs/waves/roadmap-mobile-scaling.md` (phases 0 to 5; confirmed: all users in the Okta group, multiple customers, company-managed devices; nine open questions in its section 7). These files are uncommitted edits on `main`.
4. Work from the other chat (the SDE Meeting Toolkit) does not touch this repo. See the toolkit `docs/context/` folder.
## 8. Related files
- `IMPLEMENTATION.md` (sections 2 and 4)
- `docs/waves/wave-11.md`, `wave-12.md`, `wave-13.md` (on `feat/waves-11-13`), `wave-16.md`
- `docs/waves/decisions-2026-09-17.md` (CR-019, CR-020, D18), `decisions-2026-10-02.md` (D19, D20, D21)
- `docs/waves/notes-2026-09-23.md` (on `feat/waves-11-13`)
- `docs/waves/backlog.md`

View File

@@ -0,0 +1,51 @@
# Decisions: October 12, 2026 (meeting with Nick)
Topic: mobile app (offline Field View), batch print, and related scaling questions.
Recorded by Matt Mabrey from the meeting with Nick, 2026-10-12. Follow-up answers from Matt are in the section "Follow-up answers".
This file records the answers. It does not schedule any work. The features that come from these answers are logged in `docs/waves/backlog.md` as `BL-062` to `BL-068` for a future build. The path to a mobile version that works outside the VPN is in `docs/waves/roadmap-mobile-scaling.md`. None of these have a wave or a `CR-` or `D-` id yet. Assign the ids when a wave is planned.
Question numbers (`Q2` to `Q14`) come from the decision table that Matt received. `Q1`, `Q10`, and `Q11` were not in the table. They are not recorded here.
## Answers
| Q | Decision needed | Default that was proposed | Answer |
|---|---|---|---|
| Q2 | Days the app may open offline after the last sign-in (today: 12 hours) | 7 days. Wipe a disabled account's data on the next online open. | **7 days.** The default wipe rule stands. |
| Q3 | Must planners edit in the creator offline, or only crews in Field View? | Field View only | **Only crews in the field.** The creator needs no offline mode. |
| Q4 | An offline edit meets a newer server version. What does the crew see? | Keep both, show differences, crew chooses. Server wins if nobody acts. | **Merge with a notification. Log it. Update the owner.** If the same field changed on both sides, **the field value wins**, and the app keeps a history of changes. This replaces the default. |
| Q5 | Photo limit per photo and per package | Resize to 1280 px, 50 per package | **5 MB per photo.** |
| Q6 | Maximum packages in one batch print | 50, with a message to narrow the filter | **25 on phones.** On desktop the user can scale the filter up to 50 or 100. |
| Q7 | Where drawings go in the PDF, and at what size | After the form, upload order, scaled to fit | **After the form, upload order, scaled to fit.** Same as the default. |
| Q8 | SVG drawings in the merged PDF | Leave out, with a page that names the file | **Leave out, with a page that names the file.** Same as the default. |
| Q9 | What goes on every page | Package no., print date, page X of Y | **Package no., print date, page X of Y.** Same as the default. |
| Q12 | Progress weight for Ready for QA (today 0, so a package drops from 75% to 0%) | 0.85 | **0.85.** Same as the default. |
| Q13 | What happens to a package's drawings when the package is deleted | Delete with the package | **Keep them (data retention). The storage quota is freed.** Storage will move to cloud storage in a future deployment. This replaces the default. |
| Q14 | Which cost-code list is correct (creator has 38, SOP has 39, 13 differ) | One list in the SOP. Needs codes from Accounting or the PM. | **One list in the SOP.** The **PM** supplies the codes. |
## Follow-up answers (Matt, 2026-10-12)
These answer the six questions that were open after the first record.
1. **Q5:** the 5 MB limit is **per photo**. This matches the limit that already exists for drawings (D8, enforced in `server/app.py`: 5 MB a file, 2 GB per project).
2. **Q6:** 25 on phones. Desktop users can scale up (50 or 100).
3. **Q4:** on a conflict in the same field, **the field value wins**. The app keeps a history of changes.
4. **Q13:** the quota is freed. Cloud storage will hold the kept files in a future deployment.
5. **Q14:** the **PM** gives the codes.
6. **Scaling:** Matt asked for a path to a mobile version that scales to multiple projects, outside VPN access, for field agents on phones and tablets on site. That path is in `docs/waves/roadmap-mobile-scaling.md`.
7. **Who uses it:** all end users will eventually be in the Okta group.
8. **Scale:** expect multiple **customers**. The data model needs a customer (organization) level above a project. See Phase 4 of the roadmap.
9. **Devices:** company-managed.
## Still open
1. **Q5:** the count of photos per package, and whether the app resizes a photo before upload. Phone photos are often 3 MB to 8 MB. A 5 MB cap per photo with no resize will reject some normal photos.
2. **Q4:** who "the owner" is (package owner, project owner, or the person who made the server change). Also: how long the history of changes is kept, and who can read it.
3. **Q13:** how long a kept drawing is retained, who can delete it for good, and who can recover it. The quota is freed, so the retained bytes must not count toward the 2 GB project limit. Decide where they are counted instead.
4. **Q14:** the date the PM gives the codes.
5. **Q6:** who sets the larger desktop limit (each user, or an admin setting).
6. The questions in `roadmap-mobile-scaling.md`, section 7 (nine open). The first two (one person on more than one customer, and the isolation model) change the data design. Answer them first.
## Source
Meeting with Nick. Status column in the original table: "Open. Team builds to the default" for every row, as of October 12. The answers above came after that status.

View File

@@ -0,0 +1,85 @@
# Session notes — 2026-09-23
Working notes for the `feat/waves-11-13` line (CR-019, CR-020 reserved, D18),
written up before merge. Not a spec document — `wave-11.md`, `wave-13.md` and
`decisions-2026-09-17.md` are the source of truth for scope and acceptance
criteria. This is the "what actually happened building it" record.
## What shipped today
**CR-019 (wave 11) — usage/activity metrics, T11.1 through T11.6 complete:**
- `UsageEvent` table + migration (T11.1), a capture endpoint wired into
`auth-guard.js` so every protected page pings it once per load, plus a
`login` event at Okta sign-in (T11.2).
- `GET /api/usage/summary` (T11.3) — active users by day/week/month,
per-user last-active, per-tool breakdown, filterable by date/project/
user/tool, all combinable.
- `GET /api/usage/export` (T11.4) — raw and sanitized CSV. Sanitized mode
replaces the username with an HMAC-SHA256 pseudonym (keyed with
`auth.SECRET_KEY`), stable per user across rows and across separate
export calls, so an external tool (Power BI etc.) can still group by
user without ever seeing a real name.
- A new "Activity & usage" card in the admin console (T11.5): real filter
controls, the summary tables, both export buttons. Client-side gated
admin-only same as the rest of the console; the API underneath is
independently gated server-side regardless.
- Retired the old per-browser "Usage logs" panel and `wp-usage.js`
entirely (T11.6) — it had no reader left and was never a real data
source for the new report anyway. The scattered `track()` call sites in
the creator and wizard were left in place calling a documented no-op,
rather than deleting ~45 individual call sites for the same effect.
Only **T11.7 (final wave verification)** is left before wave 11 is fully
closed out — everything under it has already been verified per-task, so
this is a consolidation pass, not new work.
**D18 (wave 13) — Okta/AD deprovisioning, T13.1 only:**
- Session lifetime changed from one flat `AUTH_SESSION_HOURS` to a sliding
idle timeout (`AUTH_IDLE_MINUTES`, default 30) capped by a hard ceiling
from original sign-in (`AUTH_SESSION_HOURS`, default 8, meaning changed
from "session length" to "absolute ceiling"). Both defaults are flagged
in `.env.example` and `DEPLOYMENT.md` as proposed, not confirmed against
the tenant's actual Okta SSO policy.
- **T13.2 onward (the actual Okta Management API sync job) is paused** —
explicit call from Matt: no Okta API credential yet, come back to it
later. Not started, not blocked on anything code-side.
**CR-020 (wave 12, bulk user editing):** not started. Reserved, scoped in
`wave-12.md`, no code touched.
## Environment work (not itself a task, but load-bearing)
- Fixed a CRLF/LF mismatch that was making every tracked file look modified
to this session's git client (`core.autocrlf true`, repo-local, no file
content changed).
- This machine had no Python. Installed it via `winget` (`Python.Python.3.12`)
and set up a `.venv` in the repo with `server/requirements.txt` installed,
specifically so `tests/baseline_shots.py` could run locally — this
sandbox has no headless-capable browser and can't download one (network
allowlist), so the 390px/1440px screenshot verification CLAUDE.md asks
for had to run on Matt's own machine instead, using the browser already
installed there (Edge).
- Established a repeatable local verification loop for every task: throwaway
SQLite, fake-Okta sign-in, promote to admin, `seed_demo.py` +
`smoketest.py` (27/27 passing throughout), plus `baseline_shots.py` for
anything touching `html/`.
## Standing constraints, still in effect
- Everything stays local on this branch line. No `git push` at any point
today.
- One task per PR discipline was kept even though all of it landed on one
branch — each commit corresponds to exactly one task ID, in wave order,
each individually verified before the next started.
## Before merging
- Run T11.7 (full wave-11 verification pass) and record it in `wave-11.md`.
- Decide where this branch actually merges to — `feat/waves-11-13` has all
of today's commits already; this notes branch was cut from it so it can
fast-forward back in, or merge as its own PR if the notes should be
reviewed separately from the code.
- T13.2+ and all of wave 12 remain explicitly out of scope until Matt says
otherwise.

View File

@@ -0,0 +1,178 @@
# Roadmap: mobile version, multiple projects, access outside the VPN
Date: 2026-10-12
Status: plan for a future build. It is not a spec. No wave, `CR-`, or `D-` id exists for it yet.
Inputs: `docs/waves/decisions-2026-10-12.md` (Q2 to Q14), `BL-062` to `BL-068` in `docs/waves/backlog.md`, and the code as of `main` (`d42ae0c`).
Goal (from Matt, 2026-10-12): field agents use the app on phones and tablets on site, from outside the VPN, across more than one project.
Scope facts confirmed by Matt on 2026-10-12:
- All end users will be in the Okta group. There are no contractors without an Okta account.
- The product will scale to **multiple customers**, not only more projects for one customer.
- The devices will be **company-managed**.
## 1. Where the product is today
| Area | Fact today | Source |
|---|---|---|
| Mobile shell | A PWA already exists: `manifest.webmanifest` (standalone, "Field View" shortcut) and `sw.js` (caches the app shell and uploaded drawings). | `html/manifest.webmanifest`, `html/sw.js` |
| Offline writes | Writes queue in a sync outbox in `project-data.js`. Reads fall back to a `localStorage` cache. `/api/*` is never cached by the service worker. | `html/sw.js` header |
| Offline drawings | Drawings for the user's assigned packages are prefetched and cached (CR-007, D8). | `html/sw.js`, `html/field.js` |
| Files | The bytes live in the Postgres `wp_files` table. Limit: 5 MB a file, 2 GB per project. | `server/models.py` |
| Sign-in | Okta OIDC. A signed cookie (`HttpOnly`, `SameSite=Lax`, `Secure` over HTTPS). Idle timeout (D18, T13.1) is on `feat/waves-11-13`, not on `main`. | `server/auth.py`, `docs/waves/wave-13.md` |
| Projects | `Project` and `ProjectMember` exist. Access is by project role. There is no organization or customer level above the project. | `server/models.py` |
| Hosting | Docker Compose on an internal host. NGINX serves the site. An external proxy ends TLS. The database and backup containers have no internet route. | `docker-compose.yml`, `nginx/conf.d/wp-suite.conf` |
| Asset picker | Reads the Micron SQL Server catalog through `MICRON_DB_URL`. That database sits on the company network. | `server/assets_db.py` |
| Abuse limits | No rate limiting or login throttling was found in `server/`. CSRF origin checks and security headers (CSP, HSTS) exist. | `server/app.py`, `wp-suite.conf` |
| Security review | The 2026-10-01 review put session revocation (SEC-06) on hold until the AWS work. Wave 16 fixed SEC-01, SEC-02, SEC-07. | `docs/waves/decisions-2026-10-02.md` |
Result: the "mobile app" is mostly built as a PWA. The main work is not a new app. It is (a) finishing offline Field View, (b) making the service safe to put on the internet, and (c) removing the single-host limits.
## 2. Path in phases
Each phase has a gate. Do not start a phase before the gate of the one before it is met, unless this file says the phases are independent. Sizes are rough estimates (S under 1 week, M 1 to 3 weeks, L over 3 weeks). They are not commitments.
### Phase 0: Close the open decisions (S)
- Answer the questions in section 7 and the "Still open" list in `decisions-2026-10-12.md`.
- Assign `CR-`/`D-` ids and write the wave files.
- **Gate:** every item in `BL-062` to `BL-068` is free of "to confirm".
### Phase 1: Finish offline Field View, still inside the VPN (L)
Scope: `BL-062`, `BL-063`, `BL-064`.
1. Land the outbox work (wave 14, T14.1 to T14.3). `wave-12.md` says T13.1 must not reach `main` before T14.1.
2. Set the offline window to 7 days (Q2). Reconcile it with the D18 idle timeout and absolute ceiling. Define what "signed in" means offline. Decide what the 7 day window does when Okta says the account is disabled (wipe on the next online open).
3. Conflict handling (Q4): merge, field value wins on the same field, history of changes, a notification to the crew, a log entry, and an update to the owner.
4. Photos: capture on the device, 5 MB per photo (Q5). Add resize before upload if Matt chooses it. Store photos like drawings (server-side, with D19 download headers).
5. Add a phone and tablet test matrix (real devices): iOS Safari installed to the home screen, Android Chrome, one tablet.
6. Test the weak-link case: no signal, then a slow signal, then a sync with a conflict.
Known platform risks to test on the devices (verify; do not assume):
- iOS may delete site storage after a period of no use unless the PWA is installed to the home screen.
- iOS Safari has no Background Sync. The outbox then sends only while the app is open. A crew member must open the app to push changes.
- A lost device holds up to 7 days of project data. Web storage is not encrypted by the app. See Phase 2.
**Gate:** the crew completes a full shift offline on a test device, reconnects, and the server holds every change. The conflict case works as Q4 says. `tests/mobile_check.py` and the 390px/1440px checks pass.
### Phase 2: Make the service safe to reach from the internet (M to L)
Today the design assumes the firewall. Outside the VPN, the app becomes the perimeter. Do this before any real user connects from the internet.
| Control | Why |
|---|---|
| Rate limiting and login throttling on the API (none today) | Stops guessing and scraping. The toolkit (`sde-meeting-toolkit`) needed the same fix. |
| A WAF or reverse proxy with request limits in front of NGINX | Filters bad requests before they reach the API. |
| Real client IP handling (`X-Forwarded-For` from the trusted proxy only) | Limits and logs must use the real address, not the proxy. |
| Okta policy for the app: MFA, session length, device trust | Okta is the only sign-in path (D15, D16). The policy is the control. Ask the Okta admin. |
| Session revocation (SEC-06) | When a phone is lost, an admin must end that session now. This was put on hold for the AWS work. Bring it forward. |
| A rule for lost devices | Devices are company-managed (decided). Require a device PIN and disk encryption through the device policy. Remote wipe through the device manager. Pair it with session revocation (above), because a wipe cannot reach an offline device. Confirm with IT that the policy covers every phone and tablet that runs the app, and that a lost device is reported quickly. |
| Okta access rule | All end users are in the Okta group (decided). Gate the app on that group. Today the API creates the account on first sign-in; check that a person who is not in the group is refused, and that removal from the group ends access (this is what D18 and wave 13 do). Ask the Okta admin to add MFA and device-trust conditions for the outside path. |
| Cookie flags on every cookie (the OIDC state cookie uses `https_only=False` in `server/app.py`) | Must be `Secure` on the public site. |
| Review the 2026-10-01 findings that stayed in backlog drafts | Re-rank them for an internet-facing service. |
| Outside review (a penetration test) | Required before the customer-IP data is reachable from the internet. |
| Alerting on failed sign-ins and on odd upload sizes | Detect abuse early. |
**Gate:** the controls above are in place and an outside test finds no high findings open.
### Phase 3: Hosting that can leave the office network (L)
The AWS work is already planned (SEC-06 note). Proposed shape. The choice of services is open for Matt and the AWS owner.
- A public edge (load balancer, WAF, TLS) in front of the API and the static site.
- API containers on a managed container service, with a managed Postgres database.
- Files in object storage instead of the database (`BL-066`, Q13). Move the bytes out of `wp_files`. Keep the metadata in Postgres. Kept files from deleted packages go to a retention area with a lifecycle rule. The quota counts only live files.
- Backups and restore tests on the new stack (the current backup container writes to a host folder).
- A private link back to the Micron asset database, or a different design for the asset picker (the catalog is on the company network). Until this exists, the picker falls back to manual entry outside the VPN. Check that fallback works for a user who is online but cannot reach the database (the code treats "unconfigured" as a normal state, but not "unreachable").
- Infrastructure as code, so the environment can be rebuilt.
- Keep the internal deployment working until the cutover.
**Gate:** a full restore test passes, the migration runs on Postgres (render the SQL for the Postgres dialect first, BL-027), and a rollback plan is written and rehearsed.
### Phase 4: Many projects and many customers (L)
Matt confirmed multiple customers. A customer boundary is now part of the plan, not an option. The data model has `Project` and `ProjectMember` but nothing above a project, so this phase changes the base design. Plan for it early. Phase 3 choices (storage layout, database layout) must already allow it.
- Add an organization (customer) level above `Project`. Every project, user role, file, report, export, and search belongs to one organization. A user may belong to more than one organization (for example Prime Controls staff who work for several customers). Decide this rule first (section 7, question 1).
- Isolation: choose the model before any code. Options: one database with an organization id on every row and checks on every query; a separate schema per customer; or a separate database or stack per customer. Matt and the AWS owner decide. The choice sets cost, backup, and the strength of the separation that a customer can be shown.
- Files: one storage prefix per customer, with its own access rule and, if a customer asks, its own encryption key.
- Customer-specific data: the cost-code list, the asset catalog (today Micron only), and the SOP templates must belong to the organization or the project. `BL-068` (one cost-code list in the SOP) must not become one list for all customers.
- Okta: map Okta groups to organizations and projects, so the Okta admin controls who sees which customer.
- Reports and the admin console: super users see only the organizations they are given. Check CR-019 usage reports, the user directory, and the bulk user tools from wave 12 against this rule.
- Onboarding a customer: a runbook with the steps and a checklist.
- Project onboarding: create a project, load its SOP, cost codes (`BL-068`, one list in the SOP, codes from the PM), people, and roles. Use the bulk user tools from wave 12.
- Okta group to project membership mapping, so the Okta admin controls who joins a project.
- Per-project storage and photo quotas (2 GB per project exists today) and per-project usage reports (CR-019 on `feat/waves-11-13`).
- Performance work with a load test: many crews syncing at shift start. Check the outbox flush, the `wp_files` move, and the list endpoints on a large project.
- Data retention rules per project.
**Gate:** two real customers run side by side. A test user in one customer cannot reach any data of the other through any screen, report, export, search, file link, or API call. A test that tries to cross the boundary on purpose passes. The load test meets targets that Matt sets.
### Phase 5: Native wrapper (only if needed) (M)
Wrap the PWA in a native shell (for example Capacitor) only if Phase 1 testing shows a platform limit that the PWA cannot meet: storage eviction, no background sync, camera control, or a company rule for app-store distribution. A wrapper adds app-store review, signing, and update cost. Do not start it by default.
## 3. Order and dependencies
```
Phase 0 -> Phase 1 -> Phase 2 -> Phase 3 -> Phase 4
\_ Phase 5 (decide after Phase 1 device tests)
```
- Phase 2 may start in part while Phase 1 runs (rate limiting, Okta policy review). No crew uses the internet path before Phase 2 is done.
- Phase 3 (storage move) and `BL-066` (retained drawings) are the same work. Do them together.
- Phase 4 needs the Phase 2 and Phase 3 gates. Opening a second project to the internet first would repeat any gap twice.
## 4. Work that this path adds to the backlog
| Id | Item | Phase |
|---|---|---|
| BL-062 to BL-068 | Already logged | 1, 3, 4 |
| New | API rate limiting and login throttling | 2 |
| New | Real client IP handling behind the proxy | 2 |
| New | Session revocation (SEC-06) brought forward | 2 |
| New | Device policy check with IT (PIN, encryption, remote wipe) | 1, 2 |
| New | Okta group gate and MFA/device-trust conditions for the outside path | 2 |
| New | Object storage and lifecycle rules for files, one prefix per customer | 3 |
| New | Asset picker when the Micron database is unreachable | 3 |
| New | Organization (customer) level above `Project`, with the isolation model chosen first | 3, 4 |
| New | Per-customer cost codes, asset catalog, and SOP templates | 4 |
| New | Okta group to organization and project mapping | 4 |
| New | Cross-customer access tests (a test that tries to cross the boundary) | 4 |
| New | Customer onboarding runbook | 4 |
| New | Load test and capacity targets | 4 |
Log these in `docs/waves/backlog.md` when Matt agrees with the plan. This file does not add them, so the backlog stays free of unconfirmed scope.
## 5. Risks
1. **Customer data on lost devices.** Offline data for 7 days on a phone or tablet. Devices are company-managed, which lowers this risk a lot, but only if the device policy (PIN, disk encryption, remote wipe) is real and covers every device. An offline device cannot receive a wipe until it connects. Keep the 7 day window only if IT confirms the policy.
2. **Customer boundary added late.** The data model has no organization level. If Phases 1 to 3 are built without it, every later change touches every table and every query. Choose the isolation model in Phase 0 and design Phase 3 storage around it.
3. **Contract terms.** Customer drawings and asset data may have rules about where they are stored and who can reach them. With several customers, each contract may differ. Check before Phase 3 (cloud storage) and before Phase 2 (internet access).
4. **Okta.** The app has no password of its own. A weak Okta policy for the outside path is a weak app. The Okta admin owns that setting.
5. **Single database host.** The current database is one container with a host-folder backup. Phase 3 fixes this. Until then, do not add projects that need high availability.
6. **Sequence risk.** T13.1 (idle timeout) must wait for T14.1 (outbox). A short idle timeout can end a session while a crew is offline. Settle the rule in Phase 1.
## 6. What does not change
These decisions stay as recorded: Actual Hours stays (CR-017), localization stays (A7), uppercase card headers stay (A5), the logged-override path for predecessors stays (A1), removed fields are hidden and not deleted (CR-002, CR-016), and the token rule for CSS. Mobile work follows the accessibility rules (C1): real buttons, visible focus, `aria-live` for status, hover is not enough on tablets.
## 7. Questions that decide the plan
Answered on 2026-10-12:
- Who are the outside users? **All end users will be in the Okta group.** No separate sign-in design is needed.
- What does "multiple projects" mean? **Multiple customers.** See Phase 4.
- Which devices? **Company-managed.**
Answer these before Phase 0 ends. The first four are new and follow from the answers above.
1. **Can one person work for more than one customer?** For example, a Prime Controls planner on two customers' projects. This sets how the organization level and the Okta groups map to each other.
2. **Which isolation model?** One database with an organization id on every row, a schema per customer, or a database or stack per customer. Do any customers already require a separate stack in their contract?
3. **Who manages the Okta groups?** One group for all users, or one group per customer or project? Does the Okta admin accept that mapping?
4. **What does the device policy enforce today?** PIN, disk encryption, remote wipe, and a time limit for a device to check in. Ask IT.
5. **Connectivity on site.** No signal for days, or weak signal that comes and goes? This sets how long the offline window must really be, and how big the photo queue gets.
6. **AWS.** Is the AWS move approved? Who owns the account? Which region? Can the Micron database be reached from it?
7. **Contract.** Can customer drawings and asset data be stored in cloud storage and reached from the internet? Does each customer have its own terms?
8. **Distribution.** Is an installed PWA enough, or does the company need an app-store app? Company-managed devices can often install a web app from a device policy. Ask IT.
9. **Targets.** How many customers, projects, and users, and how many crews syncing at the same time, in the first year?

280
docs/waves/wave-11.md Normal file
View File

@@ -0,0 +1,280 @@
# Wave 11 — Usage and activity metrics
**Items:** `CR-019`
**Depends on:** wave 10 merged (it is; this wave does not wait on anything else)
**Decision record:** `docs/waves/decisions-2026-09-17.md`
Seven tasks, one concern each, in build order. Do not start a task whose
dependency is not merged. `CR-020` (bulk user editing) is reserved but not
scoped — it does not belong in this wave.
---
### T11.1 — CR-019: `usage_events` table + migration
- **Items:** `CR-019`
- **Depends on:** nothing (first task)
- **Blocks:** T11.2
- **Surface:** `server/`
- **Files:** `server/models.py`, `server/alembic/versions/`
**Do:** Add a `UsageEvent` model — append-only, same spirit as `AuditLog` but for
navigation/feature-open events rather than business mutations. Suggested shape:
`id`, `at` (indexed), `user_id` (or username — match whatever `AuditLog.actor`
does today for consistency), `project_id` (nullable — not every event is
project-scoped, e.g. opening the admin console), `tool` (e.g. `creator`,
`wizard`, `field_view`, `dashboard`, `admin`, `directory`), `event` (e.g.
`page_open`, `login`), `detail` (JSON, optional). Write the migration. Do not
touch `AuditLog` — this is a new table, not an extension of it (see the
decision record's reasoning).
**Do not:** fold this into `AuditLog`. They serve different questions and mixing
them makes the existing audit trail noisier for its existing readers.
**Done when:**
- [ ] `UsageEvent` exists with an indexed `at` column (this table will be scanned
by date range constantly)
- [ ] migration applies cleanly against both SQLite (dev) and Postgres (prod) —
render `alembic upgrade --sql` for postgresql and read it before calling
this done, per the class of defect `BL-027` logged
- [ ] no change to `AuditLog`'s shape or behavior
---
### T11.2 — CR-019: capture the events
- **Items:** `CR-019`
- **Depends on:** T11.1
- **Blocks:** T11.3
- **Surface:** `server/` + `html/`
- **Files:** `server/app.py` (new endpoint), `html/auth-guard.js`
**Problem:** Six pages need this and none should implement it separately — that
is exactly how `S4`'s "no global nav on two pages" and the four parallel token
systems (`S5`) happened. `auth-guard.js` is already loaded first, in the `<head>`,
on all six protected pages (`index.html`, `field.html`, `users.html`,
`wp-creation-index.html`, `work-package-suite.html`, `admin.html`) and already
knows the verified user once the `wp-auth-ready` event fires. That is the one
place this belongs.
**Do:** Add a small `POST /api/usage/ping`-style endpoint that writes one
`UsageEvent` row per call, keyed to the session (server trusts the session, not
anything the client claims about identity). Call it once from `auth-guard.js`
after `wp-auth-ready`, tagging `tool` from the page's own path. Also write a
`login` event at the point a session is actually established (reuse whatever
`okta_callback` already does at sign-in — do not add a second source of truth
for "did this person log in").
**Do not:** build a per-page capture call. If a page needs this and
`auth-guard.js` does not cover it, fix `auth-guard.js`, not the page.
**Done when:**
- [ ] one `page_open` event is recorded for a real sign-in on each of the six
pages, verified per page
- [ ] exactly one `login` event per Okta sign-in, not one per page load after
it
- [ ] the endpoint rejects a request with no valid session (this is server-
enforced identity, not client-reported)
- [ ] no page other than `auth-guard.js` calls this endpoint directly
---
### T11.3 — CR-019: aggregation endpoint with filters
- **Items:** `CR-019`
- **Depends on:** T11.2
- **Blocks:** T11.4, T11.5
- **Surface:** `server/`
- **Files:** `server/app.py`
**Do:** Build the read side: active-user counts by day/week/month, per-user
last-active timestamp (derived from `UsageEvent`, not `User.last_login_at`,
which only ever holds one value), and a per-tool usage breakdown. Accept query
filters: date range, project, user, tool — combinable, per the decision record.
This is server aggregation, the same principle `B4` established for the
pipeline strip: the browser asks for a number, the server computes it from real
rows, nothing is derived client-side from a partial cache.
**Done when:**
- [ ] active-user counts are correct against a seeded fixture with known dates
- [ ] filters combine correctly (verified: user + date range + tool together
narrows correctly, not just each alone)
- [ ] a project filter that matches nothing returns an empty result, not an
error or the unfiltered total
---
### T11.4 — CR-019: export, raw and sanitized
- **Items:** `CR-019`
- **Depends on:** T11.3
- **Blocks:** T11.5
- **Surface:** `server/`
- **Files:** `server/app.py`
**Do:** A CSV export endpoint over the same filtered query T11.3 exposes.
Two modes: raw (real usernames, the console's default) and sanitized. Sanitized
mode replaces the actor field with a stable pseudonymous id — a per-user hash,
consistent across rows in the same export and across separate export runs —
so an external system (Power BI or similar) can still group and trend "by
user" without ever receiving a real name. Do not simply drop the identity
column; that breaks per-user grouping downstream, which defeats the point of
an activity export.
**Done when:**
- [ ] raw export contains real usernames
- [ ] sanitized export never contains a real username or email anywhere in the
file, including in a `detail` blob if one is included
- [ ] the same real user maps to the same pseudonymous id within one export AND
across two separate export runs (a hash of something stable, not a
per-request random id)
- [ ] both modes otherwise contain identical rows for the same filter
---
### T11.5 — CR-019: admin console Activity tab
- **Items:** `CR-019`
- **Depends on:** T11.3, T11.4
- **Blocks:** T11.7
- **Surface:** `html/`
- **Files:** `html/admin.html`, `html/admin.js`
**Do:** New tab, same role gate as the User Directory. Filters (date range,
project, user, tool) driving the tables from T11.3; export buttons (raw and
sanitized) calling T11.4. Build accessible from the start per `C1` — this is a
new component, not a legacy one carrying an old defect forward: real
`<button>`/`<select>` controls, keyboard-reachable, `aria-live` on any
count that updates without a page reload, focus visible throughout.
**Done when:**
- [x] the tab is reachable only by an admin (the card lives inside admin.html,
already gated client-side by gateByRole(); the API underneath it is
independently gated server-side by require_user_manager regardless)
- [x] every filter is a real form control, keyboard-operable (date/select/text
inputs and a `<button>`, no click-div)
- [x] both export buttons produce the files T11.4 defines (verified against
the live endpoint in T11.4's own checks, and present/wired here)
- [x] works at 390px and 1440px — verified 2026-09-23 via
`tests/baseline_shots.py --pages admin` run locally on Windows (this
sandbox has no headless browser available; the script was run on the
user's machine instead, after installing Python via winget since it
wasn't present). Screenshots in `docs/reference/baseline/admin-390.png`
/ `admin-1440.png`. No JS errors, no horizontal overflow at either
width; the card rendered with real seeded data (events, by-tool,
per-user-last-active tables) confirming the filters and summary read
correctly, not just that the markup exists.
---
### T11.6 — CR-019: retire the per-browser Usage report
- **Items:** `CR-019`
- **Depends on:** T11.5
- **Blocks:** T11.7
- **Surface:** `html/`
- **Files:** `html/admin.js` (the `usage-admin` panel, `admin.js:666-699`),
`html/wp-usage.js` and its two call sites
**Do:** Remove the old per-browser `usage-admin` panel from `admin.js` now that
the real one exists, per the 2026-09-17 decision. Decide what happens to
`wp-usage.js`'s recording calls (wizard dwell-tracking, the creator's
equivalent): they were never reliably tied to a real identity, so they are not
a data source the new report can adopt. Default to removing the recorder too
unless it is still doing something useful on its own (re-read what it actually
records before deciding — do not assume from this file alone).
**Do not:** leave the old panel in place "just in case." Two activity reports
showing two different numbers is worse than one.
**Decision (2026-09-23):** `wp-usage.js` is removed, not kept — it had no
reader left once the admin panel above it was removed (the "download the
full event log" button lived only in that panel), and per the original
decision record it was never reliably tied to a real identity, so it was
never a candidate source for the new report either. The file itself and its
three `<script>` includes (`admin.html`, `wp-creation-index.html`,
`work-package-suite.html`) are gone. Its two call sites
(`wp-creation-app.js`, `work-package-suite-app.js`) keep a local `track()`
function as a documented no-op rather than having each of their ~45
individual `track('event', …)` call sites deleted one at a time — that
would be a far larger, riskier diff for the same outcome (no more data is
recorded either way), and it keeps each call site as a marker of what was
worth recording if usage analytics are ever rebuilt server-side. The
dwell-timer plumbing that only ever fed `track()` (`work-package-suite-app.js`'s
`_stepEnter`/`trackStepDwell`) was left in place for the same reason — it is
inert now, not broken, and touching it buys nothing.
Also removed: `tests/usage_check.py` (tested exactly the retired feature —
D5/T7.10's per-browser analytics core and admin report) and its line in
`docs/reference/file-map.md`, replaced with a note pointing at this decision.
**Done when:**
- [x] the old `usage-admin` panel and its markup are gone from `admin.html`/
`admin.js`
- [x] a decision on `wp-usage.js` itself is recorded (removed, or kept with a
stated reason) — not left ambiguous — see above
- [x] nothing else in the app references the removed code; grep confirms
(only remaining hits are this file, the 2026-09-17 decision record, and
the two explanatory code comments left at the retired call sites — all
prose, not live references)
---
### T11.7 — CR-019: verification
- **Items:** `CR-019`
- **Depends on:** T11.6
- **Blocks:** nothing
- **Surface:** `html/` + `server/`
- **Files:** as touched above
**Do:** Full verification per `CLAUDE.md`: run the app locally, exercise the
new tab at 390px and 1440px, before/after screenshots, run the existing smoke
test and `seed_demo.py`, run the full suite.
**Done when:**
- [x] all `CR-019` acceptance criteria in `decisions-2026-09-17.md` are met or
a failure is stated with a reason — all met
- [x] screenshots committed — `admin-390.png`/`admin-1440.png` (and
`creator`/`sop` at both widths, touched by T11.6's script-tag removal)
- [x] smoke test and `seed_demo.py` both still pass — 27/27, verified
2026-09-24 against the exact branch state about to be considered
deployable
- [x] full test suite passes — verified 2026-09-24: ran all 39
`tests/*_check.py` scripts plus the `usage_events` migration
round-trip. Result: 38/39 exit 0.
- `token_check.py` exits 2 with no arguments by design (needs `--out`
or `--compare`) — a CLI-invocation detail, not a test failure;
already confirmed non-regression in wave 10.
- `generalinfo_check.py` has one pre-existing failure (a raw colour
literal in `html/wp-creation-styles.css`, a file no wave 11/13 task
touched) — already diagnosed as pre-existing in wave 10.
- `autosave_check.py` **did** regress from T11.6 (it asserted the
retired analytics tracker still wrote to localStorage) — found by
this run, fixed same day (commit `4f3a94b`), now 33/33.
- Migration (T11.1): SQLite upgrade → downgrade → upgrade round-trip
clean; `alembic upgrade ... --sql` against the `postgresql` dialect
renders valid DDL (`TIMESTAMP WITH TIME ZONE`, `JSON`, no
SQLite-only syntax) — no `BL-027`-class defect.
---
## Wave 11 exit criteria
- [x] real, server-side activity data exists per user, indefinitely retained
- [x] the admin console shows it, filterable by date/project/user/tool
- [x] export works in both raw and sanitized form
- [x] the old per-browser report is gone, not duplicated
- [x] `CR-019` fully accounted for, no open acceptance criteria
Wave 11 is closed out as of 2026-09-24. `CR-020` (wave 12) remains reserved
and unstarted. `D18`/wave 13 (`T13.2` onward) remains paused pending an Okta
API credential, per standing instruction — `T13.1`'s session changes were
included in this same verification pass and did not regress anything above.

330
docs/waves/wave-13.md Normal file
View File

@@ -0,0 +1,330 @@
# Wave 13 — Okta/AD deprovisioning sync
**Items:** `D18`
**Depends on:** wave 10 merged (it is). Not blocked by wave 11 or wave 12, but
shares `server/app.py` account-state surface with wave 12 (`CR-020`) — sequence
commits to avoid an avoidable conflict.
**Decision record:** `docs/waves/decisions-2026-09-17.md`
Six tasks, in build order. **T13.3's fail-closed behavior is the most
important done-when list in this wave — do not relax it to ship faster.**
---
### T13.1 — D18: idle timeout with an absolute ceiling
- **Items:** `D18`
- **Depends on:** nothing (first task, independent of the rest)
- **Blocks:** nothing
- **Surface:** `server/`
- **Files:** `server/auth.py`, `server/app.py` (`auth_gate` middleware),
`server/.env.example`, `DEPLOYMENT.md`
**Revised 2026-09-23** — originally just "shrink `AUTH_SESSION_HOURS`". Matt
asked whether an idle timeout would be a better fit than a flat session
length. It is, but not by itself — see the decision record's reasoning on why
idle-with-no-ceiling is actually a worse fit for this item's own threat model
than a flat expiry would have been. Build both.
**Do:**
- Add `login_at` to the JWT payload in `create_token()` — the original
sign-in time, distinct from `iat`, which becomes "when THIS token was
issued" once tokens start getting reissued. `login_at` never changes across
reissues; it's what the absolute ceiling is measured from.
- Add `AUTH_IDLE_MINUTES` (default 30). `AUTH_SESSION_HOURS` stays the name
for the absolute ceiling, default changing from 12 to a proposed 8 — update
its docstring/comment in `auth.py` and `.env.example`, since its MEANING is
changing (session length -> hard ceiling on top of a sliding idle window),
not just its value.
- In `auth_gate` (`server/app.py`), after confirming a request is
authenticated: if `now < login_at + AUTH_SESSION_HOURS` (the ceiling hasn't
passed), compute `new_exp = min(now + AUTH_IDLE_MINUTES, login_at +
AUTH_SESSION_HOURS)`. If `new_exp` is meaningfully later than the current
token's `exp` (throttle this — do not reissue on every single request, only
when enough time has passed to be worth a new cookie; a few minutes of
slack is fine), mint a refreshed token carrying forward `sub`/`username`/
`role`/`ver`/`login_at` unchanged, and set it on the response.
- If the ceiling HAS passed, do not refresh — let the existing token expire
on its own terms (it may already be invalid, or may tick over within the
idle window; either way, no new one is issued past the ceiling).
- The middleware should not need a DB round trip to do this — everything
needed (`sub`, `username`, `role`, `ver`, `login_at`) is already in the
validated claims. `get_current_user`'s existing per-request `is_active`/
`token_version` check is unaffected and still runs separately.
**Do not:** reissue the cookie on every request unconditionally — that's a
`Set-Cookie` header on every API call for no benefit over a coarser refresh.
Do not let the ceiling check silently vanish for tokens issued before this
change ships — a token with no `login_at` claim should fall back to treating
its own `iat` as `login_at`, not bypass the ceiling entirely.
**Done when:**
- [x] a session with continuous activity stays alive past 30 minutes but is
cut off at the `AUTH_SESSION_HOURS` ceiling regardless
*(2026-10-16, T13.6: a request every minute for 10 hours: alive well past 30 minutes, never extended past login + 8 h, cut off at the ceiling, original `login_at` kept.)*
- [x] a session with no activity for 30+ minutes is rejected on its next
request
*(2026-10-16, T13.6: an idle token gets 401 on a gated route; one inside the window gets 200.)*
- [x] the cookie is not rewritten on every single request — verify the
refresh is throttled, not unconditional
*(2026-10-16, T13.6: 20 rapid requests on a fresh session: 0 rewrites; 480 requests over 8 h: about one rewrite per 10 minutes; a refreshed cookie is not rewritten again on the next request. Mutation test: removing the throttle fails 3 checks.)*
- [x] a pre-existing token with no `login_at` claim (simulating a session
issued before this shipped) still gets a hard ceiling, via the `iat`
fallback
*(2026-10-16, T13.6: a token with iat 9 h ago and a still-valid old exp is not extended; one with iat 1 h ago is refreshed with `login_at` taken from `iat`; no `login_at` and no `iat` is not refreshed and does not crash. Mutation test: giving it a fresh ceiling fails.)*
- [x] `.env.example` and `DEPLOYMENT.md` explain both variables and flag both
defaults as proposed, not confirmed against Okta's own session policy
*(2026-10-16, T13.6: both variables are in both files, and both call the defaults proposed.)*
- [x] existing session/auth tests updated for the new mechanism, not just a
new number
*(2026-10-16, T13.6: there were none. T13.1 shipped with no test of its own. `tests/session_idle_check.py` is that test (T13.6 added it).)*
---
**Found while writing that test (T13.6, 2026-10-16):**
- Nothing under `/api/auth/` slides the session. `auth._EXEMPT_PREFIXES` covers
that whole prefix, so `auth_gate` never refreshes there (including
`/api/auth/me`). A page that only called `/api/auth/me` would not extend its
session. The spec says "after confirming a request is authenticated", and these
routes are not confirmed by the gate, so this is as written. Noted, not changed.
- The wave's exit criteria below say `AUTH_SESSION_HOURS` defaults to **2**. T13.1
(this section), `auth.py`, `.env.example` and `DEPLOYMENT.md` all say **8**. The
code and the T13.1 text agree, so the exit line looks like a leftover from
before the 2026-09-23 revision. Not changed here: it is a product number. Needs
Matt to say which one he means.
### T13.2 — D18: Okta Management API client
- **Items:** `D18`
- **Depends on:** nothing (independent of T13.1)
- **Blocks:** T13.3
- **Surface:** `server/`
- **Files:** new `server/okta_sync.py` (or extend `server/okta_auth.py` —
task's call), `server/.env.example`
**Do:** A small client for Okta's user-list endpoint, authenticated with a new,
separate credential (e.g. `OKTA_API_TOKEN`) — not the OIDC client secret used
for sign-in. Fetch the full user list (paginated per Okta's API) rather than
one-by-one lookups per local user; this app has ~20-odd accounts today, and a
list-and-diff is simpler and cheaper than N calls. Return each Okta user's
identity-claim value (matching `OKTA_IDENTITY_CLAIM`, already confirmed live
per wave 10) and status.
Follow the existing pattern for external credentials in this repo (`MICRON_DB_URL`,
`SMTP_PASSWORD`): env-only, never logged, never returned to the browser in an
error message.
**Do not:** reuse `OKTA_CLIENT_ID`/`OKTA_CLIENT_SECRET` for this. Sign-in and
the management API are different trust boundaries with different scopes;
conflating them means a compromise or rotation of one affects the other
unnecessarily.
**Done when:**
- [x] the client authenticates with its own credential, distinct from the OIDC
client
*(2026-10-16: `OKTA_API_TOKEN`, sent as `Authorization: SSWS`. `tests/okta_sync_check.py` sets a different OIDC client secret and asserts it is never sent.)*
- [x] it fetches the complete Okta user list, handling pagination
*(2026-10-16: follows `Link: rel="next"`, 200 per page, 2-page case checked. A next link to another host is refused, a chain longer than 50 pages raises instead of cutting the list off, redirects are never followed.)*
- [x] an auth failure or malformed response raises a clear, specific error
rather than returning an empty list indistinguishable from "everyone was
deprovisioned" — this distinction is what T13.3 depends on
*(2026-10-16: `OktaSyncError` for HTTP 401/403/404/429/5xx, redirects, network errors, timeouts, non-JSON, non-list, a bad item (never skipped), and an empty list. 31 failure checks. Mutation test: making the client return `[]` on an error fails 10 of them.)*
- [x] the credential is never logged or surfaced in any API response
*(2026-10-16: error messages carry the class name and HTTP status only, never the URL or a header. `describe()` has no token. Nothing is wired to an API response yet; T13.4 wires the startup log.)*
---
### T13.3 — D18: the sync job
- **Items:** `D18`
- **Depends on:** T13.2
- **Blocks:** T13.4
- **Surface:** `server/`
- **Files:** `server/okta_sync.py`, `server/app.py` (or wherever `log_event`
lives)
**Do:** Compare the Okta user list (T13.2) against local `users` rows. For any
local `is_active=True` user whose Okta identity is missing from the list, or
present with a non-active status, set `is_active=False` and write an
`AuditLog` row (`actor="system:okta_sync"`, action e.g.
`user_deprovisioned_by_sync`, detail naming the Okta status found). Never
touch a user already `is_active=False`. Never re-enable anyone.
**The fail-closed rule, non-negotiable:** if T13.2's client raises an error of
any kind (network, auth, malformed response, timeout), this task takes **no
action on any account** for that run and logs the failure clearly (server log,
at minimum). An error is never treated as "Okta returned zero active users."
Write a test that asserts this directly: feed the sync a failing client and
assert zero rows changed and zero `AuditLog` entries written.
**Done when:**
- [x] a user missing from Okta's list, or present but not active, is disabled
with a correctly-detailed audit row
*(2026-10-16: see "Decided 2026-10-16" below for which statuses count as "not active". Audit row: actor `system:okta_sync`, action `user_deprovisioned_by_sync`, `entity_type` user, `detail` `{okta_status, via}`; a missing user is recorded as `MISSING`.)*
- [x] a user already disabled is left alone (no duplicate audit row each run)
*(2026-10-16: a second run over the same data writes no new rows.)*
- [x] an active Okta user already active locally produces no audit row (only
changes are logged, not a clean bill of health every cycle)
*(2026-10-16: verified in `tests/okta_sync_job_check.py`.)*
- [x] **a simulated Okta API failure results in zero account changes and zero
audit rows** — this is the one check that must never be skipped or
weakened
*(2026-10-16: 10 stand-in failures (OktaSyncError, RuntimeError, ConnectionError, TimeoutError, ValueError, KeyError, empty list, None, two kinds of garbage), 4 through the real client (HTTP 401/403/429/500), the default path with no token, a failed commit, and a wrong identity attribute. Each: `ok` is False, zero accounts changed, zero audit rows. Mutation test: removing the fail-closed branch and the zero-match guard fails 28 of these.)*
- [x] the job never re-enables an account under any input
*(2026-10-16: a disabled account that is ACTIVE in Okta stays disabled, and `okta_sync.py` contains no assignment of `is_active` to True.)*
---
**Decided 2026-10-16 (Matt, in chat):** which Okta statuses disable an account.
The spec says "a non-active status". Read literally, that also catches
`PASSWORD_EXPIRED`, `LOCKED_OUT` and `RECOVERY`, which people leave on their
own, and this job never re-enables, so one AD password expiry would lock someone
out of the app until an admin stepped in. Decided: an account disables when it is
**missing** from Okta's list, or every Okta entry for it is **SUSPENDED** or
**DEPROVISIONED** (`DISABLING_STATUSES` in `server/okta_sync.py`). Okta's default
user list already leaves out DEPROVISIONED users, so those arrive as "missing".
**Added 2026-10-16, inside decision 4's "anything the sync can't confidently
parse":** if there are active local accounts and **none** of them matches any
Okta user, the run is refused and nothing changes. That pattern means the
identity attribute is wrong (`OKTA_SYNC_PROFILE_ATTRIBUTE`), not that the whole
company left. It is a guard, not a new feature. Raise it in the PR.
**Open, for the PR / Nick:** which Okta profile attribute equals the
`preferred_username` claim in this tenant. The client assumes `profile.login`,
Okta's default. Confirm before the first live run. A wrong value is caught by the
zero-match guard above only when it matches nobody at all; a value that matches
some accounts and not others would disable the rest. See `BL-069`.
### T13.4 — D18: run it on a schedule
- **Items:** `D18`
- **Depends on:** T13.3
- **Blocks:** T13.6
- **Surface:** `server/`, `docker-compose.yml`
- **Files:** `server/app.py` (startup hook) or a new sidecar per the `backup`
container's pattern — task's call, per the decision record's noted
alternative
**Do:** Wire T13.3 to run on an interval (proposed 15 minutes, env-overridable
— e.g. `OKTA_SYNC_INTERVAL_SECONDS`, matching `BACKUP_INTERVAL_SECONDS`'s
naming). Default choice is an in-process background task in the `api`
container; if a separate container is chosen instead, follow the `backup`
service's shape (own Dockerfile or script, `internal` network plus whatever
egress reaching Okta requires — check whether `outbound` as currently defined
is sufficient or Okta needs a distinct allowance).
**Done when:**
- [x] the job runs automatically on the configured interval without manual
invocation
*(2026-10-16: a daemon timer thread starts from the app's lifespan hook (`server/app.py`), checks once a minute, and ran the sync on its own in `tests/okta_sync_schedule_check.py`. With `OKTA_API_TOKEN` unset it starts nothing and the startup log says `Okta sync DISABLED`. Booted the real app both ways; the thread starts and ends with the app.)*
- [x] interval is env-configurable with a sane default
*(2026-10-16: `OKTA_SYNC_INTERVAL_SECONDS`, default 900. A non-number uses the default. A value under 300 is raised to 300. Passed through `docker-compose.yml`.)*
- [x] a container restart does not produce a duplicate/overlapping run, and a
slow cycle does not stack with the next one
*(2026-10-16: the api runs 2 gunicorn workers, so each starts its own timer. One row in `app_settings` (key `okta_sync`) is the gate: a worker claims a cycle with one atomic conditional UPDATE, so exactly one wins. Checked with 8 racing workers on a file database, 15 rounds plus the first-insert race: exactly 1 winner every time. The row outlives a restart, so a restart inside the interval starts nothing. A slow run cannot overlap the next: a fetch is capped at 120 s in total and the interval cannot go below 300 s. Inside one process the thread runs one cycle at a time. This needed no migration: `app_settings` already exists.)*
---
**Decided 2026-10-16 (task's call, per this task's "Do"):** in-process timer in the
`api` container, as D18 proposed, not a sidecar. The `backup` pattern was the
alternative; it would need its own image and would not remove the 2-worker
duplicate question, because the sync reads the application database anyway.
**Network check, as the task asked:** the `outbound` network the `api` service
already joins is an ordinary bridge with a gateway, so Okta is reachable with no
network change. The host firewall must allow https to the Okta org.
**One change to T13.2's code:** `fetch_users()` now also stops after 120 seconds in
total (`TOTAL_DEADLINE_SECONDS`), which is what lets the scheduler promise no
overlap. Tested in `tests/okta_sync_check.py` section 8.
**T13.4 follow-up, 2026-10-16:** the sync's INFO log lines did not appear in
`docker compose logs api`. Nothing in the app configures Python logging, and
gunicorn configures only its own loggers, so INFO from any `wpsuite.*` logger is
dropped. The T13.4 check booted the app in-process with logging switched on,
which hid it. With the exact Dockerfile launch (2 gunicorn workers), a healthy
start printed zero Okta lines. `okta_sync.py` now gives its own logger one stderr
handler (only when the root logger has none). Re-run with the real gunicorn
command: `configured` and `scheduled` (or `DISABLED`) now print once per worker.
No other module's output changed. App-wide logging stays a separate question
(BL-028 is the same family).
### T13.5 — D18: admin visibility
- **Items:** `D18`
- **Depends on:** T13.3
- **Blocks:** T13.6
- **Surface:** `html/`
- **Files:** `html/users.js` / `html/admin.js` (wherever audit history is
already surfaced)
**Do:** Confirm an auto-disable reads clearly wherever admins already look at
account history — the actor string (`system:okta_sync`) should be
self-explanatory in context, not require reading server logs to understand.
Do not build new UI beyond making sure the existing audit surface renders this
actor sensibly. Email/notification to admins on auto-disable is a noted
fast-follow (the decision record flags it as open, not required here) — do not
build it in this task; log it instead if it's tempting to add.
**Done when:**
- [x] an auto-disabled account's audit entry is visible and legible in the
existing admin UI without special-casing
*(2026-10-16: run on the PC against the real page, 1440px and 390px, `tests/okta_sync_ui_check.py` 28/28. The admin Activity log card reads: Who `system:okta_sync`, Action `user deprovisioned by sync`, Type `user`, Item the username, Detail `okta_status: SUSPENDED, via: okta_sync` (a person missing from Okta reads `okta_status: MISSING`). Search for `okta_sync` and the User accounts filter both find the rows. No sideways scroll, no JavaScript errors. The disabled account is refused on its very next request; an account still active in Okta is not. A project super user does not see these rows: they belong to no project, so only admins do, the same as other user-management events. No code in `html/` changed.)*
- [x] nothing here silently assumes `CR-019`'s activity view exists yet — this
must work standalone
*(2026-10-16: the check reads only the Activity log (AuditLog) card. The new Activity & usage card is on the page but is not read.)*
---
**No UI was built.** The existing card renders the new rows legibly as is. Two
things you might expect and did not get, both on purpose: no admin email on an
auto-disable (D18 calls it a fast-follow; not built, not logged as a new item
because D18 already names it), and no per-user history view in the User
Directory (it has none today, and the task says not to build one).
### T13.6 — D18: verification
- **Items:** `D18`
- **Depends on:** T13.4, T13.5
- **Blocks:** nothing
- **Surface:** `server/`
- **Files:** as touched above
**Do:** Full verification per `CLAUDE.md`. Beyond the usual run: specifically
re-run T13.3's fail-closed test in isolation and confirm it still passes after
T13.4's scheduling wrapper is in place — the scheduling layer must not
introduce a path that swallows the client's error and proceeds anyway.
**Done when:**
- [ ] all `D18` acceptance criteria in `decisions-2026-09-17.md` are met or a
failure is stated with a reason
- [ ] the fail-closed behavior is verified end to end through the scheduled
wrapper, not just the bare sync function
- [ ] smoke test and `seed_demo.py` both still pass
- [ ] full test suite passes
---
## Wave 13 exit criteria
- [ ] `AUTH_SESSION_HOURS` defaults to 2, documented
- [ ] the sync job runs on a schedule and correctly disables accounts Okta no
longer shows as active
- [ ] every auto-disable is individually audited with a clearly non-human actor
- [ ] an Okta API failure of any kind changes zero accounts — verified, not
assumed
- [ ] the sync never re-enables an account
- [ ] `D18` fully accounted for, no open acceptance criteria
## Release note (2026-10-16)
T13.1 (commit `6cde6e3`) is held back from `main` until T14.1 lands, per the merge
rule in `wave-12.md`. The release branch contains wave 11 and T13.2 to T13.5.
`tests/session_idle_check.py` skips (exit 0) when `auth.maybe_refresh_token` is absent.
Sessions keep the flat `AUTH_SESSION_HOURS` lifetime (default 12) until T13.1 lands.

View File

@@ -34,12 +34,12 @@
/* Every container admin.js paints a table into is a scrollport of its own, so a
sticky header always has something to stick to rather than sliding up behind
the app bar. Same rule as console.css's .tscroll. */
#comments-admin, #audit-admin, #notif-box, #usage-admin, #projects-table, #defmem-table{
#comments-admin, #audit-admin, #notif-box, #activity-admin, #projects-table, #defmem-table{
overflow:auto; max-height:min(70vh,640px); overscroll-behavior:contain; }
/* If admin.js wraps its table in its own .tscroll, the outer box steps aside so
one table never ends up with two scrollbars. */
#comments-admin:has(.tscroll), #audit-admin:has(.tscroll), #notif-box:has(.tscroll),
#usage-admin:has(.tscroll), #projects-table:has(.tscroll), #defmem-table:has(.tscroll){
#activity-admin:has(.tscroll), #projects-table:has(.tscroll), #defmem-table:has(.tscroll){
overflow:visible; max-height:none; }
/* Comment text and audit detail are the two columns you are actually here to
@@ -53,7 +53,7 @@
border:1px solid var(--border-strong); border-radius:0; margin-bottom:var(--s3); }
@media (max-width:900px){
#comments-admin, #audit-admin, #notif-box, #usage-admin, #projects-table, #defmem-table{
#comments-admin, #audit-admin, #notif-box, #activity-admin, #projects-table, #defmem-table{
max-height:none; }
}
</style>
@@ -175,16 +175,38 @@
<div id="audit-admin" class="note">Click refresh to load.</div>
</div>
<!-- USAGE LOGS -->
<!-- ACTIVITY &amp; USAGE (CR-019) -->
<div class="card">
<h2>Usage logs</h2>
<div class="sub">Engagement recorded by both tools — the work package creator and the SOP wizard —
sessions, actions and counts, with a download per tool (D5). Note: stored locally per browser,
so this reflects activity on <strong>this</strong> machine.</div>
<h2>Activity &amp; usage</h2>
<div class="sub">Who is using the suite, which tools, and how often — recorded server-side on every
sign-in and page open, kept indefinitely. Filter below, or export a CSV: raw (real usernames) for
internal use, or sanitized (each user replaced with a stable, non-reversible id) for feeding into
Power BI or another external reporting tool without carrying real identities.</div>
<div class="toolbar">
<button onclick="loadUsage()">Refresh</button>
<label for="act-from">From</label>
<input type="date" id="act-from" onchange="loadActivity()">
<label for="act-to">To</label>
<input type="date" id="act-to" onchange="loadActivity()">
<select id="act-project" onchange="loadActivity()"><option value="">All projects</option></select>
<input id="act-user" placeholder="Username…" oninput="loadActivity()">
<select id="act-tool" onchange="loadActivity()">
<option value="">All tools</option>
<option value="launcher">Launcher</option>
<option value="wizard">SOP wizard</option>
<option value="creator">Work package creator</option>
<option value="field_view">Field view</option>
<option value="admin">Admin console</option>
<option value="directory">User directory</option>
</select>
<button onclick="loadActivity()">Refresh</button>
</div>
<div id="usage-admin" class="note">Click refresh to load.</div>
<div id="activity-banner" class="banner" style="display:none"></div>
<div id="activity-admin" class="note">Loading…</div>
<div class="toolbar" style="margin-top:var(--s3)">
<button onclick="exportActivity(false)">Download CSV (raw)</button>
<button onclick="exportActivity(true)">Download CSV (sanitized)</button>
</div>
<div id="activity-export-banner" class="banner" style="display:none"></div>
</div>
<!-- DB SNAPSHOT -->
@@ -215,7 +237,6 @@
</div>
</div>
<script src="wp-usage.js"></script>
<script src="console-util.js"></script>
<script src="wp-dialog.js"></script>
<script src="admin.js"></script>

View File

@@ -24,7 +24,7 @@ function reveal(){
loadNotifications();
loadComments();
loadAudit();
loadUsage();
loadActivity();
}
function showDenied(){
document.getElementById('admin-denied').style.display='';
@@ -189,6 +189,19 @@ async function loadProjects(){
banner.style.display='none';
_adminProjects = json;
renderProjects();
populateActivityProjectFilter();
}
// The activity project filter reuses the same project list the Projects card
// already fetched — no second /api/projects call just to fill a <select>.
function populateActivityProjectFilter(){
const sel = document.getElementById('act-project');
if(!sel) return;
const cur = sel.value;
sel.innerHTML = '<option value="">All projects</option>' +
_adminProjects.slice().sort((a,b)=>String(a.name||'').localeCompare(String(b.name||'')))
.map(p => '<option value="'+uesc(p.id)+'">'+uesc(p.name||p.number||p.id)+'</option>').join('');
sel.value = cur;
}
function renderProjects(){
@@ -465,6 +478,90 @@ function renderAudit(){
'</tr>').join('')+'</tbody></table>';
}
// ── activity & usage (CR-019) ────────────────────────────────────────────────────
// Server-side, per-user activity — distinct from the "Activity log" card above
// (that's AuditLog: business mutations) and from the "Usage logs" card below
// (that's per-browser localStorage, D5/T7.10, on the way out per T11.6). This is
// /api/usage/summary and /api/usage/export: real rows, aggregated server-side,
// filterable by date/project/user/tool, and exportable raw or sanitized.
function _activityFilters(){
const q = new URLSearchParams();
const from = document.getElementById('act-from').value; if(from) q.set('from', from);
const to = document.getElementById('act-to').value; if(to) q.set('to', to);
const proj = document.getElementById('act-project').value; if(proj) q.set('project_id', proj);
const user = (document.getElementById('act-user').value||'').trim(); if(user) q.set('username', user);
const tool = document.getElementById('act-tool').value; if(tool) q.set('tool', tool);
return q;
}
async function loadActivity(){
const banner = document.getElementById('activity-banner');
const box = document.getElementById('activity-admin');
if(!box) return;
banner.style.display='none';
box.textContent = 'Loading…';
const { status, json } = await api('GET', '/api/usage/summary?'+_activityFilters().toString());
if(status===403){
banner.className='banner bad'; banner.style.display='';
banner.textContent = '✕ Your account can’t see suite-wide activity here — this needs admin, or Project Super User on at least one project.';
box.innerHTML=''; return;
}
if(status!==200 || !json){
banner.className='banner bad'; banner.style.display='';
banner.textContent = '✕ Could not load activity ('+apiError(status, json, 'load failed')+').';
box.innerHTML=''; return;
}
renderActivity(json);
}
function renderActivity(sum){
const box = document.getElementById('activity-admin');
if(!sum.event_count){ box.innerHTML = '<div class="note">No activity matches this filter.</div>'; return; }
const fmt = s => s ? wpFormatDateTime(s) : '—';
const bucket = (label, obj, take) => {
const entries = Object.entries(obj).sort((a,b)=> b[0].localeCompare(a[0])).slice(0, take);
if(!entries.length) return '';
return '<table class="kv" style="margin-top:8px"><caption style="text-align:left;font-weight:600;margin-bottom:4px">'+
uesc(label)+'</caption>'+entries.map(([k,v]) => '<tr><th>'+uesc(k)+'</th><td>'+v+' active user'+(v===1?'':'s')+'</td></tr>').join('')+
'</table>';
};
const perTool = Object.entries(sum.by_tool||{});
const toolTable = perTool.length ? '<table class="users"><thead><tr><th>Tool</th><th>Events</th></tr></thead><tbody>'+
perTool.map(([t,n]) => '<tr><td>'+uesc(t||'(login)')+'</td><td>'+n+'</td></tr>').join('')+'</tbody></table>' :
'<div class="note">No per-tool data.</div>';
const perUser = Object.entries(sum.per_user_last_active||{}).sort((a,b)=> String(b[1]).localeCompare(String(a[1])));
const userTable = perUser.length ? '<table class="users"><thead><tr><th>User</th><th>Last active</th></tr></thead><tbody>'+
perUser.map(([u,t]) => '<tr><td><strong>'+uesc(u)+'</strong></td><td style="color:var(--muted)">'+fmt(t)+'</td></tr>').join('')+
'</tbody></table>' : '<div class="note">No per-user data.</div>';
box.innerHTML =
'<div class="note">'+sum.event_count+' event'+(sum.event_count===1?'':'s')+' matched.</div>'+
'<div class="row" style="gap:var(--s4);flex-wrap:wrap;align-items:flex-start">'+
'<div>'+bucket('Active users by day', sum.active_users.by_day, 30)+'</div>'+
'<div>'+bucket('Active users by week', sum.active_users.by_week, 12)+'</div>'+
'<div>'+bucket('Active users by month', sum.active_users.by_month, 12)+'</div>'+
'</div>'+
'<h2 style="margin-top:16px">By tool</h2>'+toolTable+
'<h2 style="margin-top:16px">Per-user last active</h2>'+userTable;
}
async function exportActivity(sanitize){
const banner = document.getElementById('activity-export-banner');
banner.className='banner'; banner.style.display=''; banner.textContent='Preparing export…';
const q = _activityFilters();
if(sanitize) q.set('sanitize', 'true');
const { status, json } = await api('GET', '/api/usage/export?'+q.toString());
if(status!==200 || typeof json !== 'string'){
banner.className='banner bad';
banner.textContent = '✕ Export failed ('+apiError(status, json, 'export failed')+').';
return;
}
const blob = new Blob([json], { type:'text/csv' });
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = 'usage_export_'+(sanitize?'sanitized':'raw')+'_'+new Date().toISOString().slice(0,10)+'.csv';
document.body.appendChild(a); a.click(); a.remove();
setTimeout(() => URL.revokeObjectURL(a.href), 1000);
banner.className='banner ok';
banner.textContent = '✓ Downloaded the '+(sanitize?'sanitized':'raw')+' export.';
}
// ── notifications / email settings ──────────────────────────────────────────────
let _settings = {};
async function loadSettings(){
@@ -656,48 +753,6 @@ async function loadNotifications(){
'</tr>').join('')+'</tbody></table>';
}
// ── usage logs (read from this browser's localStorage) ──────────────────────────
// D5 / T7.10: the report for BOTH tools' recorded usage, in the one place an
// operator-facing readout belongs - behind the same admin gate as this whole
// page (gateByRole() below shows nothing else either). Data comes from
// wp-usage.js, the single implementation; the keys predate the move, so
// everything recorded before it is still here.
function loadUsage(){
const box = document.getElementById('usage-admin');
if(!box) return;
const tools = [
['Work package creator', WPUsage.KEYS.creator, 'wp-iwp-usage'],
['SOP wizard', WPUsage.KEYS.wizard, 'wp-suite-usage'],
];
let html = '';
tools.forEach(([label, key, prefix]) => {
const evs = (WPUsage.load(key).events) || [];
html += '<h2 style="margin-top:16px">' + uesc(label) + '</h2>';
if(!evs.length){
html += '<div class="note">No usage recorded in this browser yet.</div>';
return;
}
const byEvent = {}, sessions = new Set();
let first = evs[0].ts, last = evs[0].ts;
evs.forEach(e => {
byEvent[e.event] = (byEvent[e.event]||0)+1;
if(e.session) sessions.add(e.session);
if(e.ts < first) first = e.ts; if(e.ts > last) last = e.ts;
});
const fmt = v => v ? wpFormatDateTime(v) : '—';
html += '<table class="kv">'+
'<tr><th>Sessions</th><td>'+sessions.size+'</td></tr>'+
'<tr><th>Events</th><td>'+evs.length+'</td></tr>'+
'<tr><th>Range</th><td style="font-weight:600">'+fmt(first)+' → '+fmt(last)+'</td></tr></table>';
html += '<table class="users"><thead><tr><th>Event</th><th>Count</th></tr></thead><tbody>';
Object.keys(byEvent).sort().forEach(k => html += '<tr><td>'+uesc(k)+'</td><td>'+byEvent[k]+'</td></tr>');
html += '</tbody></table>';
html += '<div class="toolbar" style="margin-top:8px"><button onclick="WPUsage.download(WPUsage.KEYS.'+
(key === WPUsage.KEYS.creator ? 'creator' : 'wizard')+', ' + jsq(prefix) + ')">Download the full event log</button></div>';
});
box.innerHTML = html;
}
// ── access control: admins only ─────────────────────────────────────────────────
// auth-guard.js requires a login and sets window.WP_USER (firing 'wp-auth-ready').
// Show the console for admins; otherwise show the "Admins only" notice.

View File

@@ -125,12 +125,40 @@
// Nothing replaces it. Every signed-in page mounts the drawer, so there is no page
// left that would need a floating fallback pill.
// ── CR-019: usage ping ───────────────────────────────────────────────────
// One page_open event per authenticated load, sent from exactly ONE place
// (here) rather than from each page's own script - the shared-chrome lesson
// S4 and the token-drift lesson S5 both taught this codebase the hard way.
// Fire-and-forget: never blocks reveal(), never retries, never surfaces an
// error to the person using the app - a missed usage ping is not something
// anyone here should notice happening.
var TOOL_BY_PAGE = {
'index.html': 'launcher',
'work-package-suite.html': 'wizard',
'wp-creation-index.html': 'creator',
'field.html': 'field_view',
'admin.html': 'admin',
'users.html': 'directory'
};
function pingUsage() {
var page = (location.pathname.split('/').pop() || 'index.html');
var tool = TOOL_BY_PAGE[page] || page.replace(/\.html$/, '');
try {
fetch('/api/usage/ping', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ tool: tool })
}).catch(function () {});
} catch (e) {}
}
function proceed(user) {
clearTimeout(safety);
window.WP_USER = user;
reveal();
if (window.WP_USER) {
window.wpFlags(); // start the feature-flag fetch; pages await it as needed
pingUsage();
try { document.dispatchEvent(new CustomEvent('wp-auth-ready', { detail: window.WP_USER })); } catch (e) {}
}
}

View File

@@ -2207,20 +2207,18 @@ function loadStepComments(){
}
}
// ── USAGE ANALYTICS ─────────────────────────────────────────────────────────
// Lightweight usage analytics stored in localStorage so the tool owner can review
// engagement over time. No field VALUES are stored (field-edit events record only
// the field id), keeping captured data non-sensitive.
// D5 / T7.10: the analytics implementation lives in wp-usage.js and the report
// on the admin console. The wizard's own copy of showAnalytics() never had a
// caller here - the button lived on the creator - and once B7 dissolved the
// frame the duplicate sat in the same document as five colliding globals.
// This page only records; dwell tracking keeps its page-local state below.
// ── USAGE ANALYTICS (retired, T11.6) ────────────────────────────────────────
// This used to write to wp-usage.js's per-browser localStorage log (D5/T7.10),
// read back by the admin console's old "Usage logs" panel. CR-019 replaced
// both with real, server-side, per-user activity (UsageEvent + the Activity &
// usage card) - see decisions-2026-09-17.md for why this per-browser data was
// never a source the new report could adopt. track() is now a no-op; kept
// (rather than deleting its handful of call sites, including the dwell-timer
// plumbing below) so this stays a one-line change instead of touching every
// caller for the same outcome.
let _stepEnter = Date.now();
function track(event, detail){
WPUsage.track(WPUsage.KEYS.wizard, event, detail);
}
function track(event, detail){ /* retired, T11.6 — see comment above */ }
function trackStepDwell(){
const ms = Date.now() - _stepEnter;
if(ms > 400 && ms < 1000*60*60) track('step_dwell', {step: currentStep, ms});

View File

@@ -11,7 +11,6 @@
<!-- Addressable state (S3). Parses before the app scripts, which read the URL
during their own boot. -->
<script src="wp-url.js"></script>
<script src="wp-usage.js"></script>
<script src="wp-list-import.js"></script>
<!-- Autosave, unsaved-work guard, draft recovery (S2). -->
<script src="wp-autosave.js"></script>

View File

@@ -3993,11 +3993,17 @@ function openSopModal(){
document.getElementById('sop-modal').classList.add('open'); track('view_sop');
}
function closeSopModal(){ document.getElementById('sop-modal').classList.remove('open'); }
// D5 / T7.10: the analytics implementation lives in wp-usage.js - ONE copy for
// the whole suite - and its report lives on the admin console, where an
// operator-facing readout belongs. This page only records. Same key, same
// event shape: everything recorded before the move is still readable after it.
function track(event,detail){ if(devMode) return; WPUsage.track(WPUsage.KEYS.creator, event, detail); }
// CR-019 / T11.6 (2026-09-23): this used to write to wp-usage.js's per-browser
// localStorage log (D5/T7.10). Retired along with the admin console's old
// "Usage logs" panel, its only reader - real, server-side, per-user activity
// now exists (UsageEvent, the Activity & usage card, T11.1-T11.5) and this
// data was never reliably tied to a real identity anyway, so it was not a
// source the new report could adopt (decisions-2026-09-17.md). track() stays
// as a no-op rather than deleting its ~35 call sites throughout this file:
// removing every call individually is a much larger, riskier diff for the
// same outcome, and a call site here still documents the moment worth
// recording if usage analytics are ever rebuilt server-side.
function track(event,detail){ /* retired, T11.6 — see comment above */ }
// ── COMMENTS ─────────────────────────────────────────────────────────────────
const COMMENTS_KEY='wp_iwp_comments_v1';

View File

@@ -11,7 +11,6 @@
<!-- Addressable state (S3). Parses before the app scripts, which read the URL
during their own boot. -->
<script src="wp-url.js"></script>
<script src="wp-usage.js"></script>
<!-- Autosave, unsaved-work guard, draft recovery (S2). -->
<script src="wp-autosave.js"></script>
<!-- Which sections this project uses (CR-006). The same file the SOP wizard

View File

@@ -1,58 +0,0 @@
/* Usage analytics core — the ONE implementation (D5 / T7.10).
This existed three times: the creator's copy, the wizard's copy (which had no
caller — the button lived on the creator), and the admin console's own reader.
Once the creator stopped being an iframe (B7/T7.1) the first two sat in one
document as five colliding globals; an unreferenced duplicate is exactly what
produced D5. One core now; the pages keep only a thin track() wrapper because
page state (the creator's dev-mode pause) belongs to the page.
The storage KEYS are unchanged on purpose: everything recorded before this
file existed is still readable through it. No field VALUES are ever stored —
a field-edit event records the field id, nothing else.
Classic script, no modules: exposes window.WPUsage. */
'use strict';
(function () {
var SESSION = 's_' + Date.now().toString(36) + Math.random().toString(36).slice(2, 6);
function load(key) {
try { return JSON.parse(localStorage.getItem(key)) || { events: [] }; }
catch (e) { return { events: [] }; }
}
function save(key, data) {
try { localStorage.setItem(key, JSON.stringify(data)); }
catch (e) { /* storage unavailable — degrade silently */ }
}
function track(key, event, detail) {
try {
var d = load(key);
d.events.push({ ts: new Date().toISOString(), session: SESSION, event: event, detail: detail || null });
if (d.events.length > 5000) d.events = d.events.slice(-5000);
save(key, d);
} catch (e) { /* never let telemetry break the tool it watches */ }
}
function download(key, prefix) {
var blob = new Blob([JSON.stringify(load(key), null, 2)], { type: 'application/json' });
var a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = (prefix || 'wp-usage') + '-' + new Date().toISOString().slice(0, 10) + '.json';
document.body.appendChild(a);
a.click();
a.remove();
setTimeout(function () { URL.revokeObjectURL(a.href); }, 1000);
}
window.WPUsage = {
load: load,
save: save,
track: track,
download: download,
// The pre-D5 keys, verbatim — continuity of the recorded data is a done-when.
KEYS: { creator: 'wp_iwp_analytics_v1', wizard: 'wp_suite_analytics_v1' },
};
})();

View File

@@ -55,6 +55,36 @@ OKTA_REDIRECT_URI=CHANGE_ME
# a custom claim, …) — no code change needed, just this value.
# OKTA_IDENTITY_CLAIM=preferred_username
# ── Okta deprovisioning sync (D18, wave 13) ────────────────────────────────────
# A scheduled job asks Okta who is still active and disables local accounts
# Okta no longer shows as active. It only ever DISABLES. It never re-enables.
# If Okta cannot be reached or answers with anything unexpected, the job changes
# NOTHING that cycle and logs the failure.
#
# A read-only Okta API token, created by the person who administers the Okta
# tenant (Security > API > Tokens). This is NOT the OIDC client secret above.
# It is a different credential with a different scope. Treat it like
# AUTH_SECRET_KEY: never commit it, never log it. Leave it unset and the sync
# stays off (the startup log says so).
# OKTA_API_TOKEN=CHANGE_ME
#
# The Okta org URL that serves the Management API, e.g. https://yourorg.okta.com
# Optional. If unset, it is the scheme and host of OKTA_ISSUER. Must be https.
# OKTA_ORG_URL=https://your-org.okta.com
#
# Which Okta PROFILE attribute holds the value that matches users.username.
# Optional. By default it follows OKTA_IDENTITY_CLAIM: preferred_username reads
# profile.login, email reads profile.email, any other claim name reads the
# profile attribute of the same name. Set this only if your tenant builds the
# claim from a different attribute.
# OKTA_SYNC_PROFILE_ATTRIBUTE=login
#
# How often the sync runs, in seconds. Default 900 (15 minutes), proposed and not
# confirmed with IT. A value under 300 is raised to 300. Every api worker checks
# once a minute and one of them wins each cycle, so raising the worker count does
# not raise the call rate to Okta.
# OKTA_SYNC_INTERVAL_SECONDS=900
# ── Email notifications (optional) ─────────────────────────────────────────────
# WP-assignment emails are OFF by default and are turned on from the Admin
# console (Notifications & email card), where the SMTP host/port/from-address

View File

@@ -0,0 +1,46 @@
"""usage events (CR-019, wave 11)
Append-only navigation/session activity, separate from audit_log on purpose —
see the UsageEvent docstring in server/models.py. Retention is indefinite by
decision (docs/waves/decisions-2026-09-17.md); nothing here schedules a purge.
Revision ID: 8e2cb3003f8a
Revises: 1d60a608bb51
Create Date: 2026-09-23 00:00:00.000000
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = '8e2cb3003f8a'
down_revision = '1d60a608bb51'
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table('usage_events',
sa.Column('id', sa.String(length=40), nullable=False),
sa.Column('at', sa.DateTime(timezone=True), nullable=False),
sa.Column('username', sa.String(length=200), nullable=False),
sa.Column('project_id', sa.String(length=40), nullable=True),
sa.Column('tool', sa.String(length=40), nullable=False),
sa.Column('event', sa.String(length=40), nullable=False),
sa.Column('detail', sa.JSON(), nullable=False),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_usage_events_at'), 'usage_events', ['at'], unique=False)
op.create_index(op.f('ix_usage_events_username'), 'usage_events', ['username'], unique=False)
op.create_index(op.f('ix_usage_events_project_id'), 'usage_events', ['project_id'], unique=False)
op.create_index(op.f('ix_usage_events_tool'), 'usage_events', ['tool'], unique=False)
op.create_index(op.f('ix_usage_events_event'), 'usage_events', ['event'], unique=False)
def downgrade() -> None:
op.drop_index(op.f('ix_usage_events_event'), table_name='usage_events')
op.drop_index(op.f('ix_usage_events_tool'), table_name='usage_events')
op.drop_index(op.f('ix_usage_events_project_id'), table_name='usage_events')
op.drop_index(op.f('ix_usage_events_username'), table_name='usage_events')
op.drop_index(op.f('ix_usage_events_at'), table_name='usage_events')
op.drop_table('usage_events')

View File

@@ -9,10 +9,16 @@ Run (prod): gunicorn -k uvicorn.workers.UvicornWorker -b 127.0.0.1:8000 serve
Interactive docs: http://<host>/api/docs
"""
import base64
import csv
import contextlib
import hashlib
import hmac
import io
import logging
import os
import re
import uuid
from datetime import datetime, timedelta, timezone
from typing import Any, Optional
from urllib.parse import urlparse
@@ -22,12 +28,12 @@ from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse, RedirectResponse
from fastapi.staticfiles import StaticFiles
from pydantic import BaseModel, ConfigDict, Field
from sqlalchemy import select, delete, func
from sqlalchemy import select, delete, func, or_
from sqlalchemy.orm import Session
from starlette.middleware.sessions import SessionMiddleware
from .db import Base, engine, get_db
from . import models, auth, notify, assets_db, okta_auth, okta_fake, files_http
from .db import Base, SessionLocal, engine, get_db
from . import models, auth, notify, assets_db, okta_auth, okta_fake, files_http, okta_sync
log = logging.getLogger("wpsuite.app")
@@ -42,7 +48,21 @@ if engine.dialect.name == "sqlite":
# Interactive docs are handy in dev but hand an attacker the full API map in prod,
# so enable them only on the SQLite dev fallback (production runs on Postgres).
_docs_enabled = engine.dialect.name == "sqlite"
@contextlib.asynccontextmanager
async def _lifespan(_app):
"""Start the Okta deprovisioning sync (D18 / T13.4) when the process starts and
stop it when the process ends. With no OKTA_API_TOKEN it starts nothing and
says so in the startup log."""
stop = okta_sync.start_scheduler(SessionLocal)
try:
yield
finally:
if stop is not None:
stop.set()
app = FastAPI(
lifespan=_lifespan,
title="Work Package Suite API",
docs_url="/api/docs" if _docs_enabled else None,
redoc_url=None,
@@ -756,6 +776,11 @@ async def okta_callback(request: Request, db: Session = Depends(get_db)):
user.failed_attempts = 0
user.locked_until = None
user.last_login_at = models.utcnow()
# CR-019: one login event per Okta sign-in, written here rather than
# inferred from session creation elsewhere, so there is exactly one
# source of truth for "did this person sign in" - not one per page load
# afterward (T11.2 covers that separately, as page_open events).
db.add(models.UsageEvent(id=gen_id("uev"), username=user.username, tool="", event="login"))
db.commit()
db.refresh(user)
@@ -3675,6 +3700,177 @@ def create_feedback(body: CommentIn, user: models.User = Depends(auth.get_curren
return _save_comment(body, db, user)
# ── CR-019: usage/activity metrics ──────────────────────────────────────────
class UsageIn(BaseModel):
tool: str = ""
project_id: Optional[str] = None
@app.post("/api/usage/ping")
def usage_ping(body: UsageIn, user: models.User = Depends(auth.get_current_user), db: Session = Depends(get_db)):
"""One usage_events row per authenticated page load. Called exactly once,
from auth-guard.js after wp-auth-ready fires (T11.2) - never duplicated
per page's own script, the same lesson S4's per-page nav already taught
this codebase. `user` comes from the session via get_current_user, never
from anything the client claims - identity here is a server-enforced
fact, matching every other write in this file, not a client-reported one."""
tool = (body.tool or "").strip()[:40]
db.add(models.UsageEvent(
id=gen_id("uev"), username=user.username, project_id=body.project_id,
tool=tool, event="page_open",
))
db.commit()
return {"ok": True}
def _parse_date_q(v: str, end: bool = False) -> Optional[datetime]:
"""Accepts a plain YYYY-MM-DD (what a <input type=date> sends) or a full
ISO datetime. A date-only `to` means "through the end of that day", not
midnight at its start - otherwise a range of "today" would match nothing
from today at all."""
if not v:
return None
try:
d = datetime.fromisoformat(v)
except ValueError:
return None
if d.tzinfo is None:
d = d.replace(tzinfo=timezone.utc)
if end and len(v) <= 10: # date-only
d = d + timedelta(days=1) - timedelta(microseconds=1)
return d
def _usage_query(db: Session, caller: "models.User", date_from, date_to, project_id, username, tool):
"""Shared by the summary and export endpoints so the two can never
disagree about which rows a filter set matches - the export is a raw
dump of exactly what the summary counted, not a separately-derived view."""
stmt = select(models.UsageEvent)
managed = managed_project_ids(db, caller)
if managed is not None:
# A project_super_user (never an app admin, who gets managed=None) is
# scoped to events tied to a project they administer, plus their OWN
# suite-wide activity (admin console opens etc. carry no project_id) -
# never another user's activity outside what they manage.
if managed:
stmt = stmt.where(or_(
models.UsageEvent.project_id.in_(managed),
models.UsageEvent.username == caller.username,
))
else:
stmt = stmt.where(models.UsageEvent.username == caller.username)
df = _parse_date_q(date_from)
dt = _parse_date_q(date_to, end=True)
if df:
stmt = stmt.where(models.UsageEvent.at >= df)
if dt:
stmt = stmt.where(models.UsageEvent.at <= dt)
if project_id:
stmt = stmt.where(models.UsageEvent.project_id == project_id)
if username:
stmt = stmt.where(models.UsageEvent.username == username)
if tool:
stmt = stmt.where(models.UsageEvent.tool == tool)
return stmt.order_by(models.UsageEvent.at)
@app.get("/api/usage/summary")
def usage_summary(
date_from: Optional[str] = Query(None, alias="from"),
date_to: Optional[str] = Query(None, alias="to"),
project_id: Optional[str] = Query(None),
username: Optional[str] = Query(None),
tool: Optional[str] = Query(None),
caller: models.User = Depends(require_user_manager),
db: Session = Depends(get_db),
):
"""CR-019. Same gate as the User Directory (require_user_manager): an app
admin or a project_super_user with at least one managed project. Filters
combine. Aggregated in Python over the filtered row set rather than a SQL
GROUP BY - correct and simple at today's scale; if usage_events grows
into the millions (plausible, given retention is indefinite by decision),
the day/week/month bucketing here is the first thing to move server-side
into SQL. Not done now because nothing currently requires it."""
rows = db.scalars(_usage_query(db, caller, date_from, date_to, project_id, username, tool)).all()
by_day: dict[str, set] = {}
by_week: dict[str, set] = {}
by_month: dict[str, set] = {}
per_user_last: dict[str, datetime] = {}
per_tool: dict[str, int] = {}
for e in rows:
by_day.setdefault(e.at.date().isoformat(), set()).add(e.username)
by_week.setdefault(e.at.strftime("%G-W%V"), set()).add(e.username)
by_month.setdefault(e.at.strftime("%Y-%m"), set()).add(e.username)
if e.username not in per_user_last or e.at > per_user_last[e.username]:
per_user_last[e.username] = e.at
per_tool[e.tool] = per_tool.get(e.tool, 0) + 1
return {
"active_users": {
"by_day": {k: len(v) for k, v in sorted(by_day.items())},
"by_week": {k: len(v) for k, v in sorted(by_week.items())},
"by_month": {k: len(v) for k, v in sorted(by_month.items())},
},
"per_user_last_active": {u: models._iso(t) for u, t in sorted(per_user_last.items())},
"by_tool": dict(sorted(per_tool.items(), key=lambda kv: -kv[1])),
"event_count": len(rows),
}
def _pseudonym(username: str) -> str:
"""A stable per-user id for the sanitized export — the SAME input always
produces the SAME output, within one export and across separate export
runs, so an external system (Power BI or similar) can still group and
trend "by user" without ever receiving a real name. HMAC rather than a
plain hash: a plain sha256(username) is trivially reversed against a
wordlist of the handful of usernames this app actually has; keying it
with AUTH_SECRET_KEY (already a real secret, already required in
production — see auth.py) means recovering a username from its
pseudonym requires the signing key, not just guessing."""
digest = hmac.new(auth.SECRET_KEY.encode(), username.encode(), hashlib.sha256).hexdigest()
return "u_" + digest[:16]
@app.get("/api/usage/export")
def usage_export(
date_from: Optional[str] = Query(None, alias="from"),
date_to: Optional[str] = Query(None, alias="to"),
project_id: Optional[str] = Query(None),
username: Optional[str] = Query(None),
tool: Optional[str] = Query(None),
sanitize: bool = Query(False),
caller: models.User = Depends(require_user_manager),
db: Session = Depends(get_db),
):
"""CR-019. Same gate, same filters, same underlying row set as
usage_summary() (_usage_query) — the export can never show a different
slice of data than what the console counted for the same filters.
sanitize=true replaces `username` with a stable pseudonym (_pseudonym)
and — deliberately — the `detail` column is not exported in EITHER mode.
Every event this app writes today (login, page_open) leaves `detail`
empty, so this costs nothing now, but it also means a future event type
that DOES populate `detail` can't accidentally leak a real name into a
sanitized file through a column nobody thought to scrub. If `detail`
is ever needed in the export, it has to be sanitized explicitly, not
assumed safe because the rest of the row was."""
rows = db.scalars(_usage_query(db, caller, date_from, date_to, project_id, username, tool)).all()
buf = io.StringIO()
w = csv.writer(buf)
w.writerow(["at", "username", "project_id", "tool", "event"])
for e in rows:
who = _pseudonym(e.username) if sanitize else e.username
w.writerow([models._iso(e.at), who, e.project_id or "", e.tool, e.event])
filename = "usage_export_%s_%s.csv" % (
"sanitized" if sanitize else "raw", datetime.now(timezone.utc).strftime("%Y%m%d"),
)
return Response(
content=buf.getvalue(), media_type="text/csv",
headers={"Content-Disposition": f'attachment; filename="{filename}"'},
)
@app.get("/api/comments")
def list_comments(
source: Optional[str] = Query(None),

View File

@@ -191,8 +191,7 @@ def decode_token(token: str) -> Optional[dict]:
if not token or len(token) > _MAX_TOKEN_CHARS:
return None
try:
# Every token this app issues carries exp and sub (create_token,
# maybe_refresh_token); one without them is not ours.
# Every token this app issues carries exp and sub (create_token); one without them is not ours.
claims = jwt.decode(token, SECRET_KEY, algorithms=[JWT_ALG],
options={"require": ["exp", "sub"]})
except Exception: # noqa: BLE001 — see docstring: every failure is "not signed in"

View File

@@ -337,6 +337,45 @@ class AuditLog(Base):
}
class UsageEvent(Base):
"""CR-019: append-only record of who used the suite, when, and which tool —
navigation/session activity, not business mutations. Deliberately a SEPARATE
table from AuditLog rather than a new `action` value there: AuditLog answers
"who changed what" and is read by people auditing a specific record's
history; mixing in a `page_open` row for every authenticated page load
would make that trail noisy for its existing purpose. This table answers a
different question — "who is active, and on what" — and CR-019's admin
console reads from here, not from AuditLog.
Not a ForeignKey to `users`, matching AuditLog's own reasoning: a user who
is later removed should still show up in historical activity rather than
silently vanishing from it, and `D18`'s deprovisioning sync only ever sets
`is_active=False` — it never deletes a row — so this is defensive symmetry
rather than a live concern today.
Retention is indefinite (decided 2026-09-17, `decisions-2026-09-17.md`) —
nothing purges rows written here; that is a deliberate product decision,
not an oversight to fix later."""
__tablename__ = "usage_events"
id: Mapped[str] = mapped_column(String(40), primary_key=True)
at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True)
username: Mapped[str] = mapped_column(String(200), default="", index=True)
project_id: Mapped[Optional[str]] = mapped_column(String(40), nullable=True, index=True)
# creator | wizard | field_view | dashboard | admin | directory | ...
tool: Mapped[str] = mapped_column(String(40), default="", index=True)
# page_open | login
event: Mapped[str] = mapped_column(String(40), default="", index=True)
detail: Mapped[dict] = mapped_column(JSON, default=dict)
def to_dict(self) -> dict:
return {
"id": self.id, "at": _iso(self.at), "username": self.username,
"project_id": self.project_id, "tool": self.tool, "event": self.event,
"detail": self.detail or {},
}
class AppSetting(Base):
"""Admin-editable application settings (feature flags, SMTP config, …) stored
as key -> JSON value. Read/written via /api/settings (admin only). Secrets like

479
server/okta_sync.py Normal file
View File

@@ -0,0 +1,479 @@
"""Okta Management API client - T13.2, wave 13 (D18).
This module asks Okta one question: who is in the org, and what is each person's
status? It is the first half of the deprovisioning sync. It does not touch the
database and it does not disable anyone. T13.3 does that, using the list this
module returns.
Trust boundary. Sign-in uses the OIDC client (OKTA_CLIENT_ID and OKTA_CLIENT_SECRET,
see okta_auth.py). This module uses a different credential, OKTA_API_TOKEN, a
read-only Okta API token. The two never mix. A rotation or a leak of one does not
touch the other.
Failure rule (D18, decision 4). This is the most important rule in this file:
**an error is never an empty list.** If anything goes wrong, this module raises
OktaSyncError. It never returns [] to mean "the call failed". T13.3 reads an empty
list as "nobody is active", so a function that returned [] on a network error
would disable every account on an Okta outage. The cases that raise are:
- the token or the org URL is not set
- a network error or a timeout
- any HTTP status that is not 200 (401, 403 and 429 included)
- a body that is not a JSON list, or an item that is not shaped as expected
- a user with no usable identity value
- a list with no users at all (a real org is never empty; an empty list means
the token cannot see the users)
- a page limit reached (a cut-off list would make real users look missing)
- a "next page" link that points to a different host
T13.3 (the sync job, `run_sync` below) uses that list to disable local accounts.
Its rules are in its own docstring. In short: it only ever disables, it never
re-enables, it writes one audit row per change, and any failure means NO change.
The token is read from the environment each time it is used. It is never logged,
never put in an error message, and never returned to a browser. Errors carry the
exception class name and the HTTP status only. httpx exceptions can include the
request URL, so the URL is not copied into the message either.
"""
import logging
import os
import threading
import time
import uuid
from datetime import datetime, timedelta, timezone
from dataclasses import dataclass, field
from typing import Callable, Optional
from urllib.parse import urlparse
import httpx
from sqlalchemy import select, update
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from . import models, okta_auth
log = logging.getLogger("wpsuite.okta_sync")
def _make_logs_visible() -> None:
"""Make this module's INFO lines reach `docker compose logs api`.
Nothing in this app configures Python logging, and gunicorn only configures its
own loggers. So the root logger sits at WARNING with no handler, and every
INFO line from a `wpsuite.*` logger is dropped. Found on 2026-10-16: with the
exact production launch command, `grep "Okta sync"` found nothing on a healthy
start, because "configured", "scheduled" and "done" are INFO lines. Warnings
and errors (a disable, a skipped run) showed, but the lines that say the sync
is alive did not. An operator cannot tell "off" from "on and quiet".
This adds one stderr handler to THIS module's logger only, and only when the
root logger has none, so it never doubles up if logging is configured later and
it does not change what any other module prints. (BL-028 and the general
question of app-wide logging are separate items.)"""
if log.handlers or logging.getLogger().handlers:
return
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s"))
log.addHandler(handler)
log.setLevel(logging.INFO)
_make_logs_visible()
# Okta's own page maximum for this endpoint. Asking for the maximum keeps a
# 20-odd account org to a single call.
PAGE_LIMIT = 200
# A cut-off list is more dangerous than an error, so the loop is bounded and
# raises when the bound is reached. 50 pages of 200 is 10,000 people.
MAX_PAGES = 50
TIMEOUT_SECONDS = 15.0
# The whole fetch, all pages together, must finish inside this. T13.4 relies on it:
# a run is always shorter than the shortest allowed interval, so one cycle can
# never still be running when the next is due.
TOTAL_DEADLINE_SECONDS = 120.0
# Which Okta *profile attribute* matches each OIDC claim that OKTA_IDENTITY_CLAIM
# may name. The sign-in flow reads a claim from the ID token. This API returns
# the profile instead. For the default claim, preferred_username, Okta's own
# default maps it to profile.login. A claim that is not in this table is read
# from profile[<claim>] (for example a custom attribute).
# OKTA_SYNC_PROFILE_ATTRIBUTE overrides the whole table. Use it if the tenant
# builds the claim from a different attribute.
_CLAIM_TO_PROFILE = {
"preferred_username": "login",
"email": "email",
}
class OktaSyncError(Exception):
"""The Okta user list could not be fetched with confidence. T13.3 must take no
action on any account when it sees this."""
@dataclass(frozen=True)
class OktaUser:
identity: str # the value that matches users.username (lower case is NOT applied here)
status: str # Okta's status, upper case: ACTIVE, SUSPENDED, DEPROVISIONED, ...
# ── configuration ─────────────────────────────────────────────────────────────
def _token() -> str:
return os.getenv("OKTA_API_TOKEN", "").strip()
def org_url() -> str:
"""The Okta org base URL, for example https://yourorg.okta.com. The Management
API lives here, not at the authorization server issuer. OKTA_ORG_URL wins if
set. Otherwise it is the scheme and host of OKTA_ISSUER."""
explicit = os.getenv("OKTA_ORG_URL", "").strip().rstrip("/")
if explicit:
return explicit
issuer = (okta_auth.ISSUER or "").strip()
parsed = urlparse(issuer)
if parsed.scheme and parsed.netloc:
return f"{parsed.scheme}://{parsed.netloc}"
return ""
def is_configured() -> bool:
return bool(_token() and org_url().startswith("https://"))
def describe() -> str:
"""One line for the startup log. Never contains the token."""
if not _token():
return "Okta sync DISABLED - OKTA_API_TOKEN is not set. Deprovisioning is not detected."
base = org_url()
if not base:
return "Okta sync DISABLED - no OKTA_ORG_URL and OKTA_ISSUER has no host. Deprovisioning is not detected."
if not base.startswith("https://"):
return f"Okta sync DISABLED - org URL {base!r} is not https. The token is never sent over http."
return f"Okta sync configured - org {base}, identity claim {okta_auth.IDENTITY_CLAIM!r}"
def _profile_attribute() -> str:
override = os.getenv("OKTA_SYNC_PROFILE_ATTRIBUTE", "").strip()
if override:
return override
claim = okta_auth.IDENTITY_CLAIM
return _CLAIM_TO_PROFILE.get(claim, claim)
# ── the client ────────────────────────────────────────────────────────────────
def _fail(reason: str) -> "OktaSyncError":
# One place to build the error, so no call site can add a URL or a header by
# accident.
return OktaSyncError(f"Okta user list could not be fetched: {reason}")
def _same_origin(url: str, base: str) -> bool:
a, b = urlparse(url), urlparse(base)
return (a.scheme, a.netloc) == (b.scheme, b.netloc)
def _parse_user(item, attribute: str) -> OktaUser:
if not isinstance(item, dict):
raise _fail("a list item is not an object")
status = item.get("status")
profile = item.get("profile")
if not isinstance(status, str) or not status.strip():
raise _fail("a user has no status")
if not isinstance(profile, dict):
raise _fail("a user has no profile")
value = profile.get(attribute)
if not isinstance(value, str) or not value.strip():
raise _fail(f"a user has no value for profile attribute {attribute!r}")
return OktaUser(identity=value.strip(), status=status.strip().upper())
def fetch_users(http: Optional[httpx.Client] = None) -> "list[OktaUser]":
"""Return every user Okta lists, as OktaUser values. Raises OktaSyncError on
any problem; never returns an empty list. `http` is for tests (a client built
on httpx.MockTransport). In production it is None and a short-lived client is
made here."""
token = _token()
base = org_url()
if not token:
raise _fail("OKTA_API_TOKEN is not set")
if not base.startswith("https://"):
raise _fail("the org URL is missing or is not https")
attribute = _profile_attribute()
own_client = http is None
client = http if http is not None else httpx.Client(timeout=TIMEOUT_SECONDS)
headers = {"Accept": "application/json", "Authorization": f"SSWS {token}"}
url: Optional[str] = f"{base}/api/v1/users"
params: Optional[dict] = {"limit": PAGE_LIMIT}
users: "list[OktaUser]" = []
started = time.monotonic()
try:
for _ in range(MAX_PAGES):
if not url:
break
if time.monotonic() - started > TOTAL_DEADLINE_SECONDS:
raise _fail(f"took longer than {int(TOTAL_DEADLINE_SECONDS)} seconds in total")
try:
resp = client.get(url, params=params, headers=headers,
follow_redirects=False)
except httpx.HTTPError as exc:
# Class name only. The exception text can carry the URL.
raise _fail(f"network error ({type(exc).__name__})") from None
if resp.status_code != 200:
raise _fail(f"Okta answered HTTP {resp.status_code}")
try:
body = resp.json()
except ValueError:
raise _fail("the response is not JSON") from None
if not isinstance(body, list):
raise _fail("the response is not a list of users")
users.extend(_parse_user(item, attribute) for item in body)
nxt = resp.links.get("next", {}).get("url")
if not nxt:
url = None
break
if not _same_origin(nxt, base):
# Never send the token to a host that the Link header names.
raise _fail("a next-page link points to a different host")
url, params = nxt, None # the next link already carries its query
else:
# The loop used every page and the last one still named a next page.
raise _fail(f"more than {MAX_PAGES} pages; the list may be cut off")
finally:
if own_client:
client.close()
if not users:
raise _fail("Okta returned no users, which is not credible for a real org")
log.info("Okta user list fetched: %d user(s)", len(users))
return users
# ── T13.3: the sync job ───────────────────────────────────────────────────────
# Which Okta statuses mean "this person is gone or was blocked on purpose".
# Decided by Matt, 2026-10-16: a literal "anything that is not ACTIVE" would also
# catch PASSWORD_EXPIRED, LOCKED_OUT and RECOVERY, which people leave on their
# own, and because this job never re-enables, one AD password expiry would lock a
# person out of this app until an admin stepped in. A user who is MISSING from the
# list disables too (Okta's default list leaves out DEPROVISIONED users).
DISABLING_STATUSES = frozenset({"SUSPENDED", "DEPROVISIONED"})
SYNC_ACTOR = "system:okta_sync"
SYNC_ACTION = "user_deprovisioned_by_sync"
@dataclass
class SyncResult:
ok: bool # False = the run took NO action at all
error: str = "" # why, when ok is False
okta_users: int = 0 # size of the Okta list
checked: int = 0 # active local accounts looked at
disabled: "list[str]" = field(default_factory=list) # usernames disabled this run
def _gen_id(prefix: str) -> str:
# Same shape as server/app.py gen_id. Not imported: app.py will import this
# module (T13.4), and the reverse import would be circular.
return f"{prefix}_{uuid.uuid4().hex[:12]}"
def run_sync(db: Session, fetch: Optional[Callable[[], "list[OktaUser]"]] = None) -> SyncResult:
"""Compare the Okta user list to local accounts and disable the ones Okta no
longer vouches for.
The rules, none of which may be relaxed (D18):
1. FAIL CLOSED. If fetching raises ANY exception, or the answer cannot be
trusted, this function changes nothing and writes nothing. A failure is
never read as "nobody is active". The fetch happens before the database
is read, so there is no code path from a failed fetch to a change.
2. ONLY DISABLE. It sets is_active to False. No input makes it set True.
An account that is already disabled is not touched and gets no new
audit row, so a run that finds nothing new writes nothing.
3. One AuditLog row per disabled account, actor "system:okta_sync", so it
is never confused with an admin's action. Rows and changes are saved in
one transaction. If saving fails, none of it is kept.
4. Only changes are logged. A person who is still active in Okta and still
active here produces no row.
One guard beyond the rules above, inside decision 4's "anything the sync can't
confidently parse": if there are active local accounts and NONE of them
matches any Okta user, the identity attribute is almost certainly wrong (see
OKTA_SYNC_PROFILE_ATTRIBUTE). That is a setup mistake, not an offboarding of
the whole company, so the run is refused and nothing changes.
A local account matches an Okta user by username or by email, ignoring case,
the same two fields auth.find_user() uses. If the Okta list holds the same
identity twice with different statuses, the account is disabled only when
every one of them is a disabling status.
`fetch` is for tests. In production it is None and fetch_users() is used.
"""
fetch = fetch or fetch_users
# 1. The fetch. Nothing below runs unless it returned a list.
try:
okta_users = fetch()
except Exception as exc: # an error of ANY kind, not only OktaSyncError
msg = str(exc) if isinstance(exc, OktaSyncError) else f"unexpected {type(exc).__name__}"
log.error("Okta sync SKIPPED, no account was changed: %s", msg)
return SyncResult(ok=False, error=msg)
if not okta_users:
# fetch_users() already raises on an empty list. This covers a stand-in
# fetch (and any future change) that returns one: same rule.
msg = "the fetch returned no users"
log.error("Okta sync SKIPPED, no account was changed: %s", msg)
return SyncResult(ok=False, error=msg)
try:
by_identity: "dict[str, list[str]]" = {}
for u in okta_users:
by_identity.setdefault(u.identity.strip().lower(), []).append(u.status.strip().upper())
locals_ = list(db.scalars(select(models.User).where(models.User.is_active.is_(True))))
plan: "list[tuple[models.User, str]]" = []
matched = 0
for local in locals_:
statuses: "list[str]" = []
for key in {(local.username or "").strip().lower(), (local.email or "").strip().lower()}:
if key:
statuses.extend(by_identity.get(key, []))
if statuses:
matched += 1
if all(st in DISABLING_STATUSES for st in statuses):
plan.append((local, sorted(set(statuses))[0]))
else:
plan.append((local, "MISSING"))
if locals_ and matched == 0:
msg = (f"none of {len(locals_)} active local account(s) matches any of "
f"{len(okta_users)} Okta user(s); the identity attribute is probably wrong")
log.error("Okta sync SKIPPED, no account was changed: %s", msg)
return SyncResult(ok=False, error=msg, okta_users=len(okta_users), checked=len(locals_))
planned = [(local.username, status) for local, status in plan]
checked = len(locals_)
for local, status in plan:
local.is_active = False
db.add(models.AuditLog(
id=_gen_id("ev"), actor=SYNC_ACTOR, action=SYNC_ACTION,
entity_type="user", entity_id=local.id, project_id=None,
summary=local.username[:400],
detail={"okta_status": status, "via": "okta_sync"},
))
db.commit()
except Exception as exc:
db.rollback()
log.error("Okta sync FAILED, nothing was kept: unexpected %s", type(exc).__name__)
return SyncResult(ok=False, error=f"unexpected {type(exc).__name__}")
for name, status in planned:
log.warning("Okta sync disabled %r (Okta status: %s)", name, status)
log.info("Okta sync done: %d Okta user(s), %d active local account(s) checked, %d disabled",
len(okta_users), checked, len(planned))
return SyncResult(ok=True, okta_users=len(okta_users), checked=checked,
disabled=[name for name, _ in planned])
# ── T13.4: running it on a schedule ───────────────────────────────────────────
# The api container runs gunicorn with 2 workers, and a later deploy may run more
# than one container. Each worker starts its own timer thread below. Without a
# gate, all of them would call Okta every cycle and write overlapping changes.
# The gate is one row in app_settings (key "okta_sync"). Its updated_at is "when
# the last run was claimed". A worker claims a run with ONE atomic conditional
# UPDATE: it succeeds only if the last claim is older than the interval. Exactly
# one worker wins each cycle, on SQLite and on Postgres, with no lock held.
#
# The row survives a restart, so a restart inside the interval does not start a
# second run. A slow run cannot overlap the next: a fetch is bounded to
# TOTAL_DEADLINE_SECONDS (120) and the interval can never be set below
# INTERVAL_MIN_SECONDS (300). Inside one process the timer thread runs one cycle at
# a time, so cycles cannot stack there either.
CLAIM_KEY = "okta_sync"
INTERVAL_DEFAULT_SECONDS = 900 # 15 minutes (D18, proposed, not confirmed)
INTERVAL_MIN_SECONDS = 300
TICK_SECONDS = 60 # how often each worker checks if a run is due
START_DELAY_SECONDS = 10 # let the app finish starting first
def interval_seconds() -> int:
"""OKTA_SYNC_INTERVAL_SECONDS, default 900. A value that is not a number uses
the default. A value under 300 is raised to 300 (see above for why)."""
raw = os.getenv("OKTA_SYNC_INTERVAL_SECONDS", "").strip()
if not raw:
return INTERVAL_DEFAULT_SECONDS
try:
value = int(raw)
except ValueError:
log.warning("OKTA_SYNC_INTERVAL_SECONDS=%r is not a whole number; using %d",
raw, INTERVAL_DEFAULT_SECONDS)
return INTERVAL_DEFAULT_SECONDS
if value < INTERVAL_MIN_SECONDS:
log.warning("OKTA_SYNC_INTERVAL_SECONDS=%d is under the minimum; using %d",
value, INTERVAL_MIN_SECONDS)
return INTERVAL_MIN_SECONDS
return value
def claim(db: Session, interval: int, now: Optional[datetime] = None) -> bool:
"""Try to claim this cycle. True for exactly one caller per interval."""
now = now or datetime.now(timezone.utc)
cutoff = now - timedelta(seconds=interval)
try:
row = db.get(models.AppSetting, CLAIM_KEY)
if row is None:
db.add(models.AppSetting(key=CLAIM_KEY, value={}, updated_at=now))
db.commit() # the insert is the claim
return True
except IntegrityError:
db.rollback() # another worker inserted first: it won
return False
db.rollback() # end the read; the UPDATE starts fresh
result = db.execute(
update(models.AppSetting)
.where(models.AppSetting.key == CLAIM_KEY, models.AppSetting.updated_at <= cutoff)
.values(updated_at=now)
)
db.commit()
return result.rowcount == 1
def run_if_due(session_factory: Callable[[], Session],
fetch: Optional[Callable[[], "list[OktaUser]"]] = None,
now: Optional[datetime] = None) -> Optional[SyncResult]:
"""One timer tick: claim the cycle, and if it is ours, run the sync. Returns
the SyncResult, or None when the cycle was not ours (or could not be claimed).
Nothing here can turn a failure into a change: run_sync() owns that rule, and
anything that goes wrong around it is logged and ends the tick."""
try:
with session_factory() as db:
if not claim(db, interval_seconds(), now=now):
return None
with session_factory() as db:
return run_sync(db, fetch=fetch)
except Exception as exc:
log.error("Okta sync tick failed, no account was changed: unexpected %s", type(exc).__name__)
return None
def start_scheduler(session_factory: Callable[[], Session]) -> "Optional[threading.Event]":
"""Start the timer thread. Returns a stop Event (set it to end the thread), or
None when the sync is off because OKTA_API_TOKEN or the org URL is missing.
Logs one line either way, so an operator can see it in `docker compose logs api`."""
log.info(describe())
if not is_configured():
return None
stop = threading.Event()
def loop():
if stop.wait(START_DELAY_SECONDS):
return
while not stop.is_set():
run_if_due(session_factory)
if stop.wait(TICK_SECONDS):
return
threading.Thread(target=loop, name="okta-sync", daemon=True).start()
log.info("Okta sync scheduled: every %d seconds (checked each %d s by every worker; "
"one wins each cycle)", interval_seconds(), TICK_SECONDS)
return stop

View File

@@ -254,27 +254,23 @@ def main():
if ("el.textContent = '" in ln or "el.innerHTML = '" in ln)
and "the project" not in ln and ("✓" in ln or "⚠" in ln or "↻" in ln or "✕" in ln)])
print("\n7. the analytics dwell listener still fires")
print("\n7. autosave's own registration survives the retired analytics tracker")
# This section used to also assert the per-browser usage-analytics dwell
# listener (wp-usage.js) still fired alongside autosave, to catch autosave
# accidentally replacing it. CR-019/T11.6 (2026-09-23) retired that tracker
# on purpose - track() is now a documented no-op and wp-usage.js is gone -
# so asserting it still records an event would be asserting the wrong
# thing. What's still worth checking here is autosave's own registration,
# which is what this section actually guards.
boot(page, base, tok, "/work-package-suite.html?project=projA&tab=sop")
time.sleep(0.8)
chk("the wizard still has its analytics dwell tracker",
page.eval("typeof trackStepDwell === 'function'"))
chk("...and the wizard registered autosave too",
chk("the wizard registered autosave",
page.eval("typeof WPAutosave === 'object' && typeof sopDraftId === 'function'"))
before = page.eval("""(() => {
try { return (JSON.parse(localStorage.getItem('wp_suite_analytics_v1')||'{}').events||[]).length; }
catch(e){ return -1; }
})()""")
# trackStepDwell only records a dwell longer than 400ms, so give it one.
time.sleep(0.9)
page.eval("typeof trackStepDwell==='function' && trackStepDwell()")
time.sleep(0.4)
after_n = page.eval("""(() => {
try { return (JSON.parse(localStorage.getItem('wp_suite_analytics_v1')||'{}').events||[]).length; }
catch(e){ return -1; }
})()""")
chk("calling it records an event (it was not replaced by the guard)",
after_n > before, "%s -> %s" % (before, after_n))
chk("...and the retired dwell tracker is a harmless no-op, not missing or throwing",
page.eval("""(() => {
if (typeof trackStepDwell !== 'function') return false;
try { trackStepDwell(); return true; } catch (e) { return false; }
})()"""))
finally:
page.close()
browser.close()

252
tests/okta_sync_check.py Normal file
View File

@@ -0,0 +1,252 @@
#!/usr/bin/env python3
"""Does the Okta Management API client keep its promises? - D18 / T13.2.
The one promise that matters: a failure is never an empty list. T13.3 reads an
empty list as "nobody is active", so a client that returned [] when Okta was
down would disable the whole org. Every failure below must raise OktaSyncError.
Also checked: the client uses its own credential (never the OIDC client secret),
follows pagination, never sends the token to another host, never follows a
redirect, and never lets the token reach a log line, an error message, or
describe().
Self-contained: no server, no database, no network, no browser. httpx's
MockTransport stands in for Okta.
Exit 0 all passed, 1 a failure.
"""
import logging
import os
import sys
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
import httpx # noqa: E402
from server import okta_auth, okta_sync # noqa: E402
TOKEN = "00tok-SECRET-api-token-xyz"
CLIENT_SECRET = "oidc-client-SECRET-abc"
ORG = "https://example.okta.com"
fails = 0
def chk(label, ok, extra=""):
global fails
print(("PASS " if ok else "FAIL ") + label + (f" [{extra}]" if (extra and not ok) else ""))
if not ok:
fails += 1
class Capture(logging.Handler):
def __init__(self):
super().__init__()
self.lines = []
def emit(self, record):
self.lines.append(record.getMessage())
cap = Capture()
logging.getLogger("wpsuite").addHandler(cap)
logging.getLogger("wpsuite").setLevel(logging.DEBUG)
def reset_env():
os.environ["OKTA_API_TOKEN"] = TOKEN
os.environ["OKTA_CLIENT_SECRET"] = CLIENT_SECRET
os.environ.pop("OKTA_ORG_URL", None)
os.environ.pop("OKTA_SYNC_PROFILE_ATTRIBUTE", None)
okta_auth.ISSUER = f"{ORG}/oauth2/default"
okta_auth.IDENTITY_CLAIM = "preferred_username"
def user(login, status="ACTIVE", **profile):
p = {"login": login, "email": login}
p.update(profile)
return {"id": "00u" + login, "status": status, "profile": p}
def client(handler):
return httpx.Client(transport=httpx.MockTransport(handler))
def raises(handler, label, **kw):
"""Run fetch_users against `handler`; it must raise OktaSyncError, and the
message must not hold the token."""
reset_env()
try:
out = okta_sync.fetch_users(http=client(handler))
except okta_sync.OktaSyncError as exc:
msg = str(exc)
clean = TOKEN not in msg and CLIENT_SECRET not in msg and "example.okta.com" not in msg
chk(f"{label}: raises, message has no token or host", clean, msg)
return msg
except Exception as exc: # the wrong exception type
chk(f"{label}: raises OktaSyncError", False, f"{type(exc).__name__}: {exc}")
return ""
chk(f"{label}: raises (did NOT raise, returned {out!r})", False)
return ""
def ok(body):
return lambda req: httpx.Response(200, json=body)
# ── 1. a good fetch ───────────────────────────────────────────────────────────
print("\n1. a good fetch")
reset_env()
seen = []
def good(req):
seen.append(req)
return httpx.Response(200, json=[user("alice", "ACTIVE"), user("bob", "suspended")])
users = okta_sync.fetch_users(http=client(good))
chk("returns every user", [u.identity for u in users] == ["alice", "bob"])
chk("status is upper case", [u.status for u in users] == ["ACTIVE", "SUSPENDED"])
chk("asks the users endpoint of the org", str(seen[0].url).startswith(f"{ORG}/api/v1/users"))
chk("asks for the page maximum", seen[0].url.params.get("limit") == "200")
chk("authenticates with the API token (SSWS)", seen[0].headers.get("authorization") == f"SSWS {TOKEN}")
chk("never sends the OIDC client secret", CLIENT_SECRET not in str(seen[0].headers))
# ── 2. pagination ─────────────────────────────────────────────────────────────
print("\n2. pagination")
reset_env()
calls = []
def paged(req):
calls.append(req)
if "after=p2" not in str(req.url):
return httpx.Response(200, json=[user("alice")],
headers={"Link": f'<{ORG}/api/v1/users?limit=200&after=p2>; rel="next"'})
return httpx.Response(200, json=[user("bob")])
users = okta_sync.fetch_users(http=client(paged))
chk("follows the next link and returns both pages", [u.identity for u in users] == ["alice", "bob"])
chk("two requests, no more", len(calls) == 2)
chk("page 2 also carries the token", calls[1].headers.get("authorization") == f"SSWS {TOKEN}")
# ── 3. the failure rule: an error is never an empty list ──────────────────────
print("\n3. every failure raises (never returns an empty list)")
for code in (401, 403, 404, 429, 500, 502, 503):
raises(lambda req, c=code: httpx.Response(c, json={"errorCode": "x"}), f"HTTP {code}")
raises(lambda req: httpx.Response(302, headers={"Location": "https://evil.example/"}), "a redirect (never followed)")
raises(lambda req: (_ for _ in ()).throw(httpx.ConnectError("boom to https://example.okta.com/api/v1/users")), "connection error")
raises(lambda req: (_ for _ in ()).throw(httpx.ReadTimeout("slow")), "timeout")
raises(lambda req: httpx.Response(200, text="<html>not json</html>"), "body is not JSON")
raises(ok({"errorCode": "E0000011"}), "body is an object, not a list")
raises(ok("a string"), "body is a string")
raises(ok([]), "an empty list (not credible for a real org)")
raises(ok(["alice"]), "an item that is not an object")
raises(ok([{"profile": {"login": "a"}}]), "an item with no status")
raises(ok([{"status": "ACTIVE"}]), "an item with no profile")
raises(ok([{"status": "ACTIVE", "profile": {}}]), "an item with no login")
raises(ok([{"status": "ACTIVE", "profile": {"login": " "}}]), "an item with a blank login")
raises(ok([user("alice"), {"status": "ACTIVE", "profile": {}}]), "one bad item among good ones (never skipped)")
# ── 4. misconfiguration raises ────────────────────────────────────────────────
print("\n4. misconfiguration")
reset_env()
os.environ["OKTA_API_TOKEN"] = ""
try:
okta_sync.fetch_users(http=client(good))
chk("no token raises", False)
except okta_sync.OktaSyncError:
chk("no token raises", True)
reset_env()
okta_auth.ISSUER = ""
try:
okta_sync.fetch_users(http=client(good))
chk("no org URL raises", False)
except okta_sync.OktaSyncError:
chk("no org URL raises", True)
reset_env()
os.environ["OKTA_ORG_URL"] = "http://example.okta.com"
sent = []
try:
okta_sync.fetch_users(http=client(lambda r: sent.append(r) or httpx.Response(200, json=[user("a")])))
chk("an http org URL raises", False)
except okta_sync.OktaSyncError:
chk("an http org URL raises", True)
chk("...and no request was made (token never sent over http)", sent == [])
# ── 5. never leaks the token to another host, never loops forever ─────────────
print("\n5. next-link safety")
reset_env()
hosts = []
def evil_link(req):
hosts.append(req.url.host)
return httpx.Response(200, json=[user("alice")],
headers={"Link": '<https://evil.example/steal>; rel="next"'})
raises(evil_link, "a next link to another host")
chk("...the other host was never contacted", hosts == ["example.okta.com"])
count = []
def endless(req):
count.append(1)
return httpx.Response(200, json=[user("alice")],
headers={"Link": f'<{ORG}/api/v1/users?after={len(count)}>; rel="next"'})
raises(endless, "an endless next chain (page cap)")
chk(f"...stops at the cap ({okta_sync.MAX_PAGES} requests)", len(count) == okta_sync.MAX_PAGES, str(len(count)))
# ── 6. org URL and identity attribute ─────────────────────────────────────────
print("\n6. org URL and identity attribute")
reset_env()
chk("org URL is the issuer's scheme and host", okta_sync.org_url() == ORG)
os.environ["OKTA_ORG_URL"] = "https://corp.okta.com/"
chk("OKTA_ORG_URL wins, trailing slash removed", okta_sync.org_url() == "https://corp.okta.com")
reset_env()
okta_auth.IDENTITY_CLAIM = "email"
body = [{"status": "ACTIVE", "profile": {"login": "alice", "email": "alice@x.com"}}]
chk("claim 'email' reads profile.email", okta_sync.fetch_users(http=client(ok(body)))[0].identity == "alice@x.com")
okta_auth.IDENTITY_CLAIM = "sAMAccountName"
body = [{"status": "ACTIVE", "profile": {"login": "alice@x.com", "sAMAccountName": "alice"}}]
chk("a custom claim reads profile[<claim>]", okta_sync.fetch_users(http=client(ok(body)))[0].identity == "alice")
os.environ["OKTA_SYNC_PROFILE_ATTRIBUTE"] = "login"
chk("OKTA_SYNC_PROFILE_ATTRIBUTE overrides", okta_sync.fetch_users(http=client(ok(body)))[0].identity == "alice@x.com")
# ── 7. the token never leaks ──────────────────────────────────────────────────
print("\n7. the token never leaks")
reset_env()
chk("describe() has no token", TOKEN not in okta_sync.describe() and "configured" in okta_sync.describe())
os.environ["OKTA_API_TOKEN"] = ""
chk("describe() says DISABLED without a token", "DISABLED" in okta_sync.describe())
reset_env()
every_log = " ".join(cap.lines)
chk("no log line holds the API token", TOKEN not in every_log)
chk("no log line holds the client secret", CLIENT_SECRET not in every_log)
# ── 8. the whole fetch is time-bounded (T13.4 relies on it) ──────────────────
print("\n8. the whole fetch is time-bounded")
reset_env()
saved = okta_sync.TOTAL_DEADLINE_SECONDS
okta_sync.TOTAL_DEADLINE_SECONDS = 0.0
def slow_pages(req):
return httpx.Response(200, json=[user("alice")],
headers={"Link": f'<{ORG}/api/v1/users?after=x>; rel="next"'})
raises(slow_pages, "a fetch that runs past the total deadline")
okta_sync.TOTAL_DEADLINE_SECONDS = saved
chk("the real deadline is under the shortest allowed interval",
okta_sync.TOTAL_DEADLINE_SECONDS < okta_sync.INTERVAL_MIN_SECONDS)
print()
print("ALL PASSED" if not fails else f"{fails} FAILED")
sys.exit(0 if not fails else 1)

View File

@@ -0,0 +1,218 @@
#!/usr/bin/env python3
"""Does the Okta deprovisioning sync fail closed? - D18 / T13.3.
The rule that must never be skipped or weakened: when Okta cannot be asked, or
its answer cannot be trusted, the sync changes NO account and writes NO audit
row. Treating "could not reach Okta" as "nobody is active" would lock the whole
company out on one bad afternoon.
Also checked: the sync only ever disables (it never re-enables, under any
input), writes one audit row per change with a non-human actor, leaves an
already-disabled account alone, logs only changes, and keeps temporary Okta
states (PASSWORD_EXPIRED, LOCKED_OUT, RECOVERY) from disabling anyone.
Self-contained: its own in-memory SQLite, no server, no network, no browser.
Exit 0 all passed, 1 a failure.
"""
import os
import sys
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
import httpx # noqa: E402
from sqlalchemy import create_engine, select # noqa: E402
from sqlalchemy.orm import sessionmaker # noqa: E402
from sqlalchemy.pool import StaticPool # noqa: E402
from server import models, okta_auth, okta_sync # noqa: E402
from server.okta_sync import OktaSyncError, OktaUser, run_sync # noqa: E402
fails = 0
def chk(label, ok, extra=""):
global fails
print(("PASS " if ok else "FAIL ") + label + (f" [{extra}]" if (extra and not ok) else ""))
if not ok:
fails += 1
engine = create_engine("sqlite://", connect_args={"check_same_thread": False}, poolclass=StaticPool)
models.Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine, autoflush=False, future=True)
def seed(rows):
"""rows: list of (username, is_active[, email]). Wipes users and audit rows first."""
with Session() as db:
db.query(models.AuditLog).delete()
db.query(models.User).delete()
for r in rows:
name, active = r[0], r[1]
db.add(models.User(id="u_" + name, username=name, is_active=active,
email=(r[2] if len(r) > 2 else "")))
db.commit()
def state():
with Session() as db:
users = {u.username: u.is_active for u in db.scalars(select(models.User))}
audits = [a.to_dict() for a in db.scalars(select(models.AuditLog))]
return users, audits
def fixed(*pairs):
return lambda: [OktaUser(identity=i, status=s) for i, s in pairs]
# ── 1. the failure rule ───────────────────────────────────────────────────────
print("\n1. FAIL CLOSED: a failed or untrusted fetch changes nothing")
BASE = [("alice", True), ("bob", True), ("carol", True)]
def boom(exc):
def f():
raise exc
return f
failing = [
("OktaSyncError (auth failure)", boom(OktaSyncError("HTTP 401"))),
("RuntimeError", boom(RuntimeError("boom"))),
("ConnectionError", boom(ConnectionError("down"))),
("TimeoutError", boom(TimeoutError("slow"))),
("ValueError", boom(ValueError("bad"))),
("KeyError", boom(KeyError("x"))),
("fetch returns an empty list", lambda: []),
("fetch returns None", lambda: None),
("fetch returns garbage (strings)", lambda: ["alice", "bob"]),
("fetch returns garbage (dicts)", lambda: [{"identity": "alice"}]),
]
for label, fetch in failing:
seed(BASE)
before = state()
with Session() as db:
res = run_sync(db, fetch=fetch)
after = state()
chk(f"{label}: ok is False", res.ok is False)
chk(f"{label}: zero accounts changed", after[0] == before[0] == {"alice": True, "bob": True, "carol": True})
chk(f"{label}: zero audit rows", after[1] == [] and res.disabled == [])
print("\n ...and through the real client, not a stand-in")
okta_auth.ISSUER = "https://example.okta.com/oauth2/default"
os.environ["OKTA_API_TOKEN"] = "tok"
for code in (401, 403, 429, 500):
seed(BASE)
c = httpx.Client(transport=httpx.MockTransport(lambda r, k=code: httpx.Response(k, json={})))
with Session() as db:
res = run_sync(db, fetch=lambda c=c: okta_sync.fetch_users(http=c))
u, a = state()
chk(f"real client, HTTP {code}: nothing changed, no audit rows",
(not res.ok) and all(u.values()) and a == [])
seed(BASE)
os.environ["OKTA_API_TOKEN"] = ""
with Session() as db:
res = run_sync(db) # fetch=None: the default path, no token set
u, a = state()
chk("default fetch with no token: nothing changed, no audit rows", (not res.ok) and all(u.values()) and a == [])
print("\n ...and when saving fails")
seed(BASE)
with Session() as db:
real_commit = db.commit
def bad_commit():
raise RuntimeError("disk full")
db.commit = bad_commit
res = run_sync(db, fetch=fixed(("alice", "ACTIVE"))) # bob and carol would be disabled
u, a = state()
chk("a failed commit keeps no change and no audit row", (not res.ok) and all(u.values()) and a == [])
print("\n ...and when no local account matches any Okta user (wrong identity attribute)")
seed(BASE)
with Session() as db:
res = run_sync(db, fetch=fixed(("zed@x.com", "ACTIVE"), ("yan@x.com", "ACTIVE")))
u, a = state()
chk("zero matches is refused, not read as 'everyone left'", (not res.ok) and all(u.values()) and a == [])
chk("...and the error says why", "identity attribute" in res.error)
# ── 2. what it does disable ───────────────────────────────────────────────────
print("\n2. a good run disables the right accounts")
seed([("alice", True), ("bob", True), ("carol", True), ("dave", True),
("erin", True), ("frank", True), ("gina", True), ("hank", True)])
okta = fixed(("alice", "ACTIVE"), ("bob", "SUSPENDED"),
# carol is MISSING from the list
("dave", "DEPROVISIONED"), ("erin", "PASSWORD_EXPIRED"),
("frank", "LOCKED_OUT"), ("gina", "RECOVERY"), ("hank", "ACTIVE"))
with Session() as db:
res = run_sync(db, fetch=okta)
users, audits = state()
chk("run is ok", res.ok, res.error)
chk("missing, SUSPENDED and DEPROVISIONED are disabled",
sorted(res.disabled) == ["bob", "carol", "dave"] and
[users[n] for n in ("bob", "carol", "dave")] == [False, False, False])
chk("ACTIVE stays active", users["alice"] is True and users["hank"] is True)
chk("PASSWORD_EXPIRED, LOCKED_OUT and RECOVERY do NOT disable (temporary states)",
users["erin"] is True and users["frank"] is True and users["gina"] is True)
chk("one audit row per disabled account", len(audits) == 3, str(len(audits)))
row = next(a for a in audits if a["summary"] == "bob")
chk("audit actor is the sync job, not a person", {a["actor"] for a in audits} == {"system:okta_sync"})
chk("audit action is user_deprovisioned_by_sync", {a["action"] for a in audits} == {"user_deprovisioned_by_sync"})
chk("audit row names the user and the Okta status found",
row["entity_type"] == "user" and row["entity_id"] == "u_bob" and row["detail"].get("okta_status") == "SUSPENDED")
chk("a missing user is recorded as MISSING",
next(a for a in audits if a["summary"] == "carol")["detail"].get("okta_status") == "MISSING")
print("\n ...run it again: only changes are logged")
with Session() as db:
res2 = run_sync(db, fetch=okta)
users2, audits2 = state()
chk("second run is ok, disables nobody", res2.ok and res2.disabled == [])
chk("second run writes no new audit rows (no duplicates)", len(audits2) == 3)
chk("accounts unchanged by the second run", users2 == users)
# ── 3. never re-enables, never touches a disabled account ─────────────────────
print("\n3. never re-enables, never touches an already-disabled account")
seed([("alice", True), ("hank", False), ("gina", False)])
with Session() as db:
res = run_sync(db, fetch=fixed(("alice", "ACTIVE"), ("hank", "ACTIVE"))) # gina missing
users, audits = state()
chk("a disabled account that is ACTIVE in Okta stays disabled", users["hank"] is False)
chk("a disabled account that is missing from Okta stays as it is", users["gina"] is False)
chk("no audit row for either (nothing changed)", audits == [] and res.disabled == [])
src = open(os.path.join(ROOT, "server", "okta_sync.py")).read()
chk("no assignment of is_active to True anywhere in okta_sync.py",
"is_active = True" not in src and "is_active=True" not in src.replace("is_active.is_(True)", ""))
# ── 4. matching ───────────────────────────────────────────────────────────────
print("\n4. matching")
seed([("ALICE", True), ("bob", True, "Bob@X.com"), ("carol", True)])
with Session() as db:
res = run_sync(db, fetch=fixed(("alice", "ACTIVE"), ("bob@x.com", "ACTIVE"), ("carol", "SUSPENDED")))
users, _ = state()
chk("username match ignores case", users["ALICE"] is True)
chk("email match works, ignoring case", users["bob"] is True)
chk("a matching SUSPENDED user is disabled", users["carol"] is False)
seed([("alice", True), ("bob", True), ("carol", True)])
with Session() as db:
run_sync(db, fetch=fixed(("alice", "ACTIVE"), ("bob", "ACTIVE"), ("bob", "SUSPENDED"),
("carol", "SUSPENDED"), ("carol", "DEPROVISIONED")))
users, _ = state()
chk("an identity listed twice, one entry active: stays active", users["bob"] is True)
chk("an identity listed twice, every entry disabling: disabled", users["carol"] is False)
# ── 5. the secret never reaches the audit row or the result ───────────────────
print("\n5. no secret in what is stored or returned")
seed(BASE)
os.environ["OKTA_API_TOKEN"] = "SECRET-TOKEN-123"
c = httpx.Client(transport=httpx.MockTransport(lambda r: httpx.Response(401, json={})))
with Session() as db:
res = run_sync(db, fetch=lambda: okta_sync.fetch_users(http=c))
chk("the failure message has no token", "SECRET-TOKEN-123" not in res.error and "example.okta.com" not in res.error)
print()
print("ALL PASSED" if not fails else f"{fails} FAILED")
sys.exit(0 if not fails else 1)

View File

@@ -0,0 +1,242 @@
#!/usr/bin/env python3
"""Does the Okta sync run on a schedule without doubling up? - D18 / T13.4.
The api container runs 2 gunicorn workers, and each starts its own timer. Checked
here: exactly one of them wins each cycle (including a race on a real file
database), a restart inside the interval does not start a second run, the
interval is configurable and cannot go below the minimum, the timer thread starts
and stops, nothing starts when the sync is not configured, and a failure inside
the scheduled path still changes nothing.
Self-contained: a throwaway SQLite file, threads, no server, no network, no browser.
Exit 0 all passed, 1 a failure.
"""
import os
import sys
import tempfile
import threading
import time
from datetime import datetime, timedelta, timezone
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
from sqlalchemy import create_engine, select # noqa: E402
from sqlalchemy.orm import sessionmaker # noqa: E402
from server import models, okta_auth, okta_sync # noqa: E402
from server.okta_sync import OktaSyncError, OktaUser # noqa: E402
fails = 0
def chk(label, ok, extra=""):
global fails
print(("PASS " if ok else "FAIL ") + label + (f" [{extra}]" if (extra and not ok) else ""))
if not ok:
fails += 1
tmp = tempfile.mkdtemp(prefix="oktasync_")
engine = create_engine(f"sqlite:///{os.path.join(tmp, 't.db')}",
connect_args={"check_same_thread": False, "timeout": 20}, future=True)
models.Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine, autoflush=False, future=True)
INTERVAL = 900
os.environ["OKTA_SYNC_INTERVAL_SECONDS"] = str(INTERVAL)
T0 = datetime(2026, 10, 16, 12, 0, 0, tzinfo=timezone.utc)
def wipe(users=()):
with Session() as db:
db.query(models.AuditLog).delete()
db.query(models.User).delete()
db.query(models.AppSetting).filter(models.AppSetting.key == okta_sync.CLAIM_KEY).delete()
for name, active in users:
db.add(models.User(id="u_" + name, username=name, is_active=active))
db.commit()
def users_state():
with Session() as db:
return ({u.username: u.is_active for u in db.scalars(select(models.User))},
db.query(models.AuditLog).count())
# ── 1. interval ───────────────────────────────────────────────────────────────
print("\n1. interval")
def iv(v):
if v is None:
os.environ.pop("OKTA_SYNC_INTERVAL_SECONDS", None)
else:
os.environ["OKTA_SYNC_INTERVAL_SECONDS"] = v
return okta_sync.interval_seconds()
chk("default is 900 seconds (15 minutes)", iv(None) == 900 and iv("") == 900)
chk("an env value is used", iv("600") == 600 and iv("3600") == 3600)
chk("not a number: the default", iv("soon") == 900 and iv("1.5") == 900)
chk("under the minimum is raised to 300 (a slow cycle can never overlap the next)",
iv("60") == 300 and iv("0") == 300 and iv("-5") == 300)
chk("a fetch is bounded below the minimum interval",
okta_sync.TOTAL_DEADLINE_SECONDS < okta_sync.INTERVAL_MIN_SECONDS)
iv(str(INTERVAL))
# ── 2. the claim ──────────────────────────────────────────────────────────────
print("\n2. the claim gate")
wipe()
with Session() as db:
chk("first ever claim wins (the insert is the claim)", okta_sync.claim(db, INTERVAL, now=T0) is True)
with Session() as db:
chk("a second claim in the same instant loses", okta_sync.claim(db, INTERVAL, now=T0) is False)
with Session() as db:
chk("one second before the interval is up: loses",
okta_sync.claim(db, INTERVAL, now=T0 + timedelta(seconds=INTERVAL - 1)) is False)
with Session() as db:
chk("when the interval is up: wins",
okta_sync.claim(db, INTERVAL, now=T0 + timedelta(seconds=INTERVAL)) is True)
with Session() as db:
chk("and then the next caller loses again",
okta_sync.claim(db, INTERVAL, now=T0 + timedelta(seconds=INTERVAL + 1)) is False)
# ── 3. the race ───────────────────────────────────────────────────────────────
print("\n3. exactly one of many workers wins (a real race on a file database)")
def race(n=8):
barrier = threading.Barrier(n)
wins, errors = [], []
def worker():
try:
with Session() as db:
barrier.wait()
wins.append(okta_sync.claim(db, INTERVAL, now=datetime.now(timezone.utc)))
except Exception as exc: # a lost race may surface as an error
errors.append(type(exc).__name__)
wins.append(False)
ts = [threading.Thread(target=worker) for _ in range(n)]
[t.start() for t in ts]
[t.join() for t in ts]
return sum(1 for w in wins if w), errors
wipe()
won, errs = race()
chk("no row yet, 8 workers at once: exactly 1 wins (the insert race)", won == 1, f"{won} {errs}")
bad = []
for i in range(15):
with Session() as db: # age the claim so the next race is due again
db.query(models.AppSetting).filter(models.AppSetting.key == okta_sync.CLAIM_KEY).update(
{"updated_at": datetime.now(timezone.utc) - timedelta(seconds=INTERVAL + 5)})
db.commit()
won, errs = race()
if won != 1:
bad.append((i, won, errs))
chk("row due, 8 workers at once, 15 rounds: exactly 1 wins every round", not bad, str(bad))
# ── 4. one tick ───────────────────────────────────────────────────────────────
print("\n4. a tick runs the sync once per cycle")
wipe([("alice", True), ("bob", True)])
calls = []
def fetch_ok():
calls.append(1)
return [OktaUser("alice", "ACTIVE"), OktaUser("bob", "SUSPENDED")]
r1 = okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0)
chk("first tick claims and runs", r1 is not None and r1.ok and r1.disabled == ["bob"])
r2 = okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0 + timedelta(seconds=30))
chk("a tick 30 seconds later does nothing (not due)", r2 is None and len(calls) == 1)
r3 = okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0 + timedelta(seconds=INTERVAL))
chk("a tick when due runs again", r3 is not None and r3.ok and len(calls) == 2)
print("\n ...a restart inside the interval does not start a second run")
wipe([("alice", True), ("bob", True)])
calls.clear()
okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0)
# "restart": a brand-new caller with nothing in memory, 2 minutes later
r = okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0 + timedelta(seconds=120))
chk("the claim row outlives the process: no duplicate run", r is None and len(calls) == 1)
print("\n ...two workers tick in the same instant")
wipe([("alice", True), ("bob", True)])
calls.clear()
results = []
barrier = threading.Barrier(2)
def tick():
barrier.wait()
results.append(okta_sync.run_if_due(Session, fetch=fetch_ok))
ts = [threading.Thread(target=tick) for _ in range(2)]
[t.start() for t in ts]
[t.join() for t in ts]
chk("Okta is asked once, not twice", len(calls) == 1, str(len(calls)))
chk("exactly one worker got a result", sum(1 for r in results if r is not None) == 1)
_, audits = users_state()
chk("one audit row, not two", audits == 1, str(audits))
# ── 5. fail closed through the scheduled path ─────────────────────────────────
print("\n5. FAIL CLOSED through the scheduled wrapper (not just run_sync)")
for label, exc in (("OktaSyncError", OktaSyncError("HTTP 500")), ("RuntimeError", RuntimeError("x")),
("TimeoutError", TimeoutError("t"))):
wipe([("alice", True), ("bob", True)])
def failing(e=exc):
raise e
r = okta_sync.run_if_due(Session, fetch=failing, now=T0)
u, a = users_state()
chk(f"{label} in the scheduled path: ok is False, zero changes, zero audit rows",
r is not None and r.ok is False and all(u.values()) and a == 0)
wipe([("alice", True), ("bob", True)])
def broken_factory():
raise RuntimeError("database is down")
chk("the database failing at claim time: the tick ends quietly, returns None",
okta_sync.run_if_due(broken_factory, fetch=fetch_ok, now=T0) is None)
u, a = users_state()
chk("...and nothing changed", all(u.values()) and a == 0)
# ── 6. the thread ─────────────────────────────────────────────────────────────
print("\n6. the timer thread")
os.environ["OKTA_API_TOKEN"] = ""
okta_auth.ISSUER = "https://example.okta.com/oauth2/default"
chk("no token: nothing starts", okta_sync.start_scheduler(Session) is None)
chk("...no thread named okta-sync", not any(t.name == "okta-sync" for t in threading.enumerate()))
os.environ["OKTA_API_TOKEN"] = "tok"
wipe([("alice", True), ("bob", True)])
okta_sync.START_DELAY_SECONDS = 0.05
okta_sync.TICK_SECONDS = 0.05
ran = []
real_fetch = okta_sync.fetch_users
okta_sync.fetch_users = lambda http=None: (ran.append(1) or
[OktaUser("alice", "ACTIVE"), OktaUser("bob", "DEPROVISIONED")])
stop = okta_sync.start_scheduler(Session)
chk("configured: a stop Event comes back", stop is not None)
time.sleep(1.2) # about 20 ticks
u, a = users_state()
chk("the thread ran the sync on its own (bob was disabled without a manual call)", u["bob"] is False)
chk("many ticks, one run (the claim gate holds)", len(ran) == 1 and a == 1, f"ran={len(ran)} audits={a}")
th = [t for t in threading.enumerate() if t.name == "okta-sync"]
chk("one timer thread is alive", len(th) == 1 and th[0].is_alive())
chk("the thread is a daemon (never blocks shutdown)", th[0].daemon)
stop.set()
th[0].join(timeout=3)
chk("setting the stop Event ends the thread", not th[0].is_alive())
okta_sync.fetch_users = real_fetch
os.environ["OKTA_API_TOKEN"] = ""
print()
print("ALL PASSED" if not fails else f"{fails} FAILED")
sys.exit(0 if not fails else 1)

153
tests/okta_sync_ui_check.py Normal file
View File

@@ -0,0 +1,153 @@
#!/usr/bin/env python3
"""Does an auto-disable read clearly in the admin console? - D18 / T13.5.
T13.5 builds no new UI. It confirms that an account the Okta sync disabled shows
up legibly in the surface admins already use (the Activity log card on
admin.html), without special-casing, and that the disabled account is refused on
its very next request. This runs the REAL sync against a throwaway database,
then reads the real page in a headless browser at 1440px and 390px.
It does not rely on CR-019's Activity & usage card: only the Activity log card
(AuditLog) is read.
Boots its own throwaway SQLite + uvicorn + headless browser; run it alone.
Screenshots go to a folder printed at the end (WP_SHOTS_DIR overrides it).
Exit 0 all passed, 1 a failure, 2 could not run.
"""
import json
import os
import re
import sys
import tempfile
import time
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import cdp # noqa: E402
from browser_check import seed, start_server, chk, _PASS, _FAIL # noqa: E402
from qa_gate_check import api # noqa: E402
def ascii_(v, n=300):
return re.sub(r"\s+", " ", str(v)).encode("ascii", "replace").decode()[:n]
ROWS_JS = """JSON.stringify([...document.querySelectorAll('#audit-admin tbody tr')].map(
tr => [...tr.children].map(td => td.textContent.trim())))"""
def main():
exe = cdp.find_browser()
if not exe:
print("no headless-capable browser found; set WP_BROWSER.")
return 2
tmpdir = tempfile.mkdtemp(prefix="wpsuite-syncui-")
shots = os.environ.get("WP_SHOTS_DIR") or os.path.join(tmpdir, "shots")
db_path = os.path.join(tmpdir, "check.db")
server = browser = None
try:
tok = seed(db_path)
print("\n1. run the real sync against the throwaway database")
from server.db import SessionLocal
from server import okta_sync
from server.okta_sync import OktaUser
okta = lambda: [OktaUser("root", "ACTIVE"), OktaUser("sue", "ACTIVE"),
OktaUser("pat", "SUSPENDED"), OktaUser("mix", "ACTIVE"),
OktaUser("sam", "ACTIVE"), OktaUser("legacy", "ACTIVE")] # bob is missing
with SessionLocal() as db:
res = okta_sync.run_sync(db, fetch=okta)
chk("the sync disabled pat (SUSPENDED) and bob (missing)",
res.ok and sorted(res.disabled) == ["bob", "pat"], ascii_((res.ok, res.error, res.disabled)))
port = cdp.free_port()
base = "http://127.0.0.1:%d" % port
server = start_server(port, db_path)
if not server:
print("the server did not start")
return 2
print("\n2. a disabled account is refused on its very next request")
st, _ = api(base, "/api/auth/me", tok["pat"])
chk("pat (auto-disabled) is refused", st in (401, 403), st)
st, _ = api(base, "/api/auth/me", tok["bob"])
chk("bob (auto-disabled) is refused", st in (401, 403), st)
st, _ = api(base, "/api/auth/me", tok["mix"])
chk("mix (still active in Okta) still works", st == 200, st)
print("\n3. the audit rows through the API")
st, audit = api(base, "/api/audit?entity_type=user&limit=50", tok["root"])
mine = [e for e in (audit or []) if e.get("actor") == "system:okta_sync"]
chk("an admin gets both rows", st == 200 and len(mine) == 2, ascii_((st, len(mine))))
chk("each row names the action, the user, and the Okta status",
{(e["summary"], e["detail"].get("okta_status")) for e in mine} == {("pat", "SUSPENDED"), ("bob", "MISSING")},
ascii_([(e["summary"], e["detail"]) for e in mine]))
st, audit_sue = api(base, "/api/audit?entity_type=user&limit=50", tok["sue"])
chk("a project super user does not see them (they belong to no project)",
st == 200 and not [e for e in audit_sue if e.get("actor") == "system:okta_sync"])
print("\n4. the admin console, as an admin")
browser = cdp.Browser(exe)
page = browser.page()
for width, label in ((1440, "1440"), (390, "390")):
page.clear_cookies()
page.set_cookie("wp_session", tok["root"])
page.viewport(width, 900, mobile=(width < 500))
page.goto(base + "/admin.html")
time.sleep(2.0)
page.eval("void loadAudit()")
time.sleep(1.2)
rows = json.loads(page.eval(ROWS_JS) or "[]")
sync_rows = [r for r in rows if "system:okta_sync" in r]
chk("[%s] the Activity log lists both auto-disables" % label, len(sync_rows) == 2,
ascii_(rows[:4]))
if sync_rows:
r = next((x for x in sync_rows if "pat" in x), sync_rows[0])
# columns: When, Who, Action, Type, Item, Detail
chk("[%s] Who reads 'system:okta_sync', not a person" % label, r[1] == "system:okta_sync", r)
chk("[%s] Action reads 'user deprovisioned by sync'" % label,
r[2] == "user deprovisioned by sync", r)
chk("[%s] Type and Item name the account" % label, r[3] == "user" and r[4] == "pat", r)
chk("[%s] Detail gives the Okta status found" % label,
"SUSPENDED" in r[5] and "okta" in r[5].lower(), r)
miss = next((x for x in sync_rows if "bob" in x), None)
chk("[%s] a person missing from Okta reads 'MISSING'" % label,
bool(miss) and "MISSING" in miss[5], miss)
# filter + search work on it with no special-casing
page.eval("document.getElementById('audit-search').value='okta_sync'; renderAudit();")
rows = json.loads(page.eval(ROWS_JS) or "[]")
chk("[%s] searching 'okta_sync' finds exactly the sync rows" % label,
len(rows) == 2 and all("system:okta_sync" in r for r in rows), ascii_(rows))
page.eval("document.getElementById('audit-search').value='';"
"document.getElementById('audit-type').value='user'; renderAudit();")
rows = json.loads(page.eval(ROWS_JS) or "[]")
chk("[%s] the 'User accounts' filter keeps them" % label,
len([r for r in rows if "system:okta_sync" in r]) == 2)
page.eval("document.getElementById('audit-type').value=''; renderAudit();")
over = page.eval("document.documentElement.scrollWidth - document.documentElement.clientWidth")
chk("[%s] no sideways scroll on the page" % label, (over or 0) <= 0, over)
card = page.eval("(()=>{const e=document.getElementById('audit-admin');"
"e.scrollIntoView();return e.getBoundingClientRect().width>0})()")
chk("[%s] the card is on screen" % label, bool(card))
time.sleep(0.4)
page.screenshot(os.path.join(shots, "admin-activity-log-%s.png" % label), full_page=False)
errs = [e for e in page.js_errors() if "beforeunload" not in e]
chk("no JavaScript errors", not errs, ascii_(errs[:3]))
print("\n screenshots: " + shots)
finally:
if browser:
browser.close()
if server:
server.terminate()
print("\n" + "-" * 54)
print("%d/%d checks passed." % (len(_PASS), len(_PASS) + len(_FAIL)))
for f in _FAIL:
print(" - " + f)
return 1 if _FAIL else 0
if __name__ == "__main__":
sys.exit(main())

236
tests/session_idle_check.py Normal file
View File

@@ -0,0 +1,236 @@
#!/usr/bin/env python3
"""Does the sliding session with a hard ceiling work? - D18 / T13.1 (checked in T13.6).
T13.1 shipped with its done-when list unchecked and no test of its own; the old
session tests had no idea about idle time. This is that test.
1. Continuous activity keeps a session alive past 30 minutes, but never past the
absolute ceiling (AUTH_SESSION_HOURS) measured from the ORIGINAL sign-in.
2. A session idle for more than 30 minutes is refused on its next request.
3. The cookie is not rewritten on every request (the refresh is throttled).
4. A token from before this shipped (no login_at) still gets a hard ceiling,
from its iat.
5. A refresh carries identity and login_at forward unchanged.
6. The two variables are documented, and both defaults are flagged as proposed.
Self-contained: throwaway SQLite, in-process test client, no network, no browser.
Exit 0 all passed, 1 a failure.
"""
import os
import re
import sys
import tempfile
from datetime import datetime, timedelta, timezone
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
os.environ["DATABASE_URL"] = "sqlite:///" + os.path.join(tempfile.mkdtemp(prefix="wp-idle-"), "t.db").replace("\\", "/")
os.environ["AUTH_SECRET_KEY"] = "session-idle-check-secret-not-for-production"
os.environ.pop("AUTH_IDLE_MINUTES", None)
os.environ.pop("AUTH_SESSION_HOURS", None)
import jwt # noqa: E402
from fastapi.testclient import TestClient # noqa: E402
from server import app as appmod, auth, models # noqa: E402
from server.db import Base, SessionLocal, engine # noqa: E402
if not hasattr(auth, "maybe_refresh_token"):
print("SKIP: T13.1 (sliding session) is not on this branch. It waits for T14.1.")
sys.exit(0)
fails = 0
def chk(label, ok, extra=""):
global fails
print(("PASS " if ok else "FAIL ") + label + (f" [{extra}]" if (extra and not ok) else ""))
if not ok:
fails += 1
Base.metadata.create_all(bind=engine)
with SessionLocal() as db:
db.add(models.User(id="u1", username="alice", email="a@x.test", role=auth.ROLE_ADMIN))
db.commit()
USER = db.get(models.User, "u1")
db.expunge(USER)
# A route the auth gate covers. NOT /api/auth/me: everything under /api/auth/ is
# exempt from the gate (auth._EXEMPT_PREFIXES), so no session refresh happens there
# and a probe on it would pass without testing anything.
PROBE = "/api/projects"
IDLE = auth.IDLE_MINUTES
CEIL_H = auth.SESSION_HOURS
NOW = datetime.now(timezone.utc)
def mint(login_at=None, iat=None, exp=None, **extra):
"""A token with exactly the claims given (None = leave the claim out)."""
p = {"sub": "u1", "username": "alice", "role": auth.ROLE_ADMIN, "ver": 0}
if iat is not None:
p["iat"] = iat
if login_at is not None:
p["login_at"] = login_at.timestamp()
if exp is not None:
p["exp"] = exp
p.update(extra)
return jwt.encode(p, auth.SECRET_KEY, algorithm=auth.JWT_ALG)
def raw_claims(tok):
return jwt.decode(tok, auth.SECRET_KEY, algorithms=[auth.JWT_ALG], options={"verify_exp": False, "verify_iat": False, "verify_nbf": False})
class FakeClock(datetime):
_now = NOW
@classmethod
def now(cls, tz=None):
return cls._now
real_dt = auth.datetime
def at(t):
FakeClock._now = t
auth.datetime = FakeClock
def real():
auth.datetime = real_dt
print("defaults: idle %d min, ceiling %d h" % (IDLE, CEIL_H))
# ── 1. continuous activity, then the ceiling ──────────────────────────────────
print("\n1. continuous activity stays alive past 30 minutes, but is cut off at the ceiling")
at(NOW)
tok = auth.create_token(USER)
c = raw_claims(tok)
chk("a new token carries login_at", "login_at" in c)
chk("a new token expires one idle window out, not at the ceiling",
abs(c["exp"] - (NOW + timedelta(minutes=IDLE)).timestamp()) < 2)
login_at = NOW
cur = c
alive_past_30 = False
rewrites = 0
last_exp = cur["exp"]
t = NOW
STEP = timedelta(minutes=1) # an active user: a request every minute
end = NOW + timedelta(hours=CEIL_H + 2)
cut_off_at = None
while t < end:
t += STEP
at(t)
if cur["exp"] <= t.timestamp(): # the session already lapsed
cut_off_at = t
break
if t - NOW > timedelta(minutes=IDLE):
alive_past_30 = True
new = auth.maybe_refresh_token(cur)
if new:
rewrites += 1
cur = raw_claims(new)
last_exp = cur["exp"]
real()
chk("with a request every minute the session is alive well past 30 minutes", alive_past_30)
chk("it is never extended past the ceiling (last exp <= login + %d h)" % CEIL_H,
last_exp <= (login_at + timedelta(hours=CEIL_H)).timestamp() + 1,
str(datetime.fromtimestamp(last_exp, timezone.utc)))
chk("it IS cut off once the ceiling is reached, however active the session",
cut_off_at is not None and cut_off_at <= login_at + timedelta(hours=CEIL_H) + timedelta(minutes=IDLE) + STEP,
str(cut_off_at))
chk("the claims after the last refresh still carry the ORIGINAL login_at",
abs(cur["login_at"] - login_at.timestamp()) < 1)
steps = int((CEIL_H * 60) / 1)
chk("the cookie was rewritten far less often than once per request (%d rewrites for ~%d requests)" % (rewrites, steps),
0 < rewrites < steps / 5, f"{rewrites}/{steps}")
# ── 2. idle ───────────────────────────────────────────────────────────────────
print("\n2. a session idle for more than %d minutes is refused on its next request" % IDLE)
client = TestClient(appmod.app)
idle_tok = mint(login_at=NOW - timedelta(minutes=IDLE + 1), iat=NOW - timedelta(minutes=IDLE + 1),
exp=NOW - timedelta(minutes=1))
client.cookies.set(auth.COOKIE_NAME, idle_tok)
r = client.get(PROBE)
chk("an idle session gets 401", r.status_code == 401, r.status_code)
live_tok = mint(login_at=NOW - timedelta(minutes=5), iat=NOW - timedelta(minutes=5),
exp=NOW + timedelta(minutes=IDLE - 5))
client.cookies.clear()
client.cookies.set(auth.COOKIE_NAME, live_tok)
chk("a session inside the window gets 200", client.get(PROBE).status_code == 200)
# ── 3. throttle ───────────────────────────────────────────────────────────────
print("\n3. the cookie is not rewritten on every request")
fresh = auth.create_token(USER)
client.cookies.clear()
client.cookies.set(auth.COOKIE_NAME, fresh)
cookies_set = 0
for _ in range(20):
r = client.get(PROBE)
if "set-cookie" in r.headers and auth.COOKIE_NAME in r.headers["set-cookie"]:
cookies_set += 1
chk("20 rapid requests on a fresh session: 0 cookie rewrites", cookies_set == 0, cookies_set)
aged = mint(login_at=NOW - timedelta(minutes=20), iat=NOW - timedelta(minutes=20),
exp=NOW + timedelta(minutes=2)) # 2 minutes of idle window left
client.cookies.clear()
client.cookies.set(auth.COOKIE_NAME, aged)
r = client.get(PROBE)
chk("an aged session gets a refreshed cookie on its next request",
r.status_code == 200 and auth.COOKIE_NAME in r.headers.get("set-cookie", ""))
m = re.search(auth.COOKIE_NAME + r"=([^;]+)", r.headers.get("set-cookie", ""))
newc = raw_claims(m.group(1)) if m else {}
chk("the refreshed cookie keeps the original login_at",
abs(newc.get("login_at", 0) - (NOW - timedelta(minutes=20)).timestamp()) < 1)
client.cookies.clear()
client.cookies.set(auth.COOKIE_NAME, m.group(1) if m else "")
r2 = client.get(PROBE)
chk("...and the very next request does not rewrite it again", auth.COOKIE_NAME not in r2.headers.get("set-cookie", ""))
# ── 4. tokens from before D18 ─────────────────────────────────────────────────
print("\n4. a token with no login_at still gets a hard ceiling (via iat)")
old = {"sub": "u1", "username": "alice", "role": auth.ROLE_ADMIN, "ver": 0,
"iat": (NOW - timedelta(hours=CEIL_H + 1)).timestamp(),
"exp": (NOW + timedelta(hours=3)).timestamp()} # the old flat 12 h exp, still valid
chk("past its iat-based ceiling: no refresh, the old exp is not extended", auth.maybe_refresh_token(old) is None)
young = {"sub": "u1", "username": "alice", "role": auth.ROLE_ADMIN, "ver": 0,
"iat": (NOW - timedelta(hours=1)).timestamp(),
"exp": (NOW + timedelta(minutes=2)).timestamp()}
ref = auth.maybe_refresh_token(young)
rc = raw_claims(ref) if ref else {}
chk("inside the ceiling: refreshed, and login_at is taken from iat (not 'now')",
bool(ref) and abs(rc["login_at"] - young["iat"]) < 1)
chk("the refreshed token cannot outlive iat + ceiling",
bool(ref) and rc["exp"] <= young["iat"] + CEIL_H * 3600 + 1)
nothing = {"sub": "u1", "username": "alice", "role": auth.ROLE_ADMIN, "ver": 0,
"exp": (NOW + timedelta(minutes=2)).timestamp()}
chk("no login_at and no iat: no refresh, and no crash", auth.maybe_refresh_token(nothing) is None)
junk = dict(young, login_at="not-a-number")
chk("an unparseable login_at falls back to iat, no crash", auth.maybe_refresh_token(junk) is not None)
# ── 5. identity carried forward ───────────────────────────────────────────────
print("\n5. a refresh carries the identity forward unchanged")
src = {"sub": "u9", "username": "zed", "role": "project_admin", "ver": 7,
"iat": (NOW - timedelta(minutes=20)).timestamp(),
"login_at": (NOW - timedelta(minutes=20)).timestamp(),
"exp": (NOW + timedelta(minutes=2)).timestamp()}
rc = raw_claims(auth.maybe_refresh_token(src))
chk("sub, username, role, ver are unchanged",
(rc["sub"], rc["username"], rc["role"], rc["ver"]) == ("u9", "zed", "project_admin", 7))
# ── 6. documentation ──────────────────────────────────────────────────────────
print("\n6. the two variables are documented, defaults flagged as proposed")
env = open(os.path.join(ROOT, "server", ".env.example"), encoding="utf-8").read()
dep = open(os.path.join(ROOT, "DEPLOYMENT.md"), encoding="utf-8").read()
for name in ("AUTH_IDLE_MINUTES", "AUTH_SESSION_HOURS"):
chk(f"{name} is in .env.example", name in env)
chk(f"{name} is in DEPLOYMENT.md", name in dep)
chk(".env.example calls the defaults proposed, not confirmed", re.search(r"propos", env, re.I) is not None)
chk("DEPLOYMENT.md calls the defaults proposed, not confirmed", re.search(r"propos", dep, re.I) is not None)
print()
print("ALL PASSED" if not fails else f"{fails} FAILED")
sys.exit(0 if not fails else 1)

View File

@@ -1,192 +0,0 @@
#!/usr/bin/env python3
"""Is there exactly one analytics implementation, reported from admin? — D5, T7.10.
Usage analytics existed twice (creator + wizard), five of the nine colliding
globals `creator-frame.md` counted, and the wizard's copy had no caller. One
core survives in wp-usage.js; the pages keep thin track() wrappers; the report
and its downloads live on the admin console behind the same role gate as the
rest of that page. Keys are unchanged, so pre-move data still reads.
Self-contained: throwaway SQLite, its own uvicorn, headless Edge or Chrome.
Exit 0 all passed, 1 a failure, 2 could not run.
"""
import json
import os
import re
import sys
import tempfile
import time
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import cdp # noqa: E402
from browser_check import seed, start_server, chk, _PASS, _FAIL # noqa: E402
from sections_check import set_sop # noqa: E402
from stepper_check import dismiss_dialogs # noqa: E402
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
HTML = os.path.join(ROOT, "html")
def ascii_(v, n=280):
return re.sub(r"\s+", " ", str(v)).encode("ascii", "replace").decode()[:n]
def settle(seconds=0.6):
time.sleep(seconds)
def strip_js(src):
src = re.sub(r"/\*.*?\*/", "", src, flags=re.S)
return "\n".join(re.sub(r"(?<![:'\"])//.*$", "", ln) for ln in src.split("\n"))
def main():
exe = cdp.find_browser()
if not exe:
print("no headless-capable browser found; set WP_BROWSER.")
return 2
# ── 1. the grep half: one implementation, no leftover controls ───────────
print("\n1. grep: one implementation, nothing unreferenced")
files = {}
for name in os.listdir(HTML):
if name.endswith((".js", ".html")):
files[name] = strip_js(open(os.path.join(HTML, name), encoding="utf-8").read())
core_defs = [n for n, src in files.items() if "window.WPUsage" in src]
chk("the WPUsage core is defined in wp-usage.js and only there",
core_defs == ["wp-usage.js"], ascii_(core_defs))
chk("the pages record THROUGH it - no page touches the storage keys directly",
all("wp_iwp_analytics_v1" not in src and "wp_suite_analytics_v1" not in src
for n, src in files.items()
if n not in ("wp-usage.js",) and n.endswith(".js")))
leftovers = {n: re.findall(r"analyticsLoad|analyticsSave|downloadAnalytics|showAnalytics"
r"|ANALYTICS_KEY|USAGE_KEY|usageLoad|downloadUsage", src)
for n, src in files.items() if n != "wp-usage.js"}
leftovers = {n: v for n, v in leftovers.items() if v}
chk("none of the five colliding globals survives anywhere; grep confirms",
not leftovers, ascii_(leftovers))
chk("no analytics control remains on the creator or the wizard; grep confirms",
"Usage data" not in files["wp-creation-index.html"]
and "showAnalytics" not in files["work-package-suite.html"])
chk("both storage keys survive, verbatim, in the core (data continuity)",
"wp_iwp_analytics_v1" in files["wp-usage.js"]
and "wp_suite_analytics_v1" in files["wp-usage.js"])
tmpdir = tempfile.mkdtemp(prefix="wpsuite-usage-")
db_path = os.path.join(tmpdir, "check.db")
server = None
browser = None
try:
tok = seed(db_path)
set_sop(db_path, {})
port = cdp.free_port()
base = "http://127.0.0.1:%d" % port
server = start_server(port, db_path)
browser = cdp.Browser(exe)
page = browser.page()
page.clear_cookies()
page.set_cookie("wp_session", tok["root"])
page.viewport(1440, 900)
# ── 2. recording still works from both tools ─────────────────────────
print("\n2. the tools still record")
page.goto(base + "/wp-creation-index.html?project=projA")
dismiss_dialogs(page)
settle(2.0)
# Plant a LEGACY-format event under the pre-move key: the done-when is
# that data recorded before this task is still readable after it.
page.eval("""(() => {
const d = JSON.parse(localStorage.getItem('wp_iwp_analytics_v1')) || {events: []};
d.events.unshift({ts: '2026-07-01T10:00:00Z', session: 's_legacy',
event: 'legacy_probe_event', detail: null});
localStorage.setItem('wp_iwp_analytics_v1', JSON.stringify(d));
})()""")
n0 = page.eval("WPUsage.load(WPUsage.KEYS.creator).events.length")
page.eval("track('probe_event')")
chk("the creator's track() still lands events under the old key",
page.eval("WPUsage.load(WPUsage.KEYS.creator).events.length") == n0 + 1)
page.goto(base + "/work-package-suite.html?tab=sop")
dismiss_dialogs(page)
settle(2.0)
n0 = page.eval("WPUsage.load(WPUsage.KEYS.wizard).events.length")
page.eval("track('probe_event_wizard')")
chk("the wizard's track() still lands events under its old key",
page.eval("WPUsage.load(WPUsage.KEYS.wizard).events.length") == n0 + 1)
wiz_errors = [e for e in page.js_errors() if "beforeunload" not in e]
chk("...and the wizard page throws no errors without its old globals "
"(the blocked-beforeunload console line is BL-020, filtered not hidden)",
not wiz_errors, ascii_(wiz_errors[:2]))
# ── 3. the report, on admin, behind the admin gate ───────────────────
print("\n3. the admin report")
page.goto(base + "/admin.html")
dismiss_dialogs(page)
settle(2.0)
page.eval("loadUsage()")
settle(0.5)
report = page.eval("(document.getElementById('usage-admin')||{textContent:''}).textContent")
chk("usage data is reachable from admin.html, both tools reported",
"Work package creator" in report and "SOP wizard" in report, ascii_(report, 160))
chk("the pre-move legacy event is readable in the report",
"legacy_probe_event" in report)
chk("this session's fresh events are in it too",
"probe_event" in report and "probe_event_wizard" in report)
chk("each tool offers its download from the report",
page.eval("[...document.querySelectorAll('#usage-admin button')].length") >= 2)
page.viewport(390, 844, mobile=True)
settle(0.6)
fits = page.eval("""(() => {
const box = document.getElementById('usage-admin');
return box && box.scrollWidth <= box.clientWidth + 2
&& document.documentElement.scrollWidth <= 392;
})()""")
chk("the report is usable at 390px - no sideways scrolling", bool(fits))
page.viewport(1440, 900)
settle(0.4)
# The same role gate as the rest of the console: a non-admin sees the
# denied card and no cards, this one included.
page.clear_cookies()
page.set_cookie("wp_session", tok["pat"])
page.goto(base + "/admin.html")
dismiss_dialogs(page)
settle(2.0)
chk("a non-administrator gets the denied notice, not the usage report",
page.eval("""(() => {
const denied = document.getElementById('admin-denied');
const usage = document.getElementById('usage-admin');
const visible = el => !!el && el.offsetParent !== null;
return visible(denied) && !visible(usage);
})()"""))
js_errors = [e for e in page.js_errors() if "beforeunload" not in e]
chk("no JavaScript errors anywhere in this run", not js_errors,
ascii_(js_errors[:2]))
finally:
if browser is not None:
try:
browser.close()
except Exception:
pass
if server is not None:
try:
server.terminate()
except Exception:
pass
print("\n" + "-" * 54)
print("%d/%d checks passed." % (len(_PASS), len(_PASS) + len(_FAIL)))
for f in _FAIL:
print(" - " + f)
return 1 if _FAIL else 0
if __name__ == "__main__":
sys.exit(main())