Release: wave 11 + T13.2-T13.5 (D18 Okta sync), T13.1 held back until T14.1; mobile roadmap docs (BL-062..BL-069)
@@ -404,6 +404,72 @@ Directory page — no shell access needed.
|
||||
unreachable for everyone, including admins, until Okta is restored — the same
|
||||
posture the abandoned LDAPS design took, carried forward deliberately (D16).
|
||||
|
||||
### Accounts the Okta sync disabled (D18)
|
||||
|
||||
**Who this is for:** app admins, and project super users who manage accounts.
|
||||
|
||||
When `OKTA_API_TOKEN` is set, the `api` container asks Okta every 15 minutes
|
||||
(`OKTA_SYNC_INTERVAL_SECONDS`) who is still active. It **disables** a local
|
||||
account when the person is missing from Okta's list, or when Okta shows the
|
||||
status SUSPENDED or DEPROVISIONED. A password expiry, a lockout, or a recovery
|
||||
in progress does not disable anyone. With no token set, the sync is off and the
|
||||
startup log says `Okta sync DISABLED`.
|
||||
|
||||
The sync only ever disables. **It never re-enables an account.** A person comes
|
||||
back only when an admin enables the account by hand.
|
||||
|
||||
**How to see what the sync did**
|
||||
|
||||
1. Open the **Admin Console** (`admin.html`). Find the **Activity log** card.
|
||||
2. Click **Refresh**. Search for `okta_sync`, or set the type to **User accounts**.
|
||||
3. A row from the sync reads: Who `system:okta_sync`, Action `user deprovisioned
|
||||
by sync`, Item the username, Detail `okta_status: SUSPENDED` (or `MISSING` if
|
||||
the person was not in Okta's list at all).
|
||||
|
||||
Only app admins see these rows. They belong to no project.
|
||||
|
||||
**How to re-enable a person**
|
||||
|
||||
1. **Fix the person in Okta first.** Ask the Okta admin to make the person ACTIVE
|
||||
again (or to restore them, for a rehire). If you skip this step, the next sync
|
||||
cycle (up to 15 minutes later) disables the account again.
|
||||
2. Open the **User Directory** (`users.html`).
|
||||
3. Find the person. A disabled account shows the **Enable** button on its row.
|
||||
Click it.
|
||||
4. The person can sign in on their next attempt. The app writes a `user enabled`
|
||||
row to the Activity log under your name.
|
||||
|
||||
**Who may do step 3**
|
||||
|
||||
* An **app admin** may enable any account.
|
||||
* A **project super user** may enable an account only if every project the person
|
||||
belongs to is a project that super user administers. If the person is also on a
|
||||
project that the super user does not administer, the row is read-only and shows
|
||||
the reason. An app admin must make the change.
|
||||
* Nobody can change their own account.
|
||||
|
||||
**If no admin can sign in** (for example, a bad sync run disabled every admin),
|
||||
use the shell on the host. Replace `alice` with the username:
|
||||
|
||||
```bash
|
||||
docker compose exec api python -m server.manage_users list
|
||||
docker compose exec api python -m server.manage_users enable alice
|
||||
```
|
||||
|
||||
**If the sync disabled the wrong people**
|
||||
|
||||
1. Turn the sync off. Remove `OKTA_API_TOKEN` from `.env` and restart the api
|
||||
(`docker compose up -d api`). The sync starts only when the token is set.
|
||||
2. Find the cause before you turn it on again. The usual cause is that the Okta
|
||||
profile attribute does not match `users.username`. See
|
||||
`OKTA_SYNC_PROFILE_ATTRIBUTE` in `server/.env.example`.
|
||||
3. Enable the people who were disabled by mistake (steps above).
|
||||
4. Set the token again and restart.
|
||||
|
||||
If the sync cannot reach Okta, or Okta gives an answer the sync cannot trust, the
|
||||
sync changes **no** account in that cycle and writes a line to the api log:
|
||||
`docker compose logs api | grep "Okta sync"`.
|
||||
|
||||
## Permissions roles
|
||||
|
||||
`User.role` is the **permissions** role; `User.project_role` is the person's **job
|
||||
|
||||
@@ -49,6 +49,18 @@ services:
|
||||
# here instead, so an operator who leaves .env's copy commented out gets
|
||||
# the real default, not a broken one.
|
||||
OKTA_IDENTITY_CLAIM: ${OKTA_IDENTITY_CLAIM:-preferred_username}
|
||||
# Okta deprovisioning sync (D18, wave 13). OFF until OKTA_API_TOKEN is set; the
|
||||
# startup log says which. The token is a read-only Okta API token, NOT the OIDC
|
||||
# client secret above. Empty values here mean "unset" to the code, so the
|
||||
# ${VAR:-} form is safe for all four (unlike OKTA_IDENTITY_CLAIM above, whose
|
||||
# code default only applies when the variable is absent).
|
||||
# Reaching Okta needs outbound https. The `outbound` network the api service
|
||||
# already joins is an ordinary bridge with a gateway, so no new network is
|
||||
# needed; the host firewall must allow https to the Okta org.
|
||||
OKTA_API_TOKEN: ${OKTA_API_TOKEN:-}
|
||||
OKTA_ORG_URL: ${OKTA_ORG_URL:-}
|
||||
OKTA_SYNC_PROFILE_ATTRIBUTE: ${OKTA_SYNC_PROFILE_ATTRIBUTE:-}
|
||||
OKTA_SYNC_INTERVAL_SECONDS: ${OKTA_SYNC_INTERVAL_SECONDS:-}
|
||||
# Optional — SMTP password for WP-assignment emails. Email is off by
|
||||
# default and enabled from the Admin console; this is the only email
|
||||
# secret and it is never stored in the DB. Leave unset until configured.
|
||||
|
||||
|
Before Width: | Height: | Size: 382 KiB After Width: | Height: | Size: 307 KiB |
|
Before Width: | Height: | Size: 315 KiB After Width: | Height: | Size: 260 KiB |
|
Before Width: | Height: | Size: 193 KiB After Width: | Height: | Size: 169 KiB |
|
Before Width: | Height: | Size: 135 KiB After Width: | Height: | Size: 125 KiB |
|
Before Width: | Height: | Size: 78 KiB After Width: | Height: | Size: 66 KiB |
|
Before Width: | Height: | Size: 41 KiB After Width: | Height: | Size: 34 KiB |
@@ -283,7 +283,9 @@ python tests/triage_check.py # A6 - the sidebar answers the stand-up
|
||||
python tests/qa_gate_check.py # CR-014/D2/D9/D10 - QA gate + capture sink 41 checks
|
||||
python tests/files_check.py # CR-007/D8 - drawings upload + real offline 36 checks
|
||||
python tests/sticky_bar_check.py # B6 - save reachable on every wizard step 12 checks
|
||||
python tests/usage_check.py # D5 - one analytics core, admin report 15 checks
|
||||
# tests/usage_check.py (D5) removed at T11.6 - it tested the per-browser
|
||||
# analytics core and admin report, both retired in favour of CR-019's
|
||||
# server-side Activity & usage card (see docs/waves/decisions-2026-09-17.md).
|
||||
python tests/creator_dialogs_check.py # S1 creator - 0 natives, errors at fields 20 checks
|
||||
```
|
||||
|
||||
|
||||
@@ -696,3 +696,124 @@ deliberately deferred.
|
||||
- **Suggested wave or follow-up:** wave 14, alongside T14.1's badge changes:
|
||||
do not show the badge on pages with nothing to sync, and keep it clear of
|
||||
page controls.
|
||||
|
||||
<<<<<<< ours
|
||||
### BL-069 - The sync has no cap on one run and no last-admin guard
|
||||
|
||||
- **Found during:** T13.3 (D18)
|
||||
- **Where:** `server/okta_sync.py` `run_sync()`
|
||||
- **What:** a run disables every account the rules select, however many. If the
|
||||
identity attribute is partly wrong (it matches some accounts and not others),
|
||||
one run can disable most of the company, including every admin. The
|
||||
zero-match guard only stops the case where nothing matches. Disabled accounts
|
||||
cannot sign in, so re-enabling needs an admin who is still active, or the
|
||||
`manage_users.py` command line on the host.
|
||||
- **Why not now:** D18 does not ask for it, and the right limit is a product
|
||||
call: a count, a share of accounts, a rule that the last active admin never
|
||||
disables, or all three.
|
||||
- **Suggested wave or follow-up:** before the first live run, or in T13.6 if
|
||||
Matt decides. Needs Matt.
|
||||
|
||||
=======
|
||||
## Future build: mobile app and scaling
|
||||
|
||||
Logged 2026-10-12 from the meeting with Nick. The answers are in
|
||||
`docs/waves/decisions-2026-10-12.md` (Q2 to Q14). The path from here to a
|
||||
mobile version outside the VPN, across projects, is in
|
||||
`docs/waves/roadmap-mobile-scaling.md`. These are features for a future
|
||||
build. They have no wave and no `CR-`/`D-` id yet. Give each one an id when a
|
||||
wave is planned. Items marked "to confirm" have an open question listed in the
|
||||
decision file ("Still open"). Do not build them until it is answered.
|
||||
|
||||
Follow-up answers from Matt on 2026-10-12 are folded into the entries below.
|
||||
|
||||
Server work is needed for BL-062 to BL-066 (CLAUDE.md, "Frontend and backend
|
||||
boundary"). Do not use `localStorage` as the store of record.
|
||||
|
||||
### BL-062 - Offline Field View for crews
|
||||
|
||||
- **From:** Q2, Q3
|
||||
- **What:** Field View opens offline for up to **7 days** after the last sign-in
|
||||
(today: 12 hours). Only crews in Field View work offline. The creator has no
|
||||
offline mode. When the app opens online again and the account is disabled,
|
||||
wipe that account's local data.
|
||||
- **Why not now:** new scope. It needs an id, and it follows the outbox work in
|
||||
wave 14 (T14.1 to T14.3).
|
||||
- **Suggested wave or follow-up:** future mobile wave. Check it against the D18
|
||||
session rules (idle timeout, absolute ceiling): a 7 day offline window and a
|
||||
session ceiling must agree on what "signed in" means.
|
||||
|
||||
### BL-063 - Offline edit conflicts: merge, notify, log
|
||||
|
||||
- **From:** Q4
|
||||
- **What:** When an offline edit meets a newer server version, the app merges
|
||||
the two, shows a notification to the crew, writes a log entry, and updates the
|
||||
owner. If both sides changed the same field, **the field value wins**, and the
|
||||
app keeps a **history of changes**. This replaces the earlier default (keep
|
||||
both, crew chooses).
|
||||
- **To confirm:** who "the owner" is, how long the history is kept, and who can
|
||||
read it. See the decision file, "Still open" item 2.
|
||||
- **Suggested wave or follow-up:** same wave as BL-062.
|
||||
|
||||
### BL-064 - Field photo limits
|
||||
|
||||
- **From:** Q5
|
||||
- **What:** a limit of **5 MB per photo**. This equals the D8 limit that the
|
||||
server already enforces for drawings (`FILE_MAX_BYTES`), so photos can use the
|
||||
same upload path and the same D19 download headers.
|
||||
- **To confirm:** the count per package, and whether the app resizes a photo
|
||||
before upload (phone photos are often 3 MB to 8 MB). Decision file, "Still
|
||||
open" item 1.
|
||||
- **Suggested wave or follow-up:** same wave as BL-062.
|
||||
|
||||
### BL-065 - Batch print and merged PDF
|
||||
|
||||
- **From:** Q6, Q7, Q8, Q9
|
||||
- **What:** print many packages in one PDF.
|
||||
- Limit: **25 packages on phones**. On desktop the user can scale the filter
|
||||
up to 50 or 100. Open: who sets the larger value (each user, or an admin
|
||||
setting). Decision file, "Still open" item 5.
|
||||
- Drawings go **after the form**, in **upload order**, **scaled to fit**.
|
||||
- SVG drawings are **left out**. A page names each file that was left out.
|
||||
- Every page shows the **package number, the print date, and page X of Y**.
|
||||
- **Related:** BL-000d (merge or list attachments on export) asked the same
|
||||
question for CR-008. This entry answers the layout part of it.
|
||||
- **Suggested wave or follow-up:** future print wave.
|
||||
|
||||
### BL-066 - Keep drawings when a package is deleted
|
||||
|
||||
- **From:** Q13
|
||||
- **What:** deleting a package no longer deletes its drawings. Keep them for
|
||||
data retention. **The storage quota is freed** when the package is deleted.
|
||||
Storage will move to cloud storage in a future deployment. This replaces the
|
||||
earlier default (delete with the package).
|
||||
- **Today:** drawing bytes live in the Postgres `wp_files` table (CR-007, D8).
|
||||
The 2 GB per project limit counts them. Kept files must not count toward that
|
||||
limit, so the count has to change.
|
||||
- **To confirm:** retention period, who can delete for good, who can recover.
|
||||
See the decision file, "Still open" item 3.
|
||||
- **Suggested wave or follow-up:** roadmap Phase 3 (object storage). It affects
|
||||
D19 (safe download) paths too: recovered files must keep the attachment and
|
||||
sandbox headers.
|
||||
|
||||
### BL-067 - Ready for QA progress weight is 0.85
|
||||
|
||||
- **From:** Q12
|
||||
- **What:** change the progress weight for Ready for QA from 0 to **0.85**.
|
||||
Today a package falls from 75% to 0% when it reaches Ready for QA.
|
||||
- **Why not now:** it changes a shipped number on dashboards and roll-ups. Use
|
||||
its own item id and re-run the roll-up checks.
|
||||
- **Suggested wave or follow-up:** future wave. Small task.
|
||||
|
||||
### BL-068 - One cost-code list, kept in the SOP
|
||||
|
||||
- **From:** Q14
|
||||
- **What:** the creator has 38 cost codes, the SOP has 39, and 13 differ. Keep
|
||||
**one list, in the SOP**, and make the creator read it.
|
||||
- **Blocked on:** the correct codes. The **PM** supplies them (decided
|
||||
2026-10-12). The date is not set. This is a data question and not a code
|
||||
question.
|
||||
- **Related:** BL-019 (closed) made a stored code that has left the list stay
|
||||
as an option. Keep that behavior when the list changes.
|
||||
- **Suggested wave or follow-up:** future wave, after the codes arrive.
|
||||
>>>>>>> theirs
|
||||
|
||||
97
docs/waves/context-handoff-2026-10-12.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Context handoff: Work Package Suite
|
||||
|
||||
> Update 2026-10-16: waves 11 and 13 (T13.2 to T13.5) are on the release branch `release/waves-11-13-okta-sync`, with T13.1 held back. The branch and wave status tables below describe 2026-10-12. See `docs/waves/wave-13.md` for the current state.
|
||||
|
||||
Date: 2026-10-12
|
||||
Purpose: Give a new chat the facts about waves 11, 12, 13, and 16.
|
||||
This file is a handoff note. It is not a spec. The spec files stay the source of truth (`IMPLEMENTATION.md`, `docs/waves/*`, `CLAUDE.md`).
|
||||
It was checked against `git` on 2026-10-12. Earlier chat summaries about this repo are out of date. Use this file.
|
||||
|
||||
## 1. Branch state
|
||||
|
||||
| Branch | State |
|
||||
|---|---|
|
||||
| `main` | At `d42ae0c` (2026-10-06). Contains wave 12 (CR-020, bulk user editing) and wave 16 (D19, D20, D21), both merged by PR. |
|
||||
| `feat/waves-11-13` | Local and `origin` are the same commit, `da1325c`. It is pushed. It is not merged. It is 13 commits ahead of `main` and 19 behind. |
|
||||
| `notes/cr019-usage-metrics-d18-deprovisioning` | Holds the same wave 11 work plus the session notes. |
|
||||
|
||||
Correction to an earlier statement: the Gitea push is no longer blocked. The branch is on `origin`.
|
||||
|
||||
Merge rule from `wave-12.md`: T13.1 (idle timeout) must not reach `main` before T14.1. Do not merge `feat/waves-11-13` into `main` as a whole without a decision from the user. Wave 11 and T13.1 are on the same branch.
|
||||
|
||||
Working tree on `main` (2026-10-12) has two untracked files: `WorkPackageSuite_Consolidated_Change_Request_R2.docx` and `docs/waves/decisions-2026-08-25.md`. Do not delete them.
|
||||
|
||||
## 2. What is on `feat/waves-11-13`
|
||||
|
||||
| Commit | Task | Item |
|
||||
|---|---|---|
|
||||
| `75ac930` | Planning docs for CR-019, CR-020, D18 | CR-019, CR-020, D18 |
|
||||
| `0652fa7` | T11.1 UsageEvent model and migration | CR-019 |
|
||||
| `850b789` | T11.2 capture usage events | CR-019 |
|
||||
| `3584695`, `6cde6e3` | T13.1 idle timeout with an absolute ceiling | D18 |
|
||||
| `8e863ae` | T11.3 aggregation endpoint with filters | CR-019 |
|
||||
| `2de76d5` | T11.4 usage export (raw and sanitized CSV) | CR-019 |
|
||||
| `a6c3fbf`, `0b5ab59` | T11.5 admin console "Activity & usage" card, checked at 390px and 1440px | CR-019 |
|
||||
| `b2a083a` | T11.6 retire the per-browser Usage report | CR-019 |
|
||||
| `2842ec9` | Session notes for 2026-09-23 | none |
|
||||
| `4f3a94b` | T11.6 follow-up: fix `tests/autosave_check.py` | CR-019 |
|
||||
| `da1325c` | T11.7 full verification pass. Wave 11 closed on 2026-09-24. | CR-019 |
|
||||
|
||||
Notes:
|
||||
- `html/wp-usage.js` and `tests/usage_check.py` were deleted in T11.6. The `track()` functions in `wp-creation-app.js` and `work-package-suite-app.js` are now documented no-ops. About 45 call sites stay.
|
||||
- T11.7 found a real regression that T11.6 caused. `tests/autosave_check.py` still expected the retired tracker to write to `localStorage`. The test was fixed. It now passes 33 of 33. The cause was that T11.6 ran only `tests/baseline_shots.py`, not the full suite. Run the full suite for any frontend task.
|
||||
|
||||
## 3. T11.7 verification results (2026-09-24)
|
||||
|
||||
- Alembic: upgrade, downgrade, upgrade on SQLite passed. The SQL render for Postgres was clean (this catches SQLite-only DDL, the "BL-027 class" of defect).
|
||||
- Test scripts: 38 of 39 passed after the autosave fix.
|
||||
- Two known failures that are not regressions:
|
||||
- `token_check.py` exits 2. This is a CLI usage detail.
|
||||
- `generalinfo_check.py` finds an old color-literal gap in `wp-creation-styles.css`.
|
||||
|
||||
## 4. Wave status
|
||||
|
||||
| Wave | Items | Status |
|
||||
|---|---|---|
|
||||
| 10 | Okta OIDC sign-in (D15, D16) | Done. In production. |
|
||||
| 11 | CR-019 usage and activity metrics | Done on `feat/waves-11-13`. Not on `main`. |
|
||||
| 12 | CR-020 bulk user editing | Done. Merged to `main` on 2026-10-06 (T12.1 to T12.6). |
|
||||
| 13 | D18 Okta/AD deprovisioning sync | T13.1 done on `feat/waves-11-13`. T13.2 and later are not started. T13.3 must fail closed. Do not relax that. |
|
||||
| 14, 15 | Not checked in this handoff | T14.1 must land before T13.1 reaches `main`. Read the wave files before you plan. |
|
||||
| 16 | D19 safe downloads, D20 one SOP per project, D21 session decode fails closed | Done. Merged to `main` on 2026-10-06. |
|
||||
|
||||
Wave 16 decisions (`docs/waves/decisions-2026-10-02.md`): SEC-06 (session revocation) waits for the AWS work. Other SEC findings stay as backlog drafts in `docs/reference/security-review/backlog-drafts.md`.
|
||||
|
||||
## 5. Rules that apply here (`CLAUDE.md`)
|
||||
|
||||
- Every change traces to an item ID (`CR-`, `F`, `S`, `A`, `B`, `C`, `D`, task `T<wave>.<n>`). New scope gets a new ID.
|
||||
- One task per PR. Put the task ID and item IDs in the commit message and PR title.
|
||||
- Do not fix other problems that you see. Log them in `docs/waves/backlog.md`.
|
||||
- Do not reorder waves. Do not start a wave before the previous wave is merged, unless the task says it is independent.
|
||||
- Server work is needed for CR-004, CR-018, CR-007, CR-011, CR-014, CR-013, and B4. Do not use `localStorage` for these.
|
||||
- Do not change: Actual Hours in Closeout (CR-017), localization (A7), uppercase card headers in `console.css` (A5), the logged-override path `confirmEarlyRelease()` (A1), or hidden-not-deleted fields (CR-002, CR-016).
|
||||
- Token rule: one place defines color, spacing, and type. Do not add raw hex values to page stylesheets.
|
||||
- Accessibility (C1): use real `<button>`, `<a>`, or input elements. Add `aria-live` for status changes. Keep focus visible. Text contrast is 4.5:1 (3:1 for large text).
|
||||
- Frontend checks: run `uvicorn server.app:app` on a throwaway SQLite database. Check 390px and 1440px. Capture before and after screenshots. Run the smoke test.
|
||||
- If a done-when check cannot be run, do not mark the task complete. Say which check failed and why.
|
||||
|
||||
## 6. Environment on the user's PC
|
||||
|
||||
- Python was installed with `winget`. Tests use the repo `.venv`.
|
||||
- Browser tests (`tests/cdp.py` and about 40 `*_check.py` scripts) need Edge or Chrome. `cdp.py` starts from an elevated shell (wave 16 note). The Linux sandbox cannot run them.
|
||||
- See `C:\Users\m.mabrey\GIT\Field Value Driver\sde-meeting-toolkit\docs\context\environment-notes.md` for tool problems (PowerShell `$` removal, long git paths, the 180 second command limit).
|
||||
|
||||
## 7. Open items
|
||||
|
||||
1. Decide how `feat/waves-11-13` reaches `main`. The T13.1 and T14.1 order is the constraint.
|
||||
2. Plan the next wave. T13.2 and later need an explicit instruction.
|
||||
3. Future build list added 2026-10-12: `BL-062` to `BL-068` in `docs/waves/backlog.md` (mobile app, offline Field View, batch print, and related items). Answers are in `docs/waves/decisions-2026-10-12.md` (follow-ups from Matt included). The path to a mobile version outside the VPN is in `docs/waves/roadmap-mobile-scaling.md` (phases 0 to 5; confirmed: all users in the Okta group, multiple customers, company-managed devices; nine open questions in its section 7). These files are uncommitted edits on `main`.
|
||||
4. Work from the other chat (the SDE Meeting Toolkit) does not touch this repo. See the toolkit `docs/context/` folder.
|
||||
|
||||
## 8. Related files
|
||||
|
||||
- `IMPLEMENTATION.md` (sections 2 and 4)
|
||||
- `docs/waves/wave-11.md`, `wave-12.md`, `wave-13.md` (on `feat/waves-11-13`), `wave-16.md`
|
||||
- `docs/waves/decisions-2026-09-17.md` (CR-019, CR-020, D18), `decisions-2026-10-02.md` (D19, D20, D21)
|
||||
- `docs/waves/notes-2026-09-23.md` (on `feat/waves-11-13`)
|
||||
- `docs/waves/backlog.md`
|
||||
51
docs/waves/decisions-2026-10-12.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Decisions: October 12, 2026 (meeting with Nick)
|
||||
|
||||
Topic: mobile app (offline Field View), batch print, and related scaling questions.
|
||||
Recorded by Matt Mabrey from the meeting with Nick, 2026-10-12. Follow-up answers from Matt are in the section "Follow-up answers".
|
||||
|
||||
This file records the answers. It does not schedule any work. The features that come from these answers are logged in `docs/waves/backlog.md` as `BL-062` to `BL-068` for a future build. The path to a mobile version that works outside the VPN is in `docs/waves/roadmap-mobile-scaling.md`. None of these have a wave or a `CR-` or `D-` id yet. Assign the ids when a wave is planned.
|
||||
|
||||
Question numbers (`Q2` to `Q14`) come from the decision table that Matt received. `Q1`, `Q10`, and `Q11` were not in the table. They are not recorded here.
|
||||
|
||||
## Answers
|
||||
|
||||
| Q | Decision needed | Default that was proposed | Answer |
|
||||
|---|---|---|---|
|
||||
| Q2 | Days the app may open offline after the last sign-in (today: 12 hours) | 7 days. Wipe a disabled account's data on the next online open. | **7 days.** The default wipe rule stands. |
|
||||
| Q3 | Must planners edit in the creator offline, or only crews in Field View? | Field View only | **Only crews in the field.** The creator needs no offline mode. |
|
||||
| Q4 | An offline edit meets a newer server version. What does the crew see? | Keep both, show differences, crew chooses. Server wins if nobody acts. | **Merge with a notification. Log it. Update the owner.** If the same field changed on both sides, **the field value wins**, and the app keeps a history of changes. This replaces the default. |
|
||||
| Q5 | Photo limit per photo and per package | Resize to 1280 px, 50 per package | **5 MB per photo.** |
|
||||
| Q6 | Maximum packages in one batch print | 50, with a message to narrow the filter | **25 on phones.** On desktop the user can scale the filter up to 50 or 100. |
|
||||
| Q7 | Where drawings go in the PDF, and at what size | After the form, upload order, scaled to fit | **After the form, upload order, scaled to fit.** Same as the default. |
|
||||
| Q8 | SVG drawings in the merged PDF | Leave out, with a page that names the file | **Leave out, with a page that names the file.** Same as the default. |
|
||||
| Q9 | What goes on every page | Package no., print date, page X of Y | **Package no., print date, page X of Y.** Same as the default. |
|
||||
| Q12 | Progress weight for Ready for QA (today 0, so a package drops from 75% to 0%) | 0.85 | **0.85.** Same as the default. |
|
||||
| Q13 | What happens to a package's drawings when the package is deleted | Delete with the package | **Keep them (data retention). The storage quota is freed.** Storage will move to cloud storage in a future deployment. This replaces the default. |
|
||||
| Q14 | Which cost-code list is correct (creator has 38, SOP has 39, 13 differ) | One list in the SOP. Needs codes from Accounting or the PM. | **One list in the SOP.** The **PM** supplies the codes. |
|
||||
|
||||
## Follow-up answers (Matt, 2026-10-12)
|
||||
|
||||
These answer the six questions that were open after the first record.
|
||||
|
||||
1. **Q5:** the 5 MB limit is **per photo**. This matches the limit that already exists for drawings (D8, enforced in `server/app.py`: 5 MB a file, 2 GB per project).
|
||||
2. **Q6:** 25 on phones. Desktop users can scale up (50 or 100).
|
||||
3. **Q4:** on a conflict in the same field, **the field value wins**. The app keeps a history of changes.
|
||||
4. **Q13:** the quota is freed. Cloud storage will hold the kept files in a future deployment.
|
||||
5. **Q14:** the **PM** gives the codes.
|
||||
6. **Scaling:** Matt asked for a path to a mobile version that scales to multiple projects, outside VPN access, for field agents on phones and tablets on site. That path is in `docs/waves/roadmap-mobile-scaling.md`.
|
||||
7. **Who uses it:** all end users will eventually be in the Okta group.
|
||||
8. **Scale:** expect multiple **customers**. The data model needs a customer (organization) level above a project. See Phase 4 of the roadmap.
|
||||
9. **Devices:** company-managed.
|
||||
|
||||
## Still open
|
||||
|
||||
1. **Q5:** the count of photos per package, and whether the app resizes a photo before upload. Phone photos are often 3 MB to 8 MB. A 5 MB cap per photo with no resize will reject some normal photos.
|
||||
2. **Q4:** who "the owner" is (package owner, project owner, or the person who made the server change). Also: how long the history of changes is kept, and who can read it.
|
||||
3. **Q13:** how long a kept drawing is retained, who can delete it for good, and who can recover it. The quota is freed, so the retained bytes must not count toward the 2 GB project limit. Decide where they are counted instead.
|
||||
4. **Q14:** the date the PM gives the codes.
|
||||
5. **Q6:** who sets the larger desktop limit (each user, or an admin setting).
|
||||
6. The questions in `roadmap-mobile-scaling.md`, section 7 (nine open). The first two (one person on more than one customer, and the isolation model) change the data design. Answer them first.
|
||||
|
||||
## Source
|
||||
|
||||
Meeting with Nick. Status column in the original table: "Open. Team builds to the default" for every row, as of October 12. The answers above came after that status.
|
||||
85
docs/waves/notes-2026-09-23.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# Session notes — 2026-09-23
|
||||
|
||||
Working notes for the `feat/waves-11-13` line (CR-019, CR-020 reserved, D18),
|
||||
written up before merge. Not a spec document — `wave-11.md`, `wave-13.md` and
|
||||
`decisions-2026-09-17.md` are the source of truth for scope and acceptance
|
||||
criteria. This is the "what actually happened building it" record.
|
||||
|
||||
## What shipped today
|
||||
|
||||
**CR-019 (wave 11) — usage/activity metrics, T11.1 through T11.6 complete:**
|
||||
|
||||
- `UsageEvent` table + migration (T11.1), a capture endpoint wired into
|
||||
`auth-guard.js` so every protected page pings it once per load, plus a
|
||||
`login` event at Okta sign-in (T11.2).
|
||||
- `GET /api/usage/summary` (T11.3) — active users by day/week/month,
|
||||
per-user last-active, per-tool breakdown, filterable by date/project/
|
||||
user/tool, all combinable.
|
||||
- `GET /api/usage/export` (T11.4) — raw and sanitized CSV. Sanitized mode
|
||||
replaces the username with an HMAC-SHA256 pseudonym (keyed with
|
||||
`auth.SECRET_KEY`), stable per user across rows and across separate
|
||||
export calls, so an external tool (Power BI etc.) can still group by
|
||||
user without ever seeing a real name.
|
||||
- A new "Activity & usage" card in the admin console (T11.5): real filter
|
||||
controls, the summary tables, both export buttons. Client-side gated
|
||||
admin-only same as the rest of the console; the API underneath is
|
||||
independently gated server-side regardless.
|
||||
- Retired the old per-browser "Usage logs" panel and `wp-usage.js`
|
||||
entirely (T11.6) — it had no reader left and was never a real data
|
||||
source for the new report anyway. The scattered `track()` call sites in
|
||||
the creator and wizard were left in place calling a documented no-op,
|
||||
rather than deleting ~45 individual call sites for the same effect.
|
||||
|
||||
Only **T11.7 (final wave verification)** is left before wave 11 is fully
|
||||
closed out — everything under it has already been verified per-task, so
|
||||
this is a consolidation pass, not new work.
|
||||
|
||||
**D18 (wave 13) — Okta/AD deprovisioning, T13.1 only:**
|
||||
|
||||
- Session lifetime changed from one flat `AUTH_SESSION_HOURS` to a sliding
|
||||
idle timeout (`AUTH_IDLE_MINUTES`, default 30) capped by a hard ceiling
|
||||
from original sign-in (`AUTH_SESSION_HOURS`, default 8, meaning changed
|
||||
from "session length" to "absolute ceiling"). Both defaults are flagged
|
||||
in `.env.example` and `DEPLOYMENT.md` as proposed, not confirmed against
|
||||
the tenant's actual Okta SSO policy.
|
||||
- **T13.2 onward (the actual Okta Management API sync job) is paused** —
|
||||
explicit call from Matt: no Okta API credential yet, come back to it
|
||||
later. Not started, not blocked on anything code-side.
|
||||
|
||||
**CR-020 (wave 12, bulk user editing):** not started. Reserved, scoped in
|
||||
`wave-12.md`, no code touched.
|
||||
|
||||
## Environment work (not itself a task, but load-bearing)
|
||||
|
||||
- Fixed a CRLF/LF mismatch that was making every tracked file look modified
|
||||
to this session's git client (`core.autocrlf true`, repo-local, no file
|
||||
content changed).
|
||||
- This machine had no Python. Installed it via `winget` (`Python.Python.3.12`)
|
||||
and set up a `.venv` in the repo with `server/requirements.txt` installed,
|
||||
specifically so `tests/baseline_shots.py` could run locally — this
|
||||
sandbox has no headless-capable browser and can't download one (network
|
||||
allowlist), so the 390px/1440px screenshot verification CLAUDE.md asks
|
||||
for had to run on Matt's own machine instead, using the browser already
|
||||
installed there (Edge).
|
||||
- Established a repeatable local verification loop for every task: throwaway
|
||||
SQLite, fake-Okta sign-in, promote to admin, `seed_demo.py` +
|
||||
`smoketest.py` (27/27 passing throughout), plus `baseline_shots.py` for
|
||||
anything touching `html/`.
|
||||
|
||||
## Standing constraints, still in effect
|
||||
|
||||
- Everything stays local on this branch line. No `git push` at any point
|
||||
today.
|
||||
- One task per PR discipline was kept even though all of it landed on one
|
||||
branch — each commit corresponds to exactly one task ID, in wave order,
|
||||
each individually verified before the next started.
|
||||
|
||||
## Before merging
|
||||
|
||||
- Run T11.7 (full wave-11 verification pass) and record it in `wave-11.md`.
|
||||
- Decide where this branch actually merges to — `feat/waves-11-13` has all
|
||||
of today's commits already; this notes branch was cut from it so it can
|
||||
fast-forward back in, or merge as its own PR if the notes should be
|
||||
reviewed separately from the code.
|
||||
- T13.2+ and all of wave 12 remain explicitly out of scope until Matt says
|
||||
otherwise.
|
||||
178
docs/waves/roadmap-mobile-scaling.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# Roadmap: mobile version, multiple projects, access outside the VPN
|
||||
|
||||
Date: 2026-10-12
|
||||
Status: plan for a future build. It is not a spec. No wave, `CR-`, or `D-` id exists for it yet.
|
||||
Inputs: `docs/waves/decisions-2026-10-12.md` (Q2 to Q14), `BL-062` to `BL-068` in `docs/waves/backlog.md`, and the code as of `main` (`d42ae0c`).
|
||||
|
||||
Goal (from Matt, 2026-10-12): field agents use the app on phones and tablets on site, from outside the VPN, across more than one project.
|
||||
|
||||
Scope facts confirmed by Matt on 2026-10-12:
|
||||
- All end users will be in the Okta group. There are no contractors without an Okta account.
|
||||
- The product will scale to **multiple customers**, not only more projects for one customer.
|
||||
- The devices will be **company-managed**.
|
||||
|
||||
## 1. Where the product is today
|
||||
|
||||
| Area | Fact today | Source |
|
||||
|---|---|---|
|
||||
| Mobile shell | A PWA already exists: `manifest.webmanifest` (standalone, "Field View" shortcut) and `sw.js` (caches the app shell and uploaded drawings). | `html/manifest.webmanifest`, `html/sw.js` |
|
||||
| Offline writes | Writes queue in a sync outbox in `project-data.js`. Reads fall back to a `localStorage` cache. `/api/*` is never cached by the service worker. | `html/sw.js` header |
|
||||
| Offline drawings | Drawings for the user's assigned packages are prefetched and cached (CR-007, D8). | `html/sw.js`, `html/field.js` |
|
||||
| Files | The bytes live in the Postgres `wp_files` table. Limit: 5 MB a file, 2 GB per project. | `server/models.py` |
|
||||
| Sign-in | Okta OIDC. A signed cookie (`HttpOnly`, `SameSite=Lax`, `Secure` over HTTPS). Idle timeout (D18, T13.1) is on `feat/waves-11-13`, not on `main`. | `server/auth.py`, `docs/waves/wave-13.md` |
|
||||
| Projects | `Project` and `ProjectMember` exist. Access is by project role. There is no organization or customer level above the project. | `server/models.py` |
|
||||
| Hosting | Docker Compose on an internal host. NGINX serves the site. An external proxy ends TLS. The database and backup containers have no internet route. | `docker-compose.yml`, `nginx/conf.d/wp-suite.conf` |
|
||||
| Asset picker | Reads the Micron SQL Server catalog through `MICRON_DB_URL`. That database sits on the company network. | `server/assets_db.py` |
|
||||
| Abuse limits | No rate limiting or login throttling was found in `server/`. CSRF origin checks and security headers (CSP, HSTS) exist. | `server/app.py`, `wp-suite.conf` |
|
||||
| Security review | The 2026-10-01 review put session revocation (SEC-06) on hold until the AWS work. Wave 16 fixed SEC-01, SEC-02, SEC-07. | `docs/waves/decisions-2026-10-02.md` |
|
||||
|
||||
Result: the "mobile app" is mostly built as a PWA. The main work is not a new app. It is (a) finishing offline Field View, (b) making the service safe to put on the internet, and (c) removing the single-host limits.
|
||||
|
||||
## 2. Path in phases
|
||||
|
||||
Each phase has a gate. Do not start a phase before the gate of the one before it is met, unless this file says the phases are independent. Sizes are rough estimates (S under 1 week, M 1 to 3 weeks, L over 3 weeks). They are not commitments.
|
||||
|
||||
### Phase 0: Close the open decisions (S)
|
||||
|
||||
- Answer the questions in section 7 and the "Still open" list in `decisions-2026-10-12.md`.
|
||||
- Assign `CR-`/`D-` ids and write the wave files.
|
||||
- **Gate:** every item in `BL-062` to `BL-068` is free of "to confirm".
|
||||
|
||||
### Phase 1: Finish offline Field View, still inside the VPN (L)
|
||||
|
||||
Scope: `BL-062`, `BL-063`, `BL-064`.
|
||||
|
||||
1. Land the outbox work (wave 14, T14.1 to T14.3). `wave-12.md` says T13.1 must not reach `main` before T14.1.
|
||||
2. Set the offline window to 7 days (Q2). Reconcile it with the D18 idle timeout and absolute ceiling. Define what "signed in" means offline. Decide what the 7 day window does when Okta says the account is disabled (wipe on the next online open).
|
||||
3. Conflict handling (Q4): merge, field value wins on the same field, history of changes, a notification to the crew, a log entry, and an update to the owner.
|
||||
4. Photos: capture on the device, 5 MB per photo (Q5). Add resize before upload if Matt chooses it. Store photos like drawings (server-side, with D19 download headers).
|
||||
5. Add a phone and tablet test matrix (real devices): iOS Safari installed to the home screen, Android Chrome, one tablet.
|
||||
6. Test the weak-link case: no signal, then a slow signal, then a sync with a conflict.
|
||||
|
||||
Known platform risks to test on the devices (verify; do not assume):
|
||||
- iOS may delete site storage after a period of no use unless the PWA is installed to the home screen.
|
||||
- iOS Safari has no Background Sync. The outbox then sends only while the app is open. A crew member must open the app to push changes.
|
||||
- A lost device holds up to 7 days of project data. Web storage is not encrypted by the app. See Phase 2.
|
||||
|
||||
**Gate:** the crew completes a full shift offline on a test device, reconnects, and the server holds every change. The conflict case works as Q4 says. `tests/mobile_check.py` and the 390px/1440px checks pass.
|
||||
|
||||
### Phase 2: Make the service safe to reach from the internet (M to L)
|
||||
|
||||
Today the design assumes the firewall. Outside the VPN, the app becomes the perimeter. Do this before any real user connects from the internet.
|
||||
|
||||
| Control | Why |
|
||||
|---|---|
|
||||
| Rate limiting and login throttling on the API (none today) | Stops guessing and scraping. The toolkit (`sde-meeting-toolkit`) needed the same fix. |
|
||||
| A WAF or reverse proxy with request limits in front of NGINX | Filters bad requests before they reach the API. |
|
||||
| Real client IP handling (`X-Forwarded-For` from the trusted proxy only) | Limits and logs must use the real address, not the proxy. |
|
||||
| Okta policy for the app: MFA, session length, device trust | Okta is the only sign-in path (D15, D16). The policy is the control. Ask the Okta admin. |
|
||||
| Session revocation (SEC-06) | When a phone is lost, an admin must end that session now. This was put on hold for the AWS work. Bring it forward. |
|
||||
| A rule for lost devices | Devices are company-managed (decided). Require a device PIN and disk encryption through the device policy. Remote wipe through the device manager. Pair it with session revocation (above), because a wipe cannot reach an offline device. Confirm with IT that the policy covers every phone and tablet that runs the app, and that a lost device is reported quickly. |
|
||||
| Okta access rule | All end users are in the Okta group (decided). Gate the app on that group. Today the API creates the account on first sign-in; check that a person who is not in the group is refused, and that removal from the group ends access (this is what D18 and wave 13 do). Ask the Okta admin to add MFA and device-trust conditions for the outside path. |
|
||||
| Cookie flags on every cookie (the OIDC state cookie uses `https_only=False` in `server/app.py`) | Must be `Secure` on the public site. |
|
||||
| Review the 2026-10-01 findings that stayed in backlog drafts | Re-rank them for an internet-facing service. |
|
||||
| Outside review (a penetration test) | Required before the customer-IP data is reachable from the internet. |
|
||||
| Alerting on failed sign-ins and on odd upload sizes | Detect abuse early. |
|
||||
|
||||
**Gate:** the controls above are in place and an outside test finds no high findings open.
|
||||
|
||||
### Phase 3: Hosting that can leave the office network (L)
|
||||
|
||||
The AWS work is already planned (SEC-06 note). Proposed shape. The choice of services is open for Matt and the AWS owner.
|
||||
|
||||
- A public edge (load balancer, WAF, TLS) in front of the API and the static site.
|
||||
- API containers on a managed container service, with a managed Postgres database.
|
||||
- Files in object storage instead of the database (`BL-066`, Q13). Move the bytes out of `wp_files`. Keep the metadata in Postgres. Kept files from deleted packages go to a retention area with a lifecycle rule. The quota counts only live files.
|
||||
- Backups and restore tests on the new stack (the current backup container writes to a host folder).
|
||||
- A private link back to the Micron asset database, or a different design for the asset picker (the catalog is on the company network). Until this exists, the picker falls back to manual entry outside the VPN. Check that fallback works for a user who is online but cannot reach the database (the code treats "unconfigured" as a normal state, but not "unreachable").
|
||||
- Infrastructure as code, so the environment can be rebuilt.
|
||||
- Keep the internal deployment working until the cutover.
|
||||
|
||||
**Gate:** a full restore test passes, the migration runs on Postgres (render the SQL for the Postgres dialect first, BL-027), and a rollback plan is written and rehearsed.
|
||||
|
||||
### Phase 4: Many projects and many customers (L)
|
||||
|
||||
Matt confirmed multiple customers. A customer boundary is now part of the plan, not an option. The data model has `Project` and `ProjectMember` but nothing above a project, so this phase changes the base design. Plan for it early. Phase 3 choices (storage layout, database layout) must already allow it.
|
||||
|
||||
- Add an organization (customer) level above `Project`. Every project, user role, file, report, export, and search belongs to one organization. A user may belong to more than one organization (for example Prime Controls staff who work for several customers). Decide this rule first (section 7, question 1).
|
||||
- Isolation: choose the model before any code. Options: one database with an organization id on every row and checks on every query; a separate schema per customer; or a separate database or stack per customer. Matt and the AWS owner decide. The choice sets cost, backup, and the strength of the separation that a customer can be shown.
|
||||
- Files: one storage prefix per customer, with its own access rule and, if a customer asks, its own encryption key.
|
||||
- Customer-specific data: the cost-code list, the asset catalog (today Micron only), and the SOP templates must belong to the organization or the project. `BL-068` (one cost-code list in the SOP) must not become one list for all customers.
|
||||
- Okta: map Okta groups to organizations and projects, so the Okta admin controls who sees which customer.
|
||||
- Reports and the admin console: super users see only the organizations they are given. Check CR-019 usage reports, the user directory, and the bulk user tools from wave 12 against this rule.
|
||||
- Onboarding a customer: a runbook with the steps and a checklist.
|
||||
- Project onboarding: create a project, load its SOP, cost codes (`BL-068`, one list in the SOP, codes from the PM), people, and roles. Use the bulk user tools from wave 12.
|
||||
- Okta group to project membership mapping, so the Okta admin controls who joins a project.
|
||||
- Per-project storage and photo quotas (2 GB per project exists today) and per-project usage reports (CR-019 on `feat/waves-11-13`).
|
||||
- Performance work with a load test: many crews syncing at shift start. Check the outbox flush, the `wp_files` move, and the list endpoints on a large project.
|
||||
- Data retention rules per project.
|
||||
|
||||
**Gate:** two real customers run side by side. A test user in one customer cannot reach any data of the other through any screen, report, export, search, file link, or API call. A test that tries to cross the boundary on purpose passes. The load test meets targets that Matt sets.
|
||||
|
||||
### Phase 5: Native wrapper (only if needed) (M)
|
||||
|
||||
Wrap the PWA in a native shell (for example Capacitor) only if Phase 1 testing shows a platform limit that the PWA cannot meet: storage eviction, no background sync, camera control, or a company rule for app-store distribution. A wrapper adds app-store review, signing, and update cost. Do not start it by default.
|
||||
|
||||
## 3. Order and dependencies
|
||||
|
||||
```
|
||||
Phase 0 -> Phase 1 -> Phase 2 -> Phase 3 -> Phase 4
|
||||
\_ Phase 5 (decide after Phase 1 device tests)
|
||||
```
|
||||
|
||||
- Phase 2 may start in part while Phase 1 runs (rate limiting, Okta policy review). No crew uses the internet path before Phase 2 is done.
|
||||
- Phase 3 (storage move) and `BL-066` (retained drawings) are the same work. Do them together.
|
||||
- Phase 4 needs the Phase 2 and Phase 3 gates. Opening a second project to the internet first would repeat any gap twice.
|
||||
|
||||
## 4. Work that this path adds to the backlog
|
||||
|
||||
| Id | Item | Phase |
|
||||
|---|---|---|
|
||||
| BL-062 to BL-068 | Already logged | 1, 3, 4 |
|
||||
| New | API rate limiting and login throttling | 2 |
|
||||
| New | Real client IP handling behind the proxy | 2 |
|
||||
| New | Session revocation (SEC-06) brought forward | 2 |
|
||||
| New | Device policy check with IT (PIN, encryption, remote wipe) | 1, 2 |
|
||||
| New | Okta group gate and MFA/device-trust conditions for the outside path | 2 |
|
||||
| New | Object storage and lifecycle rules for files, one prefix per customer | 3 |
|
||||
| New | Asset picker when the Micron database is unreachable | 3 |
|
||||
| New | Organization (customer) level above `Project`, with the isolation model chosen first | 3, 4 |
|
||||
| New | Per-customer cost codes, asset catalog, and SOP templates | 4 |
|
||||
| New | Okta group to organization and project mapping | 4 |
|
||||
| New | Cross-customer access tests (a test that tries to cross the boundary) | 4 |
|
||||
| New | Customer onboarding runbook | 4 |
|
||||
| New | Load test and capacity targets | 4 |
|
||||
|
||||
Log these in `docs/waves/backlog.md` when Matt agrees with the plan. This file does not add them, so the backlog stays free of unconfirmed scope.
|
||||
|
||||
## 5. Risks
|
||||
|
||||
1. **Customer data on lost devices.** Offline data for 7 days on a phone or tablet. Devices are company-managed, which lowers this risk a lot, but only if the device policy (PIN, disk encryption, remote wipe) is real and covers every device. An offline device cannot receive a wipe until it connects. Keep the 7 day window only if IT confirms the policy.
|
||||
2. **Customer boundary added late.** The data model has no organization level. If Phases 1 to 3 are built without it, every later change touches every table and every query. Choose the isolation model in Phase 0 and design Phase 3 storage around it.
|
||||
3. **Contract terms.** Customer drawings and asset data may have rules about where they are stored and who can reach them. With several customers, each contract may differ. Check before Phase 3 (cloud storage) and before Phase 2 (internet access).
|
||||
4. **Okta.** The app has no password of its own. A weak Okta policy for the outside path is a weak app. The Okta admin owns that setting.
|
||||
5. **Single database host.** The current database is one container with a host-folder backup. Phase 3 fixes this. Until then, do not add projects that need high availability.
|
||||
6. **Sequence risk.** T13.1 (idle timeout) must wait for T14.1 (outbox). A short idle timeout can end a session while a crew is offline. Settle the rule in Phase 1.
|
||||
|
||||
## 6. What does not change
|
||||
|
||||
These decisions stay as recorded: Actual Hours stays (CR-017), localization stays (A7), uppercase card headers stay (A5), the logged-override path for predecessors stays (A1), removed fields are hidden and not deleted (CR-002, CR-016), and the token rule for CSS. Mobile work follows the accessibility rules (C1): real buttons, visible focus, `aria-live` for status, hover is not enough on tablets.
|
||||
|
||||
## 7. Questions that decide the plan
|
||||
|
||||
Answered on 2026-10-12:
|
||||
- Who are the outside users? **All end users will be in the Okta group.** No separate sign-in design is needed.
|
||||
- What does "multiple projects" mean? **Multiple customers.** See Phase 4.
|
||||
- Which devices? **Company-managed.**
|
||||
|
||||
Answer these before Phase 0 ends. The first four are new and follow from the answers above.
|
||||
|
||||
1. **Can one person work for more than one customer?** For example, a Prime Controls planner on two customers' projects. This sets how the organization level and the Okta groups map to each other.
|
||||
2. **Which isolation model?** One database with an organization id on every row, a schema per customer, or a database or stack per customer. Do any customers already require a separate stack in their contract?
|
||||
3. **Who manages the Okta groups?** One group for all users, or one group per customer or project? Does the Okta admin accept that mapping?
|
||||
4. **What does the device policy enforce today?** PIN, disk encryption, remote wipe, and a time limit for a device to check in. Ask IT.
|
||||
5. **Connectivity on site.** No signal for days, or weak signal that comes and goes? This sets how long the offline window must really be, and how big the photo queue gets.
|
||||
6. **AWS.** Is the AWS move approved? Who owns the account? Which region? Can the Micron database be reached from it?
|
||||
7. **Contract.** Can customer drawings and asset data be stored in cloud storage and reached from the internet? Does each customer have its own terms?
|
||||
8. **Distribution.** Is an installed PWA enough, or does the company need an app-store app? Company-managed devices can often install a web app from a device policy. Ask IT.
|
||||
9. **Targets.** How many customers, projects, and users, and how many crews syncing at the same time, in the first year?
|
||||
280
docs/waves/wave-11.md
Normal file
@@ -0,0 +1,280 @@
|
||||
# Wave 11 — Usage and activity metrics
|
||||
|
||||
**Items:** `CR-019`
|
||||
**Depends on:** wave 10 merged (it is; this wave does not wait on anything else)
|
||||
**Decision record:** `docs/waves/decisions-2026-09-17.md`
|
||||
|
||||
Seven tasks, one concern each, in build order. Do not start a task whose
|
||||
dependency is not merged. `CR-020` (bulk user editing) is reserved but not
|
||||
scoped — it does not belong in this wave.
|
||||
|
||||
---
|
||||
|
||||
### T11.1 — CR-019: `usage_events` table + migration
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** nothing (first task)
|
||||
- **Blocks:** T11.2
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/models.py`, `server/alembic/versions/`
|
||||
|
||||
**Do:** Add a `UsageEvent` model — append-only, same spirit as `AuditLog` but for
|
||||
navigation/feature-open events rather than business mutations. Suggested shape:
|
||||
`id`, `at` (indexed), `user_id` (or username — match whatever `AuditLog.actor`
|
||||
does today for consistency), `project_id` (nullable — not every event is
|
||||
project-scoped, e.g. opening the admin console), `tool` (e.g. `creator`,
|
||||
`wizard`, `field_view`, `dashboard`, `admin`, `directory`), `event` (e.g.
|
||||
`page_open`, `login`), `detail` (JSON, optional). Write the migration. Do not
|
||||
touch `AuditLog` — this is a new table, not an extension of it (see the
|
||||
decision record's reasoning).
|
||||
|
||||
**Do not:** fold this into `AuditLog`. They serve different questions and mixing
|
||||
them makes the existing audit trail noisier for its existing readers.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] `UsageEvent` exists with an indexed `at` column (this table will be scanned
|
||||
by date range constantly)
|
||||
- [ ] migration applies cleanly against both SQLite (dev) and Postgres (prod) —
|
||||
render `alembic upgrade --sql` for postgresql and read it before calling
|
||||
this done, per the class of defect `BL-027` logged
|
||||
- [ ] no change to `AuditLog`'s shape or behavior
|
||||
|
||||
---
|
||||
|
||||
### T11.2 — CR-019: capture the events
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.1
|
||||
- **Blocks:** T11.3
|
||||
- **Surface:** `server/` + `html/`
|
||||
- **Files:** `server/app.py` (new endpoint), `html/auth-guard.js`
|
||||
|
||||
**Problem:** Six pages need this and none should implement it separately — that
|
||||
is exactly how `S4`'s "no global nav on two pages" and the four parallel token
|
||||
systems (`S5`) happened. `auth-guard.js` is already loaded first, in the `<head>`,
|
||||
on all six protected pages (`index.html`, `field.html`, `users.html`,
|
||||
`wp-creation-index.html`, `work-package-suite.html`, `admin.html`) and already
|
||||
knows the verified user once the `wp-auth-ready` event fires. That is the one
|
||||
place this belongs.
|
||||
|
||||
**Do:** Add a small `POST /api/usage/ping`-style endpoint that writes one
|
||||
`UsageEvent` row per call, keyed to the session (server trusts the session, not
|
||||
anything the client claims about identity). Call it once from `auth-guard.js`
|
||||
after `wp-auth-ready`, tagging `tool` from the page's own path. Also write a
|
||||
`login` event at the point a session is actually established (reuse whatever
|
||||
`okta_callback` already does at sign-in — do not add a second source of truth
|
||||
for "did this person log in").
|
||||
|
||||
**Do not:** build a per-page capture call. If a page needs this and
|
||||
`auth-guard.js` does not cover it, fix `auth-guard.js`, not the page.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] one `page_open` event is recorded for a real sign-in on each of the six
|
||||
pages, verified per page
|
||||
- [ ] exactly one `login` event per Okta sign-in, not one per page load after
|
||||
it
|
||||
- [ ] the endpoint rejects a request with no valid session (this is server-
|
||||
enforced identity, not client-reported)
|
||||
- [ ] no page other than `auth-guard.js` calls this endpoint directly
|
||||
|
||||
---
|
||||
|
||||
### T11.3 — CR-019: aggregation endpoint with filters
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.2
|
||||
- **Blocks:** T11.4, T11.5
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/app.py`
|
||||
|
||||
**Do:** Build the read side: active-user counts by day/week/month, per-user
|
||||
last-active timestamp (derived from `UsageEvent`, not `User.last_login_at`,
|
||||
which only ever holds one value), and a per-tool usage breakdown. Accept query
|
||||
filters: date range, project, user, tool — combinable, per the decision record.
|
||||
This is server aggregation, the same principle `B4` established for the
|
||||
pipeline strip: the browser asks for a number, the server computes it from real
|
||||
rows, nothing is derived client-side from a partial cache.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] active-user counts are correct against a seeded fixture with known dates
|
||||
- [ ] filters combine correctly (verified: user + date range + tool together
|
||||
narrows correctly, not just each alone)
|
||||
- [ ] a project filter that matches nothing returns an empty result, not an
|
||||
error or the unfiltered total
|
||||
|
||||
---
|
||||
|
||||
### T11.4 — CR-019: export, raw and sanitized
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.3
|
||||
- **Blocks:** T11.5
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/app.py`
|
||||
|
||||
**Do:** A CSV export endpoint over the same filtered query T11.3 exposes.
|
||||
Two modes: raw (real usernames, the console's default) and sanitized. Sanitized
|
||||
mode replaces the actor field with a stable pseudonymous id — a per-user hash,
|
||||
consistent across rows in the same export and across separate export runs —
|
||||
so an external system (Power BI or similar) can still group and trend "by
|
||||
user" without ever receiving a real name. Do not simply drop the identity
|
||||
column; that breaks per-user grouping downstream, which defeats the point of
|
||||
an activity export.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] raw export contains real usernames
|
||||
- [ ] sanitized export never contains a real username or email anywhere in the
|
||||
file, including in a `detail` blob if one is included
|
||||
- [ ] the same real user maps to the same pseudonymous id within one export AND
|
||||
across two separate export runs (a hash of something stable, not a
|
||||
per-request random id)
|
||||
- [ ] both modes otherwise contain identical rows for the same filter
|
||||
|
||||
---
|
||||
|
||||
### T11.5 — CR-019: admin console Activity tab
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.3, T11.4
|
||||
- **Blocks:** T11.7
|
||||
- **Surface:** `html/`
|
||||
- **Files:** `html/admin.html`, `html/admin.js`
|
||||
|
||||
**Do:** New tab, same role gate as the User Directory. Filters (date range,
|
||||
project, user, tool) driving the tables from T11.3; export buttons (raw and
|
||||
sanitized) calling T11.4. Build accessible from the start per `C1` — this is a
|
||||
new component, not a legacy one carrying an old defect forward: real
|
||||
`<button>`/`<select>` controls, keyboard-reachable, `aria-live` on any
|
||||
count that updates without a page reload, focus visible throughout.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] the tab is reachable only by an admin (the card lives inside admin.html,
|
||||
already gated client-side by gateByRole(); the API underneath it is
|
||||
independently gated server-side by require_user_manager regardless)
|
||||
- [x] every filter is a real form control, keyboard-operable (date/select/text
|
||||
inputs and a `<button>`, no click-div)
|
||||
- [x] both export buttons produce the files T11.4 defines (verified against
|
||||
the live endpoint in T11.4's own checks, and present/wired here)
|
||||
- [x] works at 390px and 1440px — verified 2026-09-23 via
|
||||
`tests/baseline_shots.py --pages admin` run locally on Windows (this
|
||||
sandbox has no headless browser available; the script was run on the
|
||||
user's machine instead, after installing Python via winget since it
|
||||
wasn't present). Screenshots in `docs/reference/baseline/admin-390.png`
|
||||
/ `admin-1440.png`. No JS errors, no horizontal overflow at either
|
||||
width; the card rendered with real seeded data (events, by-tool,
|
||||
per-user-last-active tables) confirming the filters and summary read
|
||||
correctly, not just that the markup exists.
|
||||
|
||||
---
|
||||
|
||||
### T11.6 — CR-019: retire the per-browser Usage report
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.5
|
||||
- **Blocks:** T11.7
|
||||
- **Surface:** `html/`
|
||||
- **Files:** `html/admin.js` (the `usage-admin` panel, `admin.js:666-699`),
|
||||
`html/wp-usage.js` and its two call sites
|
||||
|
||||
**Do:** Remove the old per-browser `usage-admin` panel from `admin.js` now that
|
||||
the real one exists, per the 2026-09-17 decision. Decide what happens to
|
||||
`wp-usage.js`'s recording calls (wizard dwell-tracking, the creator's
|
||||
equivalent): they were never reliably tied to a real identity, so they are not
|
||||
a data source the new report can adopt. Default to removing the recorder too
|
||||
unless it is still doing something useful on its own (re-read what it actually
|
||||
records before deciding — do not assume from this file alone).
|
||||
|
||||
**Do not:** leave the old panel in place "just in case." Two activity reports
|
||||
showing two different numbers is worse than one.
|
||||
|
||||
**Decision (2026-09-23):** `wp-usage.js` is removed, not kept — it had no
|
||||
reader left once the admin panel above it was removed (the "download the
|
||||
full event log" button lived only in that panel), and per the original
|
||||
decision record it was never reliably tied to a real identity, so it was
|
||||
never a candidate source for the new report either. The file itself and its
|
||||
three `<script>` includes (`admin.html`, `wp-creation-index.html`,
|
||||
`work-package-suite.html`) are gone. Its two call sites
|
||||
(`wp-creation-app.js`, `work-package-suite-app.js`) keep a local `track()`
|
||||
function as a documented no-op rather than having each of their ~45
|
||||
individual `track('event', …)` call sites deleted one at a time — that
|
||||
would be a far larger, riskier diff for the same outcome (no more data is
|
||||
recorded either way), and it keeps each call site as a marker of what was
|
||||
worth recording if usage analytics are ever rebuilt server-side. The
|
||||
dwell-timer plumbing that only ever fed `track()` (`work-package-suite-app.js`'s
|
||||
`_stepEnter`/`trackStepDwell`) was left in place for the same reason — it is
|
||||
inert now, not broken, and touching it buys nothing.
|
||||
|
||||
Also removed: `tests/usage_check.py` (tested exactly the retired feature —
|
||||
D5/T7.10's per-browser analytics core and admin report) and its line in
|
||||
`docs/reference/file-map.md`, replaced with a note pointing at this decision.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] the old `usage-admin` panel and its markup are gone from `admin.html`/
|
||||
`admin.js`
|
||||
- [x] a decision on `wp-usage.js` itself is recorded (removed, or kept with a
|
||||
stated reason) — not left ambiguous — see above
|
||||
- [x] nothing else in the app references the removed code; grep confirms
|
||||
(only remaining hits are this file, the 2026-09-17 decision record, and
|
||||
the two explanatory code comments left at the retired call sites — all
|
||||
prose, not live references)
|
||||
|
||||
---
|
||||
|
||||
### T11.7 — CR-019: verification
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.6
|
||||
- **Blocks:** nothing
|
||||
- **Surface:** `html/` + `server/`
|
||||
- **Files:** as touched above
|
||||
|
||||
**Do:** Full verification per `CLAUDE.md`: run the app locally, exercise the
|
||||
new tab at 390px and 1440px, before/after screenshots, run the existing smoke
|
||||
test and `seed_demo.py`, run the full suite.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] all `CR-019` acceptance criteria in `decisions-2026-09-17.md` are met or
|
||||
a failure is stated with a reason — all met
|
||||
- [x] screenshots committed — `admin-390.png`/`admin-1440.png` (and
|
||||
`creator`/`sop` at both widths, touched by T11.6's script-tag removal)
|
||||
- [x] smoke test and `seed_demo.py` both still pass — 27/27, verified
|
||||
2026-09-24 against the exact branch state about to be considered
|
||||
deployable
|
||||
- [x] full test suite passes — verified 2026-09-24: ran all 39
|
||||
`tests/*_check.py` scripts plus the `usage_events` migration
|
||||
round-trip. Result: 38/39 exit 0.
|
||||
- `token_check.py` exits 2 with no arguments by design (needs `--out`
|
||||
or `--compare`) — a CLI-invocation detail, not a test failure;
|
||||
already confirmed non-regression in wave 10.
|
||||
- `generalinfo_check.py` has one pre-existing failure (a raw colour
|
||||
literal in `html/wp-creation-styles.css`, a file no wave 11/13 task
|
||||
touched) — already diagnosed as pre-existing in wave 10.
|
||||
- `autosave_check.py` **did** regress from T11.6 (it asserted the
|
||||
retired analytics tracker still wrote to localStorage) — found by
|
||||
this run, fixed same day (commit `4f3a94b`), now 33/33.
|
||||
- Migration (T11.1): SQLite upgrade → downgrade → upgrade round-trip
|
||||
clean; `alembic upgrade ... --sql` against the `postgresql` dialect
|
||||
renders valid DDL (`TIMESTAMP WITH TIME ZONE`, `JSON`, no
|
||||
SQLite-only syntax) — no `BL-027`-class defect.
|
||||
|
||||
---
|
||||
|
||||
## Wave 11 exit criteria
|
||||
|
||||
- [x] real, server-side activity data exists per user, indefinitely retained
|
||||
- [x] the admin console shows it, filterable by date/project/user/tool
|
||||
- [x] export works in both raw and sanitized form
|
||||
- [x] the old per-browser report is gone, not duplicated
|
||||
- [x] `CR-019` fully accounted for, no open acceptance criteria
|
||||
|
||||
Wave 11 is closed out as of 2026-09-24. `CR-020` (wave 12) remains reserved
|
||||
and unstarted. `D18`/wave 13 (`T13.2` onward) remains paused pending an Okta
|
||||
API credential, per standing instruction — `T13.1`'s session changes were
|
||||
included in this same verification pass and did not regress anything above.
|
||||
330
docs/waves/wave-13.md
Normal file
@@ -0,0 +1,330 @@
|
||||
# Wave 13 — Okta/AD deprovisioning sync
|
||||
|
||||
**Items:** `D18`
|
||||
**Depends on:** wave 10 merged (it is). Not blocked by wave 11 or wave 12, but
|
||||
shares `server/app.py` account-state surface with wave 12 (`CR-020`) — sequence
|
||||
commits to avoid an avoidable conflict.
|
||||
**Decision record:** `docs/waves/decisions-2026-09-17.md`
|
||||
|
||||
Six tasks, in build order. **T13.3's fail-closed behavior is the most
|
||||
important done-when list in this wave — do not relax it to ship faster.**
|
||||
|
||||
---
|
||||
|
||||
### T13.1 — D18: idle timeout with an absolute ceiling
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** nothing (first task, independent of the rest)
|
||||
- **Blocks:** nothing
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/auth.py`, `server/app.py` (`auth_gate` middleware),
|
||||
`server/.env.example`, `DEPLOYMENT.md`
|
||||
|
||||
**Revised 2026-09-23** — originally just "shrink `AUTH_SESSION_HOURS`". Matt
|
||||
asked whether an idle timeout would be a better fit than a flat session
|
||||
length. It is, but not by itself — see the decision record's reasoning on why
|
||||
idle-with-no-ceiling is actually a worse fit for this item's own threat model
|
||||
than a flat expiry would have been. Build both.
|
||||
|
||||
**Do:**
|
||||
|
||||
- Add `login_at` to the JWT payload in `create_token()` — the original
|
||||
sign-in time, distinct from `iat`, which becomes "when THIS token was
|
||||
issued" once tokens start getting reissued. `login_at` never changes across
|
||||
reissues; it's what the absolute ceiling is measured from.
|
||||
- Add `AUTH_IDLE_MINUTES` (default 30). `AUTH_SESSION_HOURS` stays the name
|
||||
for the absolute ceiling, default changing from 12 to a proposed 8 — update
|
||||
its docstring/comment in `auth.py` and `.env.example`, since its MEANING is
|
||||
changing (session length -> hard ceiling on top of a sliding idle window),
|
||||
not just its value.
|
||||
- In `auth_gate` (`server/app.py`), after confirming a request is
|
||||
authenticated: if `now < login_at + AUTH_SESSION_HOURS` (the ceiling hasn't
|
||||
passed), compute `new_exp = min(now + AUTH_IDLE_MINUTES, login_at +
|
||||
AUTH_SESSION_HOURS)`. If `new_exp` is meaningfully later than the current
|
||||
token's `exp` (throttle this — do not reissue on every single request, only
|
||||
when enough time has passed to be worth a new cookie; a few minutes of
|
||||
slack is fine), mint a refreshed token carrying forward `sub`/`username`/
|
||||
`role`/`ver`/`login_at` unchanged, and set it on the response.
|
||||
- If the ceiling HAS passed, do not refresh — let the existing token expire
|
||||
on its own terms (it may already be invalid, or may tick over within the
|
||||
idle window; either way, no new one is issued past the ceiling).
|
||||
- The middleware should not need a DB round trip to do this — everything
|
||||
needed (`sub`, `username`, `role`, `ver`, `login_at`) is already in the
|
||||
validated claims. `get_current_user`'s existing per-request `is_active`/
|
||||
`token_version` check is unaffected and still runs separately.
|
||||
|
||||
**Do not:** reissue the cookie on every request unconditionally — that's a
|
||||
`Set-Cookie` header on every API call for no benefit over a coarser refresh.
|
||||
Do not let the ceiling check silently vanish for tokens issued before this
|
||||
change ships — a token with no `login_at` claim should fall back to treating
|
||||
its own `iat` as `login_at`, not bypass the ceiling entirely.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] a session with continuous activity stays alive past 30 minutes but is
|
||||
cut off at the `AUTH_SESSION_HOURS` ceiling regardless
|
||||
*(2026-10-16, T13.6: a request every minute for 10 hours: alive well past 30 minutes, never extended past login + 8 h, cut off at the ceiling, original `login_at` kept.)*
|
||||
- [x] a session with no activity for 30+ minutes is rejected on its next
|
||||
request
|
||||
*(2026-10-16, T13.6: an idle token gets 401 on a gated route; one inside the window gets 200.)*
|
||||
- [x] the cookie is not rewritten on every single request — verify the
|
||||
refresh is throttled, not unconditional
|
||||
*(2026-10-16, T13.6: 20 rapid requests on a fresh session: 0 rewrites; 480 requests over 8 h: about one rewrite per 10 minutes; a refreshed cookie is not rewritten again on the next request. Mutation test: removing the throttle fails 3 checks.)*
|
||||
- [x] a pre-existing token with no `login_at` claim (simulating a session
|
||||
issued before this shipped) still gets a hard ceiling, via the `iat`
|
||||
fallback
|
||||
*(2026-10-16, T13.6: a token with iat 9 h ago and a still-valid old exp is not extended; one with iat 1 h ago is refreshed with `login_at` taken from `iat`; no `login_at` and no `iat` is not refreshed and does not crash. Mutation test: giving it a fresh ceiling fails.)*
|
||||
- [x] `.env.example` and `DEPLOYMENT.md` explain both variables and flag both
|
||||
defaults as proposed, not confirmed against Okta's own session policy
|
||||
*(2026-10-16, T13.6: both variables are in both files, and both call the defaults proposed.)*
|
||||
- [x] existing session/auth tests updated for the new mechanism, not just a
|
||||
new number
|
||||
*(2026-10-16, T13.6: there were none. T13.1 shipped with no test of its own. `tests/session_idle_check.py` is that test (T13.6 added it).)*
|
||||
|
||||
---
|
||||
|
||||
**Found while writing that test (T13.6, 2026-10-16):**
|
||||
- Nothing under `/api/auth/` slides the session. `auth._EXEMPT_PREFIXES` covers
|
||||
that whole prefix, so `auth_gate` never refreshes there (including
|
||||
`/api/auth/me`). A page that only called `/api/auth/me` would not extend its
|
||||
session. The spec says "after confirming a request is authenticated", and these
|
||||
routes are not confirmed by the gate, so this is as written. Noted, not changed.
|
||||
- The wave's exit criteria below say `AUTH_SESSION_HOURS` defaults to **2**. T13.1
|
||||
(this section), `auth.py`, `.env.example` and `DEPLOYMENT.md` all say **8**. The
|
||||
code and the T13.1 text agree, so the exit line looks like a leftover from
|
||||
before the 2026-09-23 revision. Not changed here: it is a product number. Needs
|
||||
Matt to say which one he means.
|
||||
|
||||
### T13.2 — D18: Okta Management API client
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** nothing (independent of T13.1)
|
||||
- **Blocks:** T13.3
|
||||
- **Surface:** `server/`
|
||||
- **Files:** new `server/okta_sync.py` (or extend `server/okta_auth.py` —
|
||||
task's call), `server/.env.example`
|
||||
|
||||
**Do:** A small client for Okta's user-list endpoint, authenticated with a new,
|
||||
separate credential (e.g. `OKTA_API_TOKEN`) — not the OIDC client secret used
|
||||
for sign-in. Fetch the full user list (paginated per Okta's API) rather than
|
||||
one-by-one lookups per local user; this app has ~20-odd accounts today, and a
|
||||
list-and-diff is simpler and cheaper than N calls. Return each Okta user's
|
||||
identity-claim value (matching `OKTA_IDENTITY_CLAIM`, already confirmed live
|
||||
per wave 10) and status.
|
||||
|
||||
Follow the existing pattern for external credentials in this repo (`MICRON_DB_URL`,
|
||||
`SMTP_PASSWORD`): env-only, never logged, never returned to the browser in an
|
||||
error message.
|
||||
|
||||
**Do not:** reuse `OKTA_CLIENT_ID`/`OKTA_CLIENT_SECRET` for this. Sign-in and
|
||||
the management API are different trust boundaries with different scopes;
|
||||
conflating them means a compromise or rotation of one affects the other
|
||||
unnecessarily.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] the client authenticates with its own credential, distinct from the OIDC
|
||||
client
|
||||
*(2026-10-16: `OKTA_API_TOKEN`, sent as `Authorization: SSWS`. `tests/okta_sync_check.py` sets a different OIDC client secret and asserts it is never sent.)*
|
||||
- [x] it fetches the complete Okta user list, handling pagination
|
||||
*(2026-10-16: follows `Link: rel="next"`, 200 per page, 2-page case checked. A next link to another host is refused, a chain longer than 50 pages raises instead of cutting the list off, redirects are never followed.)*
|
||||
- [x] an auth failure or malformed response raises a clear, specific error
|
||||
rather than returning an empty list indistinguishable from "everyone was
|
||||
deprovisioned" — this distinction is what T13.3 depends on
|
||||
*(2026-10-16: `OktaSyncError` for HTTP 401/403/404/429/5xx, redirects, network errors, timeouts, non-JSON, non-list, a bad item (never skipped), and an empty list. 31 failure checks. Mutation test: making the client return `[]` on an error fails 10 of them.)*
|
||||
- [x] the credential is never logged or surfaced in any API response
|
||||
*(2026-10-16: error messages carry the class name and HTTP status only, never the URL or a header. `describe()` has no token. Nothing is wired to an API response yet; T13.4 wires the startup log.)*
|
||||
|
||||
---
|
||||
|
||||
### T13.3 — D18: the sync job
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** T13.2
|
||||
- **Blocks:** T13.4
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/okta_sync.py`, `server/app.py` (or wherever `log_event`
|
||||
lives)
|
||||
|
||||
**Do:** Compare the Okta user list (T13.2) against local `users` rows. For any
|
||||
local `is_active=True` user whose Okta identity is missing from the list, or
|
||||
present with a non-active status, set `is_active=False` and write an
|
||||
`AuditLog` row (`actor="system:okta_sync"`, action e.g.
|
||||
`user_deprovisioned_by_sync`, detail naming the Okta status found). Never
|
||||
touch a user already `is_active=False`. Never re-enable anyone.
|
||||
|
||||
**The fail-closed rule, non-negotiable:** if T13.2's client raises an error of
|
||||
any kind (network, auth, malformed response, timeout), this task takes **no
|
||||
action on any account** for that run and logs the failure clearly (server log,
|
||||
at minimum). An error is never treated as "Okta returned zero active users."
|
||||
Write a test that asserts this directly: feed the sync a failing client and
|
||||
assert zero rows changed and zero `AuditLog` entries written.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] a user missing from Okta's list, or present but not active, is disabled
|
||||
with a correctly-detailed audit row
|
||||
*(2026-10-16: see "Decided 2026-10-16" below for which statuses count as "not active". Audit row: actor `system:okta_sync`, action `user_deprovisioned_by_sync`, `entity_type` user, `detail` `{okta_status, via}`; a missing user is recorded as `MISSING`.)*
|
||||
- [x] a user already disabled is left alone (no duplicate audit row each run)
|
||||
*(2026-10-16: a second run over the same data writes no new rows.)*
|
||||
- [x] an active Okta user already active locally produces no audit row (only
|
||||
changes are logged, not a clean bill of health every cycle)
|
||||
*(2026-10-16: verified in `tests/okta_sync_job_check.py`.)*
|
||||
- [x] **a simulated Okta API failure results in zero account changes and zero
|
||||
audit rows** — this is the one check that must never be skipped or
|
||||
weakened
|
||||
*(2026-10-16: 10 stand-in failures (OktaSyncError, RuntimeError, ConnectionError, TimeoutError, ValueError, KeyError, empty list, None, two kinds of garbage), 4 through the real client (HTTP 401/403/429/500), the default path with no token, a failed commit, and a wrong identity attribute. Each: `ok` is False, zero accounts changed, zero audit rows. Mutation test: removing the fail-closed branch and the zero-match guard fails 28 of these.)*
|
||||
- [x] the job never re-enables an account under any input
|
||||
*(2026-10-16: a disabled account that is ACTIVE in Okta stays disabled, and `okta_sync.py` contains no assignment of `is_active` to True.)*
|
||||
|
||||
---
|
||||
|
||||
**Decided 2026-10-16 (Matt, in chat):** which Okta statuses disable an account.
|
||||
The spec says "a non-active status". Read literally, that also catches
|
||||
`PASSWORD_EXPIRED`, `LOCKED_OUT` and `RECOVERY`, which people leave on their
|
||||
own, and this job never re-enables, so one AD password expiry would lock someone
|
||||
out of the app until an admin stepped in. Decided: an account disables when it is
|
||||
**missing** from Okta's list, or every Okta entry for it is **SUSPENDED** or
|
||||
**DEPROVISIONED** (`DISABLING_STATUSES` in `server/okta_sync.py`). Okta's default
|
||||
user list already leaves out DEPROVISIONED users, so those arrive as "missing".
|
||||
|
||||
**Added 2026-10-16, inside decision 4's "anything the sync can't confidently
|
||||
parse":** if there are active local accounts and **none** of them matches any
|
||||
Okta user, the run is refused and nothing changes. That pattern means the
|
||||
identity attribute is wrong (`OKTA_SYNC_PROFILE_ATTRIBUTE`), not that the whole
|
||||
company left. It is a guard, not a new feature. Raise it in the PR.
|
||||
|
||||
**Open, for the PR / Nick:** which Okta profile attribute equals the
|
||||
`preferred_username` claim in this tenant. The client assumes `profile.login`,
|
||||
Okta's default. Confirm before the first live run. A wrong value is caught by the
|
||||
zero-match guard above only when it matches nobody at all; a value that matches
|
||||
some accounts and not others would disable the rest. See `BL-069`.
|
||||
|
||||
### T13.4 — D18: run it on a schedule
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** T13.3
|
||||
- **Blocks:** T13.6
|
||||
- **Surface:** `server/`, `docker-compose.yml`
|
||||
- **Files:** `server/app.py` (startup hook) or a new sidecar per the `backup`
|
||||
container's pattern — task's call, per the decision record's noted
|
||||
alternative
|
||||
|
||||
**Do:** Wire T13.3 to run on an interval (proposed 15 minutes, env-overridable
|
||||
— e.g. `OKTA_SYNC_INTERVAL_SECONDS`, matching `BACKUP_INTERVAL_SECONDS`'s
|
||||
naming). Default choice is an in-process background task in the `api`
|
||||
container; if a separate container is chosen instead, follow the `backup`
|
||||
service's shape (own Dockerfile or script, `internal` network plus whatever
|
||||
egress reaching Okta requires — check whether `outbound` as currently defined
|
||||
is sufficient or Okta needs a distinct allowance).
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] the job runs automatically on the configured interval without manual
|
||||
invocation
|
||||
*(2026-10-16: a daemon timer thread starts from the app's lifespan hook (`server/app.py`), checks once a minute, and ran the sync on its own in `tests/okta_sync_schedule_check.py`. With `OKTA_API_TOKEN` unset it starts nothing and the startup log says `Okta sync DISABLED`. Booted the real app both ways; the thread starts and ends with the app.)*
|
||||
- [x] interval is env-configurable with a sane default
|
||||
*(2026-10-16: `OKTA_SYNC_INTERVAL_SECONDS`, default 900. A non-number uses the default. A value under 300 is raised to 300. Passed through `docker-compose.yml`.)*
|
||||
- [x] a container restart does not produce a duplicate/overlapping run, and a
|
||||
slow cycle does not stack with the next one
|
||||
*(2026-10-16: the api runs 2 gunicorn workers, so each starts its own timer. One row in `app_settings` (key `okta_sync`) is the gate: a worker claims a cycle with one atomic conditional UPDATE, so exactly one wins. Checked with 8 racing workers on a file database, 15 rounds plus the first-insert race: exactly 1 winner every time. The row outlives a restart, so a restart inside the interval starts nothing. A slow run cannot overlap the next: a fetch is capped at 120 s in total and the interval cannot go below 300 s. Inside one process the thread runs one cycle at a time. This needed no migration: `app_settings` already exists.)*
|
||||
|
||||
---
|
||||
|
||||
**Decided 2026-10-16 (task's call, per this task's "Do"):** in-process timer in the
|
||||
`api` container, as D18 proposed, not a sidecar. The `backup` pattern was the
|
||||
alternative; it would need its own image and would not remove the 2-worker
|
||||
duplicate question, because the sync reads the application database anyway.
|
||||
**Network check, as the task asked:** the `outbound` network the `api` service
|
||||
already joins is an ordinary bridge with a gateway, so Okta is reachable with no
|
||||
network change. The host firewall must allow https to the Okta org.
|
||||
**One change to T13.2's code:** `fetch_users()` now also stops after 120 seconds in
|
||||
total (`TOTAL_DEADLINE_SECONDS`), which is what lets the scheduler promise no
|
||||
overlap. Tested in `tests/okta_sync_check.py` section 8.
|
||||
|
||||
**T13.4 follow-up, 2026-10-16:** the sync's INFO log lines did not appear in
|
||||
`docker compose logs api`. Nothing in the app configures Python logging, and
|
||||
gunicorn configures only its own loggers, so INFO from any `wpsuite.*` logger is
|
||||
dropped. The T13.4 check booted the app in-process with logging switched on,
|
||||
which hid it. With the exact Dockerfile launch (2 gunicorn workers), a healthy
|
||||
start printed zero Okta lines. `okta_sync.py` now gives its own logger one stderr
|
||||
handler (only when the root logger has none). Re-run with the real gunicorn
|
||||
command: `configured` and `scheduled` (or `DISABLED`) now print once per worker.
|
||||
No other module's output changed. App-wide logging stays a separate question
|
||||
(BL-028 is the same family).
|
||||
|
||||
### T13.5 — D18: admin visibility
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** T13.3
|
||||
- **Blocks:** T13.6
|
||||
- **Surface:** `html/`
|
||||
- **Files:** `html/users.js` / `html/admin.js` (wherever audit history is
|
||||
already surfaced)
|
||||
|
||||
**Do:** Confirm an auto-disable reads clearly wherever admins already look at
|
||||
account history — the actor string (`system:okta_sync`) should be
|
||||
self-explanatory in context, not require reading server logs to understand.
|
||||
Do not build new UI beyond making sure the existing audit surface renders this
|
||||
actor sensibly. Email/notification to admins on auto-disable is a noted
|
||||
fast-follow (the decision record flags it as open, not required here) — do not
|
||||
build it in this task; log it instead if it's tempting to add.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] an auto-disabled account's audit entry is visible and legible in the
|
||||
existing admin UI without special-casing
|
||||
*(2026-10-16: run on the PC against the real page, 1440px and 390px, `tests/okta_sync_ui_check.py` 28/28. The admin Activity log card reads: Who `system:okta_sync`, Action `user deprovisioned by sync`, Type `user`, Item the username, Detail `okta_status: SUSPENDED, via: okta_sync` (a person missing from Okta reads `okta_status: MISSING`). Search for `okta_sync` and the User accounts filter both find the rows. No sideways scroll, no JavaScript errors. The disabled account is refused on its very next request; an account still active in Okta is not. A project super user does not see these rows: they belong to no project, so only admins do, the same as other user-management events. No code in `html/` changed.)*
|
||||
- [x] nothing here silently assumes `CR-019`'s activity view exists yet — this
|
||||
must work standalone
|
||||
*(2026-10-16: the check reads only the Activity log (AuditLog) card. The new Activity & usage card is on the page but is not read.)*
|
||||
|
||||
---
|
||||
|
||||
**No UI was built.** The existing card renders the new rows legibly as is. Two
|
||||
things you might expect and did not get, both on purpose: no admin email on an
|
||||
auto-disable (D18 calls it a fast-follow; not built, not logged as a new item
|
||||
because D18 already names it), and no per-user history view in the User
|
||||
Directory (it has none today, and the task says not to build one).
|
||||
|
||||
### T13.6 — D18: verification
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** T13.4, T13.5
|
||||
- **Blocks:** nothing
|
||||
- **Surface:** `server/`
|
||||
- **Files:** as touched above
|
||||
|
||||
**Do:** Full verification per `CLAUDE.md`. Beyond the usual run: specifically
|
||||
re-run T13.3's fail-closed test in isolation and confirm it still passes after
|
||||
T13.4's scheduling wrapper is in place — the scheduling layer must not
|
||||
introduce a path that swallows the client's error and proceeds anyway.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] all `D18` acceptance criteria in `decisions-2026-09-17.md` are met or a
|
||||
failure is stated with a reason
|
||||
- [ ] the fail-closed behavior is verified end to end through the scheduled
|
||||
wrapper, not just the bare sync function
|
||||
- [ ] smoke test and `seed_demo.py` both still pass
|
||||
- [ ] full test suite passes
|
||||
|
||||
---
|
||||
|
||||
## Wave 13 exit criteria
|
||||
|
||||
- [ ] `AUTH_SESSION_HOURS` defaults to 2, documented
|
||||
- [ ] the sync job runs on a schedule and correctly disables accounts Okta no
|
||||
longer shows as active
|
||||
- [ ] every auto-disable is individually audited with a clearly non-human actor
|
||||
- [ ] an Okta API failure of any kind changes zero accounts — verified, not
|
||||
assumed
|
||||
- [ ] the sync never re-enables an account
|
||||
- [ ] `D18` fully accounted for, no open acceptance criteria
|
||||
|
||||
## Release note (2026-10-16)
|
||||
|
||||
T13.1 (commit `6cde6e3`) is held back from `main` until T14.1 lands, per the merge
|
||||
rule in `wave-12.md`. The release branch contains wave 11 and T13.2 to T13.5.
|
||||
`tests/session_idle_check.py` skips (exit 0) when `auth.maybe_refresh_token` is absent.
|
||||
Sessions keep the flat `AUTH_SESSION_HOURS` lifetime (default 12) until T13.1 lands.
|
||||
@@ -34,12 +34,12 @@
|
||||
/* Every container admin.js paints a table into is a scrollport of its own, so a
|
||||
sticky header always has something to stick to rather than sliding up behind
|
||||
the app bar. Same rule as console.css's .tscroll. */
|
||||
#comments-admin, #audit-admin, #notif-box, #usage-admin, #projects-table, #defmem-table{
|
||||
#comments-admin, #audit-admin, #notif-box, #activity-admin, #projects-table, #defmem-table{
|
||||
overflow:auto; max-height:min(70vh,640px); overscroll-behavior:contain; }
|
||||
/* If admin.js wraps its table in its own .tscroll, the outer box steps aside so
|
||||
one table never ends up with two scrollbars. */
|
||||
#comments-admin:has(.tscroll), #audit-admin:has(.tscroll), #notif-box:has(.tscroll),
|
||||
#usage-admin:has(.tscroll), #projects-table:has(.tscroll), #defmem-table:has(.tscroll){
|
||||
#activity-admin:has(.tscroll), #projects-table:has(.tscroll), #defmem-table:has(.tscroll){
|
||||
overflow:visible; max-height:none; }
|
||||
|
||||
/* Comment text and audit detail are the two columns you are actually here to
|
||||
@@ -53,7 +53,7 @@
|
||||
border:1px solid var(--border-strong); border-radius:0; margin-bottom:var(--s3); }
|
||||
|
||||
@media (max-width:900px){
|
||||
#comments-admin, #audit-admin, #notif-box, #usage-admin, #projects-table, #defmem-table{
|
||||
#comments-admin, #audit-admin, #notif-box, #activity-admin, #projects-table, #defmem-table{
|
||||
max-height:none; }
|
||||
}
|
||||
</style>
|
||||
@@ -175,16 +175,38 @@
|
||||
<div id="audit-admin" class="note">Click refresh to load.</div>
|
||||
</div>
|
||||
|
||||
<!-- USAGE LOGS -->
|
||||
<!-- ACTIVITY & USAGE (CR-019) -->
|
||||
<div class="card">
|
||||
<h2>Usage logs</h2>
|
||||
<div class="sub">Engagement recorded by both tools — the work package creator and the SOP wizard —
|
||||
sessions, actions and counts, with a download per tool (D5). Note: stored locally per browser,
|
||||
so this reflects activity on <strong>this</strong> machine.</div>
|
||||
<h2>Activity & usage</h2>
|
||||
<div class="sub">Who is using the suite, which tools, and how often — recorded server-side on every
|
||||
sign-in and page open, kept indefinitely. Filter below, or export a CSV: raw (real usernames) for
|
||||
internal use, or sanitized (each user replaced with a stable, non-reversible id) for feeding into
|
||||
Power BI or another external reporting tool without carrying real identities.</div>
|
||||
<div class="toolbar">
|
||||
<button onclick="loadUsage()">Refresh</button>
|
||||
<label for="act-from">From</label>
|
||||
<input type="date" id="act-from" onchange="loadActivity()">
|
||||
<label for="act-to">To</label>
|
||||
<input type="date" id="act-to" onchange="loadActivity()">
|
||||
<select id="act-project" onchange="loadActivity()"><option value="">All projects</option></select>
|
||||
<input id="act-user" placeholder="Username…" oninput="loadActivity()">
|
||||
<select id="act-tool" onchange="loadActivity()">
|
||||
<option value="">All tools</option>
|
||||
<option value="launcher">Launcher</option>
|
||||
<option value="wizard">SOP wizard</option>
|
||||
<option value="creator">Work package creator</option>
|
||||
<option value="field_view">Field view</option>
|
||||
<option value="admin">Admin console</option>
|
||||
<option value="directory">User directory</option>
|
||||
</select>
|
||||
<button onclick="loadActivity()">Refresh</button>
|
||||
</div>
|
||||
<div id="usage-admin" class="note">Click refresh to load.</div>
|
||||
<div id="activity-banner" class="banner" style="display:none"></div>
|
||||
<div id="activity-admin" class="note">Loading…</div>
|
||||
<div class="toolbar" style="margin-top:var(--s3)">
|
||||
<button onclick="exportActivity(false)">Download CSV (raw)</button>
|
||||
<button onclick="exportActivity(true)">Download CSV (sanitized)</button>
|
||||
</div>
|
||||
<div id="activity-export-banner" class="banner" style="display:none"></div>
|
||||
</div>
|
||||
|
||||
<!-- DB SNAPSHOT -->
|
||||
@@ -215,7 +237,6 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="wp-usage.js"></script>
|
||||
<script src="console-util.js"></script>
|
||||
<script src="wp-dialog.js"></script>
|
||||
<script src="admin.js"></script>
|
||||
|
||||
141
html/admin.js
@@ -24,7 +24,7 @@ function reveal(){
|
||||
loadNotifications();
|
||||
loadComments();
|
||||
loadAudit();
|
||||
loadUsage();
|
||||
loadActivity();
|
||||
}
|
||||
function showDenied(){
|
||||
document.getElementById('admin-denied').style.display='';
|
||||
@@ -189,6 +189,19 @@ async function loadProjects(){
|
||||
banner.style.display='none';
|
||||
_adminProjects = json;
|
||||
renderProjects();
|
||||
populateActivityProjectFilter();
|
||||
}
|
||||
|
||||
// The activity project filter reuses the same project list the Projects card
|
||||
// already fetched — no second /api/projects call just to fill a <select>.
|
||||
function populateActivityProjectFilter(){
|
||||
const sel = document.getElementById('act-project');
|
||||
if(!sel) return;
|
||||
const cur = sel.value;
|
||||
sel.innerHTML = '<option value="">All projects</option>' +
|
||||
_adminProjects.slice().sort((a,b)=>String(a.name||'').localeCompare(String(b.name||'')))
|
||||
.map(p => '<option value="'+uesc(p.id)+'">'+uesc(p.name||p.number||p.id)+'</option>').join('');
|
||||
sel.value = cur;
|
||||
}
|
||||
|
||||
function renderProjects(){
|
||||
@@ -465,6 +478,90 @@ function renderAudit(){
|
||||
'</tr>').join('')+'</tbody></table>';
|
||||
}
|
||||
|
||||
// ── activity & usage (CR-019) ────────────────────────────────────────────────────
|
||||
// Server-side, per-user activity — distinct from the "Activity log" card above
|
||||
// (that's AuditLog: business mutations) and from the "Usage logs" card below
|
||||
// (that's per-browser localStorage, D5/T7.10, on the way out per T11.6). This is
|
||||
// /api/usage/summary and /api/usage/export: real rows, aggregated server-side,
|
||||
// filterable by date/project/user/tool, and exportable raw or sanitized.
|
||||
function _activityFilters(){
|
||||
const q = new URLSearchParams();
|
||||
const from = document.getElementById('act-from').value; if(from) q.set('from', from);
|
||||
const to = document.getElementById('act-to').value; if(to) q.set('to', to);
|
||||
const proj = document.getElementById('act-project').value; if(proj) q.set('project_id', proj);
|
||||
const user = (document.getElementById('act-user').value||'').trim(); if(user) q.set('username', user);
|
||||
const tool = document.getElementById('act-tool').value; if(tool) q.set('tool', tool);
|
||||
return q;
|
||||
}
|
||||
async function loadActivity(){
|
||||
const banner = document.getElementById('activity-banner');
|
||||
const box = document.getElementById('activity-admin');
|
||||
if(!box) return;
|
||||
banner.style.display='none';
|
||||
box.textContent = 'Loading…';
|
||||
const { status, json } = await api('GET', '/api/usage/summary?'+_activityFilters().toString());
|
||||
if(status===403){
|
||||
banner.className='banner bad'; banner.style.display='';
|
||||
banner.textContent = '✕ Your account can’t see suite-wide activity here — this needs admin, or Project Super User on at least one project.';
|
||||
box.innerHTML=''; return;
|
||||
}
|
||||
if(status!==200 || !json){
|
||||
banner.className='banner bad'; banner.style.display='';
|
||||
banner.textContent = '✕ Could not load activity ('+apiError(status, json, 'load failed')+').';
|
||||
box.innerHTML=''; return;
|
||||
}
|
||||
renderActivity(json);
|
||||
}
|
||||
function renderActivity(sum){
|
||||
const box = document.getElementById('activity-admin');
|
||||
if(!sum.event_count){ box.innerHTML = '<div class="note">No activity matches this filter.</div>'; return; }
|
||||
const fmt = s => s ? wpFormatDateTime(s) : '—';
|
||||
const bucket = (label, obj, take) => {
|
||||
const entries = Object.entries(obj).sort((a,b)=> b[0].localeCompare(a[0])).slice(0, take);
|
||||
if(!entries.length) return '';
|
||||
return '<table class="kv" style="margin-top:8px"><caption style="text-align:left;font-weight:600;margin-bottom:4px">'+
|
||||
uesc(label)+'</caption>'+entries.map(([k,v]) => '<tr><th>'+uesc(k)+'</th><td>'+v+' active user'+(v===1?'':'s')+'</td></tr>').join('')+
|
||||
'</table>';
|
||||
};
|
||||
const perTool = Object.entries(sum.by_tool||{});
|
||||
const toolTable = perTool.length ? '<table class="users"><thead><tr><th>Tool</th><th>Events</th></tr></thead><tbody>'+
|
||||
perTool.map(([t,n]) => '<tr><td>'+uesc(t||'(login)')+'</td><td>'+n+'</td></tr>').join('')+'</tbody></table>' :
|
||||
'<div class="note">No per-tool data.</div>';
|
||||
const perUser = Object.entries(sum.per_user_last_active||{}).sort((a,b)=> String(b[1]).localeCompare(String(a[1])));
|
||||
const userTable = perUser.length ? '<table class="users"><thead><tr><th>User</th><th>Last active</th></tr></thead><tbody>'+
|
||||
perUser.map(([u,t]) => '<tr><td><strong>'+uesc(u)+'</strong></td><td style="color:var(--muted)">'+fmt(t)+'</td></tr>').join('')+
|
||||
'</tbody></table>' : '<div class="note">No per-user data.</div>';
|
||||
box.innerHTML =
|
||||
'<div class="note">'+sum.event_count+' event'+(sum.event_count===1?'':'s')+' matched.</div>'+
|
||||
'<div class="row" style="gap:var(--s4);flex-wrap:wrap;align-items:flex-start">'+
|
||||
'<div>'+bucket('Active users by day', sum.active_users.by_day, 30)+'</div>'+
|
||||
'<div>'+bucket('Active users by week', sum.active_users.by_week, 12)+'</div>'+
|
||||
'<div>'+bucket('Active users by month', sum.active_users.by_month, 12)+'</div>'+
|
||||
'</div>'+
|
||||
'<h2 style="margin-top:16px">By tool</h2>'+toolTable+
|
||||
'<h2 style="margin-top:16px">Per-user last active</h2>'+userTable;
|
||||
}
|
||||
async function exportActivity(sanitize){
|
||||
const banner = document.getElementById('activity-export-banner');
|
||||
banner.className='banner'; banner.style.display=''; banner.textContent='Preparing export…';
|
||||
const q = _activityFilters();
|
||||
if(sanitize) q.set('sanitize', 'true');
|
||||
const { status, json } = await api('GET', '/api/usage/export?'+q.toString());
|
||||
if(status!==200 || typeof json !== 'string'){
|
||||
banner.className='banner bad';
|
||||
banner.textContent = '✕ Export failed ('+apiError(status, json, 'export failed')+').';
|
||||
return;
|
||||
}
|
||||
const blob = new Blob([json], { type:'text/csv' });
|
||||
const a = document.createElement('a');
|
||||
a.href = URL.createObjectURL(blob);
|
||||
a.download = 'usage_export_'+(sanitize?'sanitized':'raw')+'_'+new Date().toISOString().slice(0,10)+'.csv';
|
||||
document.body.appendChild(a); a.click(); a.remove();
|
||||
setTimeout(() => URL.revokeObjectURL(a.href), 1000);
|
||||
banner.className='banner ok';
|
||||
banner.textContent = '✓ Downloaded the '+(sanitize?'sanitized':'raw')+' export.';
|
||||
}
|
||||
|
||||
// ── notifications / email settings ──────────────────────────────────────────────
|
||||
let _settings = {};
|
||||
async function loadSettings(){
|
||||
@@ -656,48 +753,6 @@ async function loadNotifications(){
|
||||
'</tr>').join('')+'</tbody></table>';
|
||||
}
|
||||
|
||||
// ── usage logs (read from this browser's localStorage) ──────────────────────────
|
||||
// D5 / T7.10: the report for BOTH tools' recorded usage, in the one place an
|
||||
// operator-facing readout belongs - behind the same admin gate as this whole
|
||||
// page (gateByRole() below shows nothing else either). Data comes from
|
||||
// wp-usage.js, the single implementation; the keys predate the move, so
|
||||
// everything recorded before it is still here.
|
||||
function loadUsage(){
|
||||
const box = document.getElementById('usage-admin');
|
||||
if(!box) return;
|
||||
const tools = [
|
||||
['Work package creator', WPUsage.KEYS.creator, 'wp-iwp-usage'],
|
||||
['SOP wizard', WPUsage.KEYS.wizard, 'wp-suite-usage'],
|
||||
];
|
||||
let html = '';
|
||||
tools.forEach(([label, key, prefix]) => {
|
||||
const evs = (WPUsage.load(key).events) || [];
|
||||
html += '<h2 style="margin-top:16px">' + uesc(label) + '</h2>';
|
||||
if(!evs.length){
|
||||
html += '<div class="note">No usage recorded in this browser yet.</div>';
|
||||
return;
|
||||
}
|
||||
const byEvent = {}, sessions = new Set();
|
||||
let first = evs[0].ts, last = evs[0].ts;
|
||||
evs.forEach(e => {
|
||||
byEvent[e.event] = (byEvent[e.event]||0)+1;
|
||||
if(e.session) sessions.add(e.session);
|
||||
if(e.ts < first) first = e.ts; if(e.ts > last) last = e.ts;
|
||||
});
|
||||
const fmt = v => v ? wpFormatDateTime(v) : '—';
|
||||
html += '<table class="kv">'+
|
||||
'<tr><th>Sessions</th><td>'+sessions.size+'</td></tr>'+
|
||||
'<tr><th>Events</th><td>'+evs.length+'</td></tr>'+
|
||||
'<tr><th>Range</th><td style="font-weight:600">'+fmt(first)+' → '+fmt(last)+'</td></tr></table>';
|
||||
html += '<table class="users"><thead><tr><th>Event</th><th>Count</th></tr></thead><tbody>';
|
||||
Object.keys(byEvent).sort().forEach(k => html += '<tr><td>'+uesc(k)+'</td><td>'+byEvent[k]+'</td></tr>');
|
||||
html += '</tbody></table>';
|
||||
html += '<div class="toolbar" style="margin-top:8px"><button onclick="WPUsage.download(WPUsage.KEYS.'+
|
||||
(key === WPUsage.KEYS.creator ? 'creator' : 'wizard')+', ' + jsq(prefix) + ')">Download the full event log</button></div>';
|
||||
});
|
||||
box.innerHTML = html;
|
||||
}
|
||||
|
||||
// ── access control: admins only ─────────────────────────────────────────────────
|
||||
// auth-guard.js requires a login and sets window.WP_USER (firing 'wp-auth-ready').
|
||||
// Show the console for admins; otherwise show the "Admins only" notice.
|
||||
|
||||
@@ -125,12 +125,40 @@
|
||||
// Nothing replaces it. Every signed-in page mounts the drawer, so there is no page
|
||||
// left that would need a floating fallback pill.
|
||||
|
||||
// ── CR-019: usage ping ───────────────────────────────────────────────────
|
||||
// One page_open event per authenticated load, sent from exactly ONE place
|
||||
// (here) rather than from each page's own script - the shared-chrome lesson
|
||||
// S4 and the token-drift lesson S5 both taught this codebase the hard way.
|
||||
// Fire-and-forget: never blocks reveal(), never retries, never surfaces an
|
||||
// error to the person using the app - a missed usage ping is not something
|
||||
// anyone here should notice happening.
|
||||
var TOOL_BY_PAGE = {
|
||||
'index.html': 'launcher',
|
||||
'work-package-suite.html': 'wizard',
|
||||
'wp-creation-index.html': 'creator',
|
||||
'field.html': 'field_view',
|
||||
'admin.html': 'admin',
|
||||
'users.html': 'directory'
|
||||
};
|
||||
function pingUsage() {
|
||||
var page = (location.pathname.split('/').pop() || 'index.html');
|
||||
var tool = TOOL_BY_PAGE[page] || page.replace(/\.html$/, '');
|
||||
try {
|
||||
fetch('/api/usage/ping', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ tool: tool })
|
||||
}).catch(function () {});
|
||||
} catch (e) {}
|
||||
}
|
||||
|
||||
function proceed(user) {
|
||||
clearTimeout(safety);
|
||||
window.WP_USER = user;
|
||||
reveal();
|
||||
if (window.WP_USER) {
|
||||
window.wpFlags(); // start the feature-flag fetch; pages await it as needed
|
||||
pingUsage();
|
||||
try { document.dispatchEvent(new CustomEvent('wp-auth-ready', { detail: window.WP_USER })); } catch (e) {}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2207,20 +2207,18 @@ function loadStepComments(){
|
||||
}
|
||||
}
|
||||
|
||||
// ── USAGE ANALYTICS ─────────────────────────────────────────────────────────
|
||||
// Lightweight usage analytics stored in localStorage so the tool owner can review
|
||||
// engagement over time. No field VALUES are stored (field-edit events record only
|
||||
// the field id), keeping captured data non-sensitive.
|
||||
// D5 / T7.10: the analytics implementation lives in wp-usage.js and the report
|
||||
// on the admin console. The wizard's own copy of showAnalytics() never had a
|
||||
// caller here - the button lived on the creator - and once B7 dissolved the
|
||||
// frame the duplicate sat in the same document as five colliding globals.
|
||||
// This page only records; dwell tracking keeps its page-local state below.
|
||||
// ── USAGE ANALYTICS (retired, T11.6) ────────────────────────────────────────
|
||||
// This used to write to wp-usage.js's per-browser localStorage log (D5/T7.10),
|
||||
// read back by the admin console's old "Usage logs" panel. CR-019 replaced
|
||||
// both with real, server-side, per-user activity (UsageEvent + the Activity &
|
||||
// usage card) - see decisions-2026-09-17.md for why this per-browser data was
|
||||
// never a source the new report could adopt. track() is now a no-op; kept
|
||||
// (rather than deleting its handful of call sites, including the dwell-timer
|
||||
// plumbing below) so this stays a one-line change instead of touching every
|
||||
// caller for the same outcome.
|
||||
let _stepEnter = Date.now();
|
||||
|
||||
function track(event, detail){
|
||||
WPUsage.track(WPUsage.KEYS.wizard, event, detail);
|
||||
}
|
||||
function track(event, detail){ /* retired, T11.6 — see comment above */ }
|
||||
function trackStepDwell(){
|
||||
const ms = Date.now() - _stepEnter;
|
||||
if(ms > 400 && ms < 1000*60*60) track('step_dwell', {step: currentStep, ms});
|
||||
|
||||
@@ -11,7 +11,6 @@
|
||||
<!-- Addressable state (S3). Parses before the app scripts, which read the URL
|
||||
during their own boot. -->
|
||||
<script src="wp-url.js"></script>
|
||||
<script src="wp-usage.js"></script>
|
||||
<script src="wp-list-import.js"></script>
|
||||
<!-- Autosave, unsaved-work guard, draft recovery (S2). -->
|
||||
<script src="wp-autosave.js"></script>
|
||||
|
||||
@@ -3993,11 +3993,17 @@ function openSopModal(){
|
||||
document.getElementById('sop-modal').classList.add('open'); track('view_sop');
|
||||
}
|
||||
function closeSopModal(){ document.getElementById('sop-modal').classList.remove('open'); }
|
||||
// D5 / T7.10: the analytics implementation lives in wp-usage.js - ONE copy for
|
||||
// the whole suite - and its report lives on the admin console, where an
|
||||
// operator-facing readout belongs. This page only records. Same key, same
|
||||
// event shape: everything recorded before the move is still readable after it.
|
||||
function track(event,detail){ if(devMode) return; WPUsage.track(WPUsage.KEYS.creator, event, detail); }
|
||||
// CR-019 / T11.6 (2026-09-23): this used to write to wp-usage.js's per-browser
|
||||
// localStorage log (D5/T7.10). Retired along with the admin console's old
|
||||
// "Usage logs" panel, its only reader - real, server-side, per-user activity
|
||||
// now exists (UsageEvent, the Activity & usage card, T11.1-T11.5) and this
|
||||
// data was never reliably tied to a real identity anyway, so it was not a
|
||||
// source the new report could adopt (decisions-2026-09-17.md). track() stays
|
||||
// as a no-op rather than deleting its ~35 call sites throughout this file:
|
||||
// removing every call individually is a much larger, riskier diff for the
|
||||
// same outcome, and a call site here still documents the moment worth
|
||||
// recording if usage analytics are ever rebuilt server-side.
|
||||
function track(event,detail){ /* retired, T11.6 — see comment above */ }
|
||||
|
||||
// ── COMMENTS ─────────────────────────────────────────────────────────────────
|
||||
const COMMENTS_KEY='wp_iwp_comments_v1';
|
||||
|
||||
@@ -11,7 +11,6 @@
|
||||
<!-- Addressable state (S3). Parses before the app scripts, which read the URL
|
||||
during their own boot. -->
|
||||
<script src="wp-url.js"></script>
|
||||
<script src="wp-usage.js"></script>
|
||||
<!-- Autosave, unsaved-work guard, draft recovery (S2). -->
|
||||
<script src="wp-autosave.js"></script>
|
||||
<!-- Which sections this project uses (CR-006). The same file the SOP wizard
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
/* Usage analytics core — the ONE implementation (D5 / T7.10).
|
||||
|
||||
This existed three times: the creator's copy, the wizard's copy (which had no
|
||||
caller — the button lived on the creator), and the admin console's own reader.
|
||||
Once the creator stopped being an iframe (B7/T7.1) the first two sat in one
|
||||
document as five colliding globals; an unreferenced duplicate is exactly what
|
||||
produced D5. One core now; the pages keep only a thin track() wrapper because
|
||||
page state (the creator's dev-mode pause) belongs to the page.
|
||||
|
||||
The storage KEYS are unchanged on purpose: everything recorded before this
|
||||
file existed is still readable through it. No field VALUES are ever stored —
|
||||
a field-edit event records the field id, nothing else.
|
||||
|
||||
Classic script, no modules: exposes window.WPUsage. */
|
||||
'use strict';
|
||||
|
||||
(function () {
|
||||
var SESSION = 's_' + Date.now().toString(36) + Math.random().toString(36).slice(2, 6);
|
||||
|
||||
function load(key) {
|
||||
try { return JSON.parse(localStorage.getItem(key)) || { events: [] }; }
|
||||
catch (e) { return { events: [] }; }
|
||||
}
|
||||
|
||||
function save(key, data) {
|
||||
try { localStorage.setItem(key, JSON.stringify(data)); }
|
||||
catch (e) { /* storage unavailable — degrade silently */ }
|
||||
}
|
||||
|
||||
function track(key, event, detail) {
|
||||
try {
|
||||
var d = load(key);
|
||||
d.events.push({ ts: new Date().toISOString(), session: SESSION, event: event, detail: detail || null });
|
||||
if (d.events.length > 5000) d.events = d.events.slice(-5000);
|
||||
save(key, d);
|
||||
} catch (e) { /* never let telemetry break the tool it watches */ }
|
||||
}
|
||||
|
||||
function download(key, prefix) {
|
||||
var blob = new Blob([JSON.stringify(load(key), null, 2)], { type: 'application/json' });
|
||||
var a = document.createElement('a');
|
||||
a.href = URL.createObjectURL(blob);
|
||||
a.download = (prefix || 'wp-usage') + '-' + new Date().toISOString().slice(0, 10) + '.json';
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
setTimeout(function () { URL.revokeObjectURL(a.href); }, 1000);
|
||||
}
|
||||
|
||||
window.WPUsage = {
|
||||
load: load,
|
||||
save: save,
|
||||
track: track,
|
||||
download: download,
|
||||
// The pre-D5 keys, verbatim — continuity of the recorded data is a done-when.
|
||||
KEYS: { creator: 'wp_iwp_analytics_v1', wizard: 'wp_suite_analytics_v1' },
|
||||
};
|
||||
})();
|
||||
@@ -55,6 +55,36 @@ OKTA_REDIRECT_URI=CHANGE_ME
|
||||
# a custom claim, …) — no code change needed, just this value.
|
||||
# OKTA_IDENTITY_CLAIM=preferred_username
|
||||
|
||||
# ── Okta deprovisioning sync (D18, wave 13) ────────────────────────────────────
|
||||
# A scheduled job asks Okta who is still active and disables local accounts
|
||||
# Okta no longer shows as active. It only ever DISABLES. It never re-enables.
|
||||
# If Okta cannot be reached or answers with anything unexpected, the job changes
|
||||
# NOTHING that cycle and logs the failure.
|
||||
#
|
||||
# A read-only Okta API token, created by the person who administers the Okta
|
||||
# tenant (Security > API > Tokens). This is NOT the OIDC client secret above.
|
||||
# It is a different credential with a different scope. Treat it like
|
||||
# AUTH_SECRET_KEY: never commit it, never log it. Leave it unset and the sync
|
||||
# stays off (the startup log says so).
|
||||
# OKTA_API_TOKEN=CHANGE_ME
|
||||
#
|
||||
# The Okta org URL that serves the Management API, e.g. https://yourorg.okta.com
|
||||
# Optional. If unset, it is the scheme and host of OKTA_ISSUER. Must be https.
|
||||
# OKTA_ORG_URL=https://your-org.okta.com
|
||||
#
|
||||
# Which Okta PROFILE attribute holds the value that matches users.username.
|
||||
# Optional. By default it follows OKTA_IDENTITY_CLAIM: preferred_username reads
|
||||
# profile.login, email reads profile.email, any other claim name reads the
|
||||
# profile attribute of the same name. Set this only if your tenant builds the
|
||||
# claim from a different attribute.
|
||||
# OKTA_SYNC_PROFILE_ATTRIBUTE=login
|
||||
#
|
||||
# How often the sync runs, in seconds. Default 900 (15 minutes), proposed and not
|
||||
# confirmed with IT. A value under 300 is raised to 300. Every api worker checks
|
||||
# once a minute and one of them wins each cycle, so raising the worker count does
|
||||
# not raise the call rate to Okta.
|
||||
# OKTA_SYNC_INTERVAL_SECONDS=900
|
||||
|
||||
# ── Email notifications (optional) ─────────────────────────────────────────────
|
||||
# WP-assignment emails are OFF by default and are turned on from the Admin
|
||||
# console (Notifications & email card), where the SMTP host/port/from-address
|
||||
|
||||
46
server/alembic/versions/8e2cb3003f8a_usage_events.py
Normal file
@@ -0,0 +1,46 @@
|
||||
"""usage events (CR-019, wave 11)
|
||||
|
||||
Append-only navigation/session activity, separate from audit_log on purpose —
|
||||
see the UsageEvent docstring in server/models.py. Retention is indefinite by
|
||||
decision (docs/waves/decisions-2026-09-17.md); nothing here schedules a purge.
|
||||
|
||||
Revision ID: 8e2cb3003f8a
|
||||
Revises: 1d60a608bb51
|
||||
Create Date: 2026-09-23 00:00:00.000000
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision = '8e2cb3003f8a'
|
||||
down_revision = '1d60a608bb51'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table('usage_events',
|
||||
sa.Column('id', sa.String(length=40), nullable=False),
|
||||
sa.Column('at', sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column('username', sa.String(length=200), nullable=False),
|
||||
sa.Column('project_id', sa.String(length=40), nullable=True),
|
||||
sa.Column('tool', sa.String(length=40), nullable=False),
|
||||
sa.Column('event', sa.String(length=40), nullable=False),
|
||||
sa.Column('detail', sa.JSON(), nullable=False),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index(op.f('ix_usage_events_at'), 'usage_events', ['at'], unique=False)
|
||||
op.create_index(op.f('ix_usage_events_username'), 'usage_events', ['username'], unique=False)
|
||||
op.create_index(op.f('ix_usage_events_project_id'), 'usage_events', ['project_id'], unique=False)
|
||||
op.create_index(op.f('ix_usage_events_tool'), 'usage_events', ['tool'], unique=False)
|
||||
op.create_index(op.f('ix_usage_events_event'), 'usage_events', ['event'], unique=False)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index(op.f('ix_usage_events_event'), table_name='usage_events')
|
||||
op.drop_index(op.f('ix_usage_events_tool'), table_name='usage_events')
|
||||
op.drop_index(op.f('ix_usage_events_project_id'), table_name='usage_events')
|
||||
op.drop_index(op.f('ix_usage_events_username'), table_name='usage_events')
|
||||
op.drop_index(op.f('ix_usage_events_at'), table_name='usage_events')
|
||||
op.drop_table('usage_events')
|
||||
202
server/app.py
@@ -9,10 +9,16 @@ Run (prod): gunicorn -k uvicorn.workers.UvicornWorker -b 127.0.0.1:8000 serve
|
||||
Interactive docs: http://<host>/api/docs
|
||||
"""
|
||||
import base64
|
||||
import csv
|
||||
import contextlib
|
||||
import hashlib
|
||||
import hmac
|
||||
import io
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Any, Optional
|
||||
from urllib.parse import urlparse
|
||||
|
||||
@@ -22,12 +28,12 @@ from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import JSONResponse, RedirectResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
from sqlalchemy import select, delete, func
|
||||
from sqlalchemy import select, delete, func, or_
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.middleware.sessions import SessionMiddleware
|
||||
|
||||
from .db import Base, engine, get_db
|
||||
from . import models, auth, notify, assets_db, okta_auth, okta_fake, files_http
|
||||
from .db import Base, SessionLocal, engine, get_db
|
||||
from . import models, auth, notify, assets_db, okta_auth, okta_fake, files_http, okta_sync
|
||||
|
||||
log = logging.getLogger("wpsuite.app")
|
||||
|
||||
@@ -42,7 +48,21 @@ if engine.dialect.name == "sqlite":
|
||||
# Interactive docs are handy in dev but hand an attacker the full API map in prod,
|
||||
# so enable them only on the SQLite dev fallback (production runs on Postgres).
|
||||
_docs_enabled = engine.dialect.name == "sqlite"
|
||||
@contextlib.asynccontextmanager
|
||||
async def _lifespan(_app):
|
||||
"""Start the Okta deprovisioning sync (D18 / T13.4) when the process starts and
|
||||
stop it when the process ends. With no OKTA_API_TOKEN it starts nothing and
|
||||
says so in the startup log."""
|
||||
stop = okta_sync.start_scheduler(SessionLocal)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
if stop is not None:
|
||||
stop.set()
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
lifespan=_lifespan,
|
||||
title="Work Package Suite API",
|
||||
docs_url="/api/docs" if _docs_enabled else None,
|
||||
redoc_url=None,
|
||||
@@ -756,6 +776,11 @@ async def okta_callback(request: Request, db: Session = Depends(get_db)):
|
||||
user.failed_attempts = 0
|
||||
user.locked_until = None
|
||||
user.last_login_at = models.utcnow()
|
||||
# CR-019: one login event per Okta sign-in, written here rather than
|
||||
# inferred from session creation elsewhere, so there is exactly one
|
||||
# source of truth for "did this person sign in" - not one per page load
|
||||
# afterward (T11.2 covers that separately, as page_open events).
|
||||
db.add(models.UsageEvent(id=gen_id("uev"), username=user.username, tool="", event="login"))
|
||||
db.commit()
|
||||
db.refresh(user)
|
||||
|
||||
@@ -3675,6 +3700,177 @@ def create_feedback(body: CommentIn, user: models.User = Depends(auth.get_curren
|
||||
return _save_comment(body, db, user)
|
||||
|
||||
|
||||
# ── CR-019: usage/activity metrics ──────────────────────────────────────────
|
||||
class UsageIn(BaseModel):
|
||||
tool: str = ""
|
||||
project_id: Optional[str] = None
|
||||
|
||||
|
||||
@app.post("/api/usage/ping")
|
||||
def usage_ping(body: UsageIn, user: models.User = Depends(auth.get_current_user), db: Session = Depends(get_db)):
|
||||
"""One usage_events row per authenticated page load. Called exactly once,
|
||||
from auth-guard.js after wp-auth-ready fires (T11.2) - never duplicated
|
||||
per page's own script, the same lesson S4's per-page nav already taught
|
||||
this codebase. `user` comes from the session via get_current_user, never
|
||||
from anything the client claims - identity here is a server-enforced
|
||||
fact, matching every other write in this file, not a client-reported one."""
|
||||
tool = (body.tool or "").strip()[:40]
|
||||
db.add(models.UsageEvent(
|
||||
id=gen_id("uev"), username=user.username, project_id=body.project_id,
|
||||
tool=tool, event="page_open",
|
||||
))
|
||||
db.commit()
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
def _parse_date_q(v: str, end: bool = False) -> Optional[datetime]:
|
||||
"""Accepts a plain YYYY-MM-DD (what a <input type=date> sends) or a full
|
||||
ISO datetime. A date-only `to` means "through the end of that day", not
|
||||
midnight at its start - otherwise a range of "today" would match nothing
|
||||
from today at all."""
|
||||
if not v:
|
||||
return None
|
||||
try:
|
||||
d = datetime.fromisoformat(v)
|
||||
except ValueError:
|
||||
return None
|
||||
if d.tzinfo is None:
|
||||
d = d.replace(tzinfo=timezone.utc)
|
||||
if end and len(v) <= 10: # date-only
|
||||
d = d + timedelta(days=1) - timedelta(microseconds=1)
|
||||
return d
|
||||
|
||||
|
||||
def _usage_query(db: Session, caller: "models.User", date_from, date_to, project_id, username, tool):
|
||||
"""Shared by the summary and export endpoints so the two can never
|
||||
disagree about which rows a filter set matches - the export is a raw
|
||||
dump of exactly what the summary counted, not a separately-derived view."""
|
||||
stmt = select(models.UsageEvent)
|
||||
managed = managed_project_ids(db, caller)
|
||||
if managed is not None:
|
||||
# A project_super_user (never an app admin, who gets managed=None) is
|
||||
# scoped to events tied to a project they administer, plus their OWN
|
||||
# suite-wide activity (admin console opens etc. carry no project_id) -
|
||||
# never another user's activity outside what they manage.
|
||||
if managed:
|
||||
stmt = stmt.where(or_(
|
||||
models.UsageEvent.project_id.in_(managed),
|
||||
models.UsageEvent.username == caller.username,
|
||||
))
|
||||
else:
|
||||
stmt = stmt.where(models.UsageEvent.username == caller.username)
|
||||
df = _parse_date_q(date_from)
|
||||
dt = _parse_date_q(date_to, end=True)
|
||||
if df:
|
||||
stmt = stmt.where(models.UsageEvent.at >= df)
|
||||
if dt:
|
||||
stmt = stmt.where(models.UsageEvent.at <= dt)
|
||||
if project_id:
|
||||
stmt = stmt.where(models.UsageEvent.project_id == project_id)
|
||||
if username:
|
||||
stmt = stmt.where(models.UsageEvent.username == username)
|
||||
if tool:
|
||||
stmt = stmt.where(models.UsageEvent.tool == tool)
|
||||
return stmt.order_by(models.UsageEvent.at)
|
||||
|
||||
|
||||
@app.get("/api/usage/summary")
|
||||
def usage_summary(
|
||||
date_from: Optional[str] = Query(None, alias="from"),
|
||||
date_to: Optional[str] = Query(None, alias="to"),
|
||||
project_id: Optional[str] = Query(None),
|
||||
username: Optional[str] = Query(None),
|
||||
tool: Optional[str] = Query(None),
|
||||
caller: models.User = Depends(require_user_manager),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""CR-019. Same gate as the User Directory (require_user_manager): an app
|
||||
admin or a project_super_user with at least one managed project. Filters
|
||||
combine. Aggregated in Python over the filtered row set rather than a SQL
|
||||
GROUP BY - correct and simple at today's scale; if usage_events grows
|
||||
into the millions (plausible, given retention is indefinite by decision),
|
||||
the day/week/month bucketing here is the first thing to move server-side
|
||||
into SQL. Not done now because nothing currently requires it."""
|
||||
rows = db.scalars(_usage_query(db, caller, date_from, date_to, project_id, username, tool)).all()
|
||||
|
||||
by_day: dict[str, set] = {}
|
||||
by_week: dict[str, set] = {}
|
||||
by_month: dict[str, set] = {}
|
||||
per_user_last: dict[str, datetime] = {}
|
||||
per_tool: dict[str, int] = {}
|
||||
for e in rows:
|
||||
by_day.setdefault(e.at.date().isoformat(), set()).add(e.username)
|
||||
by_week.setdefault(e.at.strftime("%G-W%V"), set()).add(e.username)
|
||||
by_month.setdefault(e.at.strftime("%Y-%m"), set()).add(e.username)
|
||||
if e.username not in per_user_last or e.at > per_user_last[e.username]:
|
||||
per_user_last[e.username] = e.at
|
||||
per_tool[e.tool] = per_tool.get(e.tool, 0) + 1
|
||||
|
||||
return {
|
||||
"active_users": {
|
||||
"by_day": {k: len(v) for k, v in sorted(by_day.items())},
|
||||
"by_week": {k: len(v) for k, v in sorted(by_week.items())},
|
||||
"by_month": {k: len(v) for k, v in sorted(by_month.items())},
|
||||
},
|
||||
"per_user_last_active": {u: models._iso(t) for u, t in sorted(per_user_last.items())},
|
||||
"by_tool": dict(sorted(per_tool.items(), key=lambda kv: -kv[1])),
|
||||
"event_count": len(rows),
|
||||
}
|
||||
|
||||
|
||||
def _pseudonym(username: str) -> str:
|
||||
"""A stable per-user id for the sanitized export — the SAME input always
|
||||
produces the SAME output, within one export and across separate export
|
||||
runs, so an external system (Power BI or similar) can still group and
|
||||
trend "by user" without ever receiving a real name. HMAC rather than a
|
||||
plain hash: a plain sha256(username) is trivially reversed against a
|
||||
wordlist of the handful of usernames this app actually has; keying it
|
||||
with AUTH_SECRET_KEY (already a real secret, already required in
|
||||
production — see auth.py) means recovering a username from its
|
||||
pseudonym requires the signing key, not just guessing."""
|
||||
digest = hmac.new(auth.SECRET_KEY.encode(), username.encode(), hashlib.sha256).hexdigest()
|
||||
return "u_" + digest[:16]
|
||||
|
||||
|
||||
@app.get("/api/usage/export")
|
||||
def usage_export(
|
||||
date_from: Optional[str] = Query(None, alias="from"),
|
||||
date_to: Optional[str] = Query(None, alias="to"),
|
||||
project_id: Optional[str] = Query(None),
|
||||
username: Optional[str] = Query(None),
|
||||
tool: Optional[str] = Query(None),
|
||||
sanitize: bool = Query(False),
|
||||
caller: models.User = Depends(require_user_manager),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""CR-019. Same gate, same filters, same underlying row set as
|
||||
usage_summary() (_usage_query) — the export can never show a different
|
||||
slice of data than what the console counted for the same filters.
|
||||
|
||||
sanitize=true replaces `username` with a stable pseudonym (_pseudonym)
|
||||
and — deliberately — the `detail` column is not exported in EITHER mode.
|
||||
Every event this app writes today (login, page_open) leaves `detail`
|
||||
empty, so this costs nothing now, but it also means a future event type
|
||||
that DOES populate `detail` can't accidentally leak a real name into a
|
||||
sanitized file through a column nobody thought to scrub. If `detail`
|
||||
is ever needed in the export, it has to be sanitized explicitly, not
|
||||
assumed safe because the rest of the row was."""
|
||||
rows = db.scalars(_usage_query(db, caller, date_from, date_to, project_id, username, tool)).all()
|
||||
buf = io.StringIO()
|
||||
w = csv.writer(buf)
|
||||
w.writerow(["at", "username", "project_id", "tool", "event"])
|
||||
for e in rows:
|
||||
who = _pseudonym(e.username) if sanitize else e.username
|
||||
w.writerow([models._iso(e.at), who, e.project_id or "", e.tool, e.event])
|
||||
filename = "usage_export_%s_%s.csv" % (
|
||||
"sanitized" if sanitize else "raw", datetime.now(timezone.utc).strftime("%Y%m%d"),
|
||||
)
|
||||
return Response(
|
||||
content=buf.getvalue(), media_type="text/csv",
|
||||
headers={"Content-Disposition": f'attachment; filename="{filename}"'},
|
||||
)
|
||||
|
||||
|
||||
@app.get("/api/comments")
|
||||
def list_comments(
|
||||
source: Optional[str] = Query(None),
|
||||
|
||||
@@ -191,8 +191,7 @@ def decode_token(token: str) -> Optional[dict]:
|
||||
if not token or len(token) > _MAX_TOKEN_CHARS:
|
||||
return None
|
||||
try:
|
||||
# Every token this app issues carries exp and sub (create_token,
|
||||
# maybe_refresh_token); one without them is not ours.
|
||||
# Every token this app issues carries exp and sub (create_token); one without them is not ours.
|
||||
claims = jwt.decode(token, SECRET_KEY, algorithms=[JWT_ALG],
|
||||
options={"require": ["exp", "sub"]})
|
||||
except Exception: # noqa: BLE001 — see docstring: every failure is "not signed in"
|
||||
|
||||
@@ -337,6 +337,45 @@ class AuditLog(Base):
|
||||
}
|
||||
|
||||
|
||||
class UsageEvent(Base):
|
||||
"""CR-019: append-only record of who used the suite, when, and which tool —
|
||||
navigation/session activity, not business mutations. Deliberately a SEPARATE
|
||||
table from AuditLog rather than a new `action` value there: AuditLog answers
|
||||
"who changed what" and is read by people auditing a specific record's
|
||||
history; mixing in a `page_open` row for every authenticated page load
|
||||
would make that trail noisy for its existing purpose. This table answers a
|
||||
different question — "who is active, and on what" — and CR-019's admin
|
||||
console reads from here, not from AuditLog.
|
||||
|
||||
Not a ForeignKey to `users`, matching AuditLog's own reasoning: a user who
|
||||
is later removed should still show up in historical activity rather than
|
||||
silently vanishing from it, and `D18`'s deprovisioning sync only ever sets
|
||||
`is_active=False` — it never deletes a row — so this is defensive symmetry
|
||||
rather than a live concern today.
|
||||
|
||||
Retention is indefinite (decided 2026-09-17, `decisions-2026-09-17.md`) —
|
||||
nothing purges rows written here; that is a deliberate product decision,
|
||||
not an oversight to fix later."""
|
||||
__tablename__ = "usage_events"
|
||||
|
||||
id: Mapped[str] = mapped_column(String(40), primary_key=True)
|
||||
at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True)
|
||||
username: Mapped[str] = mapped_column(String(200), default="", index=True)
|
||||
project_id: Mapped[Optional[str]] = mapped_column(String(40), nullable=True, index=True)
|
||||
# creator | wizard | field_view | dashboard | admin | directory | ...
|
||||
tool: Mapped[str] = mapped_column(String(40), default="", index=True)
|
||||
# page_open | login
|
||||
event: Mapped[str] = mapped_column(String(40), default="", index=True)
|
||||
detail: Mapped[dict] = mapped_column(JSON, default=dict)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"id": self.id, "at": _iso(self.at), "username": self.username,
|
||||
"project_id": self.project_id, "tool": self.tool, "event": self.event,
|
||||
"detail": self.detail or {},
|
||||
}
|
||||
|
||||
|
||||
class AppSetting(Base):
|
||||
"""Admin-editable application settings (feature flags, SMTP config, …) stored
|
||||
as key -> JSON value. Read/written via /api/settings (admin only). Secrets like
|
||||
|
||||
479
server/okta_sync.py
Normal file
@@ -0,0 +1,479 @@
|
||||
"""Okta Management API client - T13.2, wave 13 (D18).
|
||||
|
||||
This module asks Okta one question: who is in the org, and what is each person's
|
||||
status? It is the first half of the deprovisioning sync. It does not touch the
|
||||
database and it does not disable anyone. T13.3 does that, using the list this
|
||||
module returns.
|
||||
|
||||
Trust boundary. Sign-in uses the OIDC client (OKTA_CLIENT_ID and OKTA_CLIENT_SECRET,
|
||||
see okta_auth.py). This module uses a different credential, OKTA_API_TOKEN, a
|
||||
read-only Okta API token. The two never mix. A rotation or a leak of one does not
|
||||
touch the other.
|
||||
|
||||
Failure rule (D18, decision 4). This is the most important rule in this file:
|
||||
**an error is never an empty list.** If anything goes wrong, this module raises
|
||||
OktaSyncError. It never returns [] to mean "the call failed". T13.3 reads an empty
|
||||
list as "nobody is active", so a function that returned [] on a network error
|
||||
would disable every account on an Okta outage. The cases that raise are:
|
||||
|
||||
- the token or the org URL is not set
|
||||
- a network error or a timeout
|
||||
- any HTTP status that is not 200 (401, 403 and 429 included)
|
||||
- a body that is not a JSON list, or an item that is not shaped as expected
|
||||
- a user with no usable identity value
|
||||
- a list with no users at all (a real org is never empty; an empty list means
|
||||
the token cannot see the users)
|
||||
- a page limit reached (a cut-off list would make real users look missing)
|
||||
- a "next page" link that points to a different host
|
||||
|
||||
T13.3 (the sync job, `run_sync` below) uses that list to disable local accounts.
|
||||
Its rules are in its own docstring. In short: it only ever disables, it never
|
||||
re-enables, it writes one audit row per change, and any failure means NO change.
|
||||
|
||||
The token is read from the environment each time it is used. It is never logged,
|
||||
never put in an error message, and never returned to a browser. Errors carry the
|
||||
exception class name and the HTTP status only. httpx exceptions can include the
|
||||
request URL, so the URL is not copied into the message either.
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Callable, Optional
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import httpx
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from . import models, okta_auth
|
||||
|
||||
log = logging.getLogger("wpsuite.okta_sync")
|
||||
|
||||
|
||||
def _make_logs_visible() -> None:
|
||||
"""Make this module's INFO lines reach `docker compose logs api`.
|
||||
|
||||
Nothing in this app configures Python logging, and gunicorn only configures its
|
||||
own loggers. So the root logger sits at WARNING with no handler, and every
|
||||
INFO line from a `wpsuite.*` logger is dropped. Found on 2026-10-16: with the
|
||||
exact production launch command, `grep "Okta sync"` found nothing on a healthy
|
||||
start, because "configured", "scheduled" and "done" are INFO lines. Warnings
|
||||
and errors (a disable, a skipped run) showed, but the lines that say the sync
|
||||
is alive did not. An operator cannot tell "off" from "on and quiet".
|
||||
|
||||
This adds one stderr handler to THIS module's logger only, and only when the
|
||||
root logger has none, so it never doubles up if logging is configured later and
|
||||
it does not change what any other module prints. (BL-028 and the general
|
||||
question of app-wide logging are separate items.)"""
|
||||
if log.handlers or logging.getLogger().handlers:
|
||||
return
|
||||
handler = logging.StreamHandler()
|
||||
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s"))
|
||||
log.addHandler(handler)
|
||||
log.setLevel(logging.INFO)
|
||||
|
||||
|
||||
_make_logs_visible()
|
||||
|
||||
# Okta's own page maximum for this endpoint. Asking for the maximum keeps a
|
||||
# 20-odd account org to a single call.
|
||||
PAGE_LIMIT = 200
|
||||
# A cut-off list is more dangerous than an error, so the loop is bounded and
|
||||
# raises when the bound is reached. 50 pages of 200 is 10,000 people.
|
||||
MAX_PAGES = 50
|
||||
TIMEOUT_SECONDS = 15.0
|
||||
# The whole fetch, all pages together, must finish inside this. T13.4 relies on it:
|
||||
# a run is always shorter than the shortest allowed interval, so one cycle can
|
||||
# never still be running when the next is due.
|
||||
TOTAL_DEADLINE_SECONDS = 120.0
|
||||
|
||||
# Which Okta *profile attribute* matches each OIDC claim that OKTA_IDENTITY_CLAIM
|
||||
# may name. The sign-in flow reads a claim from the ID token. This API returns
|
||||
# the profile instead. For the default claim, preferred_username, Okta's own
|
||||
# default maps it to profile.login. A claim that is not in this table is read
|
||||
# from profile[<claim>] (for example a custom attribute).
|
||||
# OKTA_SYNC_PROFILE_ATTRIBUTE overrides the whole table. Use it if the tenant
|
||||
# builds the claim from a different attribute.
|
||||
_CLAIM_TO_PROFILE = {
|
||||
"preferred_username": "login",
|
||||
"email": "email",
|
||||
}
|
||||
|
||||
|
||||
class OktaSyncError(Exception):
|
||||
"""The Okta user list could not be fetched with confidence. T13.3 must take no
|
||||
action on any account when it sees this."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OktaUser:
|
||||
identity: str # the value that matches users.username (lower case is NOT applied here)
|
||||
status: str # Okta's status, upper case: ACTIVE, SUSPENDED, DEPROVISIONED, ...
|
||||
|
||||
|
||||
# ── configuration ─────────────────────────────────────────────────────────────
|
||||
def _token() -> str:
|
||||
return os.getenv("OKTA_API_TOKEN", "").strip()
|
||||
|
||||
|
||||
def org_url() -> str:
|
||||
"""The Okta org base URL, for example https://yourorg.okta.com. The Management
|
||||
API lives here, not at the authorization server issuer. OKTA_ORG_URL wins if
|
||||
set. Otherwise it is the scheme and host of OKTA_ISSUER."""
|
||||
explicit = os.getenv("OKTA_ORG_URL", "").strip().rstrip("/")
|
||||
if explicit:
|
||||
return explicit
|
||||
issuer = (okta_auth.ISSUER or "").strip()
|
||||
parsed = urlparse(issuer)
|
||||
if parsed.scheme and parsed.netloc:
|
||||
return f"{parsed.scheme}://{parsed.netloc}"
|
||||
return ""
|
||||
|
||||
|
||||
def is_configured() -> bool:
|
||||
return bool(_token() and org_url().startswith("https://"))
|
||||
|
||||
|
||||
def describe() -> str:
|
||||
"""One line for the startup log. Never contains the token."""
|
||||
if not _token():
|
||||
return "Okta sync DISABLED - OKTA_API_TOKEN is not set. Deprovisioning is not detected."
|
||||
base = org_url()
|
||||
if not base:
|
||||
return "Okta sync DISABLED - no OKTA_ORG_URL and OKTA_ISSUER has no host. Deprovisioning is not detected."
|
||||
if not base.startswith("https://"):
|
||||
return f"Okta sync DISABLED - org URL {base!r} is not https. The token is never sent over http."
|
||||
return f"Okta sync configured - org {base}, identity claim {okta_auth.IDENTITY_CLAIM!r}"
|
||||
|
||||
|
||||
def _profile_attribute() -> str:
|
||||
override = os.getenv("OKTA_SYNC_PROFILE_ATTRIBUTE", "").strip()
|
||||
if override:
|
||||
return override
|
||||
claim = okta_auth.IDENTITY_CLAIM
|
||||
return _CLAIM_TO_PROFILE.get(claim, claim)
|
||||
|
||||
|
||||
# ── the client ────────────────────────────────────────────────────────────────
|
||||
def _fail(reason: str) -> "OktaSyncError":
|
||||
# One place to build the error, so no call site can add a URL or a header by
|
||||
# accident.
|
||||
return OktaSyncError(f"Okta user list could not be fetched: {reason}")
|
||||
|
||||
|
||||
def _same_origin(url: str, base: str) -> bool:
|
||||
a, b = urlparse(url), urlparse(base)
|
||||
return (a.scheme, a.netloc) == (b.scheme, b.netloc)
|
||||
|
||||
|
||||
def _parse_user(item, attribute: str) -> OktaUser:
|
||||
if not isinstance(item, dict):
|
||||
raise _fail("a list item is not an object")
|
||||
status = item.get("status")
|
||||
profile = item.get("profile")
|
||||
if not isinstance(status, str) or not status.strip():
|
||||
raise _fail("a user has no status")
|
||||
if not isinstance(profile, dict):
|
||||
raise _fail("a user has no profile")
|
||||
value = profile.get(attribute)
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
raise _fail(f"a user has no value for profile attribute {attribute!r}")
|
||||
return OktaUser(identity=value.strip(), status=status.strip().upper())
|
||||
|
||||
|
||||
def fetch_users(http: Optional[httpx.Client] = None) -> "list[OktaUser]":
|
||||
"""Return every user Okta lists, as OktaUser values. Raises OktaSyncError on
|
||||
any problem; never returns an empty list. `http` is for tests (a client built
|
||||
on httpx.MockTransport). In production it is None and a short-lived client is
|
||||
made here."""
|
||||
token = _token()
|
||||
base = org_url()
|
||||
if not token:
|
||||
raise _fail("OKTA_API_TOKEN is not set")
|
||||
if not base.startswith("https://"):
|
||||
raise _fail("the org URL is missing or is not https")
|
||||
attribute = _profile_attribute()
|
||||
|
||||
own_client = http is None
|
||||
client = http if http is not None else httpx.Client(timeout=TIMEOUT_SECONDS)
|
||||
headers = {"Accept": "application/json", "Authorization": f"SSWS {token}"}
|
||||
url: Optional[str] = f"{base}/api/v1/users"
|
||||
params: Optional[dict] = {"limit": PAGE_LIMIT}
|
||||
users: "list[OktaUser]" = []
|
||||
started = time.monotonic()
|
||||
try:
|
||||
for _ in range(MAX_PAGES):
|
||||
if not url:
|
||||
break
|
||||
if time.monotonic() - started > TOTAL_DEADLINE_SECONDS:
|
||||
raise _fail(f"took longer than {int(TOTAL_DEADLINE_SECONDS)} seconds in total")
|
||||
try:
|
||||
resp = client.get(url, params=params, headers=headers,
|
||||
follow_redirects=False)
|
||||
except httpx.HTTPError as exc:
|
||||
# Class name only. The exception text can carry the URL.
|
||||
raise _fail(f"network error ({type(exc).__name__})") from None
|
||||
if resp.status_code != 200:
|
||||
raise _fail(f"Okta answered HTTP {resp.status_code}")
|
||||
try:
|
||||
body = resp.json()
|
||||
except ValueError:
|
||||
raise _fail("the response is not JSON") from None
|
||||
if not isinstance(body, list):
|
||||
raise _fail("the response is not a list of users")
|
||||
users.extend(_parse_user(item, attribute) for item in body)
|
||||
|
||||
nxt = resp.links.get("next", {}).get("url")
|
||||
if not nxt:
|
||||
url = None
|
||||
break
|
||||
if not _same_origin(nxt, base):
|
||||
# Never send the token to a host that the Link header names.
|
||||
raise _fail("a next-page link points to a different host")
|
||||
url, params = nxt, None # the next link already carries its query
|
||||
else:
|
||||
# The loop used every page and the last one still named a next page.
|
||||
raise _fail(f"more than {MAX_PAGES} pages; the list may be cut off")
|
||||
finally:
|
||||
if own_client:
|
||||
client.close()
|
||||
|
||||
if not users:
|
||||
raise _fail("Okta returned no users, which is not credible for a real org")
|
||||
log.info("Okta user list fetched: %d user(s)", len(users))
|
||||
return users
|
||||
|
||||
|
||||
# ── T13.3: the sync job ───────────────────────────────────────────────────────
|
||||
# Which Okta statuses mean "this person is gone or was blocked on purpose".
|
||||
# Decided by Matt, 2026-10-16: a literal "anything that is not ACTIVE" would also
|
||||
# catch PASSWORD_EXPIRED, LOCKED_OUT and RECOVERY, which people leave on their
|
||||
# own, and because this job never re-enables, one AD password expiry would lock a
|
||||
# person out of this app until an admin stepped in. A user who is MISSING from the
|
||||
# list disables too (Okta's default list leaves out DEPROVISIONED users).
|
||||
DISABLING_STATUSES = frozenset({"SUSPENDED", "DEPROVISIONED"})
|
||||
|
||||
SYNC_ACTOR = "system:okta_sync"
|
||||
SYNC_ACTION = "user_deprovisioned_by_sync"
|
||||
|
||||
|
||||
@dataclass
|
||||
class SyncResult:
|
||||
ok: bool # False = the run took NO action at all
|
||||
error: str = "" # why, when ok is False
|
||||
okta_users: int = 0 # size of the Okta list
|
||||
checked: int = 0 # active local accounts looked at
|
||||
disabled: "list[str]" = field(default_factory=list) # usernames disabled this run
|
||||
|
||||
|
||||
def _gen_id(prefix: str) -> str:
|
||||
# Same shape as server/app.py gen_id. Not imported: app.py will import this
|
||||
# module (T13.4), and the reverse import would be circular.
|
||||
return f"{prefix}_{uuid.uuid4().hex[:12]}"
|
||||
|
||||
|
||||
def run_sync(db: Session, fetch: Optional[Callable[[], "list[OktaUser]"]] = None) -> SyncResult:
|
||||
"""Compare the Okta user list to local accounts and disable the ones Okta no
|
||||
longer vouches for.
|
||||
|
||||
The rules, none of which may be relaxed (D18):
|
||||
|
||||
1. FAIL CLOSED. If fetching raises ANY exception, or the answer cannot be
|
||||
trusted, this function changes nothing and writes nothing. A failure is
|
||||
never read as "nobody is active". The fetch happens before the database
|
||||
is read, so there is no code path from a failed fetch to a change.
|
||||
2. ONLY DISABLE. It sets is_active to False. No input makes it set True.
|
||||
An account that is already disabled is not touched and gets no new
|
||||
audit row, so a run that finds nothing new writes nothing.
|
||||
3. One AuditLog row per disabled account, actor "system:okta_sync", so it
|
||||
is never confused with an admin's action. Rows and changes are saved in
|
||||
one transaction. If saving fails, none of it is kept.
|
||||
4. Only changes are logged. A person who is still active in Okta and still
|
||||
active here produces no row.
|
||||
|
||||
One guard beyond the rules above, inside decision 4's "anything the sync can't
|
||||
confidently parse": if there are active local accounts and NONE of them
|
||||
matches any Okta user, the identity attribute is almost certainly wrong (see
|
||||
OKTA_SYNC_PROFILE_ATTRIBUTE). That is a setup mistake, not an offboarding of
|
||||
the whole company, so the run is refused and nothing changes.
|
||||
|
||||
A local account matches an Okta user by username or by email, ignoring case,
|
||||
the same two fields auth.find_user() uses. If the Okta list holds the same
|
||||
identity twice with different statuses, the account is disabled only when
|
||||
every one of them is a disabling status.
|
||||
|
||||
`fetch` is for tests. In production it is None and fetch_users() is used.
|
||||
"""
|
||||
fetch = fetch or fetch_users
|
||||
|
||||
# 1. The fetch. Nothing below runs unless it returned a list.
|
||||
try:
|
||||
okta_users = fetch()
|
||||
except Exception as exc: # an error of ANY kind, not only OktaSyncError
|
||||
msg = str(exc) if isinstance(exc, OktaSyncError) else f"unexpected {type(exc).__name__}"
|
||||
log.error("Okta sync SKIPPED, no account was changed: %s", msg)
|
||||
return SyncResult(ok=False, error=msg)
|
||||
if not okta_users:
|
||||
# fetch_users() already raises on an empty list. This covers a stand-in
|
||||
# fetch (and any future change) that returns one: same rule.
|
||||
msg = "the fetch returned no users"
|
||||
log.error("Okta sync SKIPPED, no account was changed: %s", msg)
|
||||
return SyncResult(ok=False, error=msg)
|
||||
|
||||
try:
|
||||
by_identity: "dict[str, list[str]]" = {}
|
||||
for u in okta_users:
|
||||
by_identity.setdefault(u.identity.strip().lower(), []).append(u.status.strip().upper())
|
||||
|
||||
locals_ = list(db.scalars(select(models.User).where(models.User.is_active.is_(True))))
|
||||
plan: "list[tuple[models.User, str]]" = []
|
||||
matched = 0
|
||||
for local in locals_:
|
||||
statuses: "list[str]" = []
|
||||
for key in {(local.username or "").strip().lower(), (local.email or "").strip().lower()}:
|
||||
if key:
|
||||
statuses.extend(by_identity.get(key, []))
|
||||
if statuses:
|
||||
matched += 1
|
||||
if all(st in DISABLING_STATUSES for st in statuses):
|
||||
plan.append((local, sorted(set(statuses))[0]))
|
||||
else:
|
||||
plan.append((local, "MISSING"))
|
||||
|
||||
if locals_ and matched == 0:
|
||||
msg = (f"none of {len(locals_)} active local account(s) matches any of "
|
||||
f"{len(okta_users)} Okta user(s); the identity attribute is probably wrong")
|
||||
log.error("Okta sync SKIPPED, no account was changed: %s", msg)
|
||||
return SyncResult(ok=False, error=msg, okta_users=len(okta_users), checked=len(locals_))
|
||||
|
||||
planned = [(local.username, status) for local, status in plan]
|
||||
checked = len(locals_)
|
||||
for local, status in plan:
|
||||
local.is_active = False
|
||||
db.add(models.AuditLog(
|
||||
id=_gen_id("ev"), actor=SYNC_ACTOR, action=SYNC_ACTION,
|
||||
entity_type="user", entity_id=local.id, project_id=None,
|
||||
summary=local.username[:400],
|
||||
detail={"okta_status": status, "via": "okta_sync"},
|
||||
))
|
||||
db.commit()
|
||||
except Exception as exc:
|
||||
db.rollback()
|
||||
log.error("Okta sync FAILED, nothing was kept: unexpected %s", type(exc).__name__)
|
||||
return SyncResult(ok=False, error=f"unexpected {type(exc).__name__}")
|
||||
|
||||
for name, status in planned:
|
||||
log.warning("Okta sync disabled %r (Okta status: %s)", name, status)
|
||||
log.info("Okta sync done: %d Okta user(s), %d active local account(s) checked, %d disabled",
|
||||
len(okta_users), checked, len(planned))
|
||||
return SyncResult(ok=True, okta_users=len(okta_users), checked=checked,
|
||||
disabled=[name for name, _ in planned])
|
||||
|
||||
|
||||
# ── T13.4: running it on a schedule ───────────────────────────────────────────
|
||||
# The api container runs gunicorn with 2 workers, and a later deploy may run more
|
||||
# than one container. Each worker starts its own timer thread below. Without a
|
||||
# gate, all of them would call Okta every cycle and write overlapping changes.
|
||||
# The gate is one row in app_settings (key "okta_sync"). Its updated_at is "when
|
||||
# the last run was claimed". A worker claims a run with ONE atomic conditional
|
||||
# UPDATE: it succeeds only if the last claim is older than the interval. Exactly
|
||||
# one worker wins each cycle, on SQLite and on Postgres, with no lock held.
|
||||
#
|
||||
# The row survives a restart, so a restart inside the interval does not start a
|
||||
# second run. A slow run cannot overlap the next: a fetch is bounded to
|
||||
# TOTAL_DEADLINE_SECONDS (120) and the interval can never be set below
|
||||
# INTERVAL_MIN_SECONDS (300). Inside one process the timer thread runs one cycle at
|
||||
# a time, so cycles cannot stack there either.
|
||||
CLAIM_KEY = "okta_sync"
|
||||
INTERVAL_DEFAULT_SECONDS = 900 # 15 minutes (D18, proposed, not confirmed)
|
||||
INTERVAL_MIN_SECONDS = 300
|
||||
TICK_SECONDS = 60 # how often each worker checks if a run is due
|
||||
START_DELAY_SECONDS = 10 # let the app finish starting first
|
||||
|
||||
|
||||
def interval_seconds() -> int:
|
||||
"""OKTA_SYNC_INTERVAL_SECONDS, default 900. A value that is not a number uses
|
||||
the default. A value under 300 is raised to 300 (see above for why)."""
|
||||
raw = os.getenv("OKTA_SYNC_INTERVAL_SECONDS", "").strip()
|
||||
if not raw:
|
||||
return INTERVAL_DEFAULT_SECONDS
|
||||
try:
|
||||
value = int(raw)
|
||||
except ValueError:
|
||||
log.warning("OKTA_SYNC_INTERVAL_SECONDS=%r is not a whole number; using %d",
|
||||
raw, INTERVAL_DEFAULT_SECONDS)
|
||||
return INTERVAL_DEFAULT_SECONDS
|
||||
if value < INTERVAL_MIN_SECONDS:
|
||||
log.warning("OKTA_SYNC_INTERVAL_SECONDS=%d is under the minimum; using %d",
|
||||
value, INTERVAL_MIN_SECONDS)
|
||||
return INTERVAL_MIN_SECONDS
|
||||
return value
|
||||
|
||||
|
||||
def claim(db: Session, interval: int, now: Optional[datetime] = None) -> bool:
|
||||
"""Try to claim this cycle. True for exactly one caller per interval."""
|
||||
now = now or datetime.now(timezone.utc)
|
||||
cutoff = now - timedelta(seconds=interval)
|
||||
try:
|
||||
row = db.get(models.AppSetting, CLAIM_KEY)
|
||||
if row is None:
|
||||
db.add(models.AppSetting(key=CLAIM_KEY, value={}, updated_at=now))
|
||||
db.commit() # the insert is the claim
|
||||
return True
|
||||
except IntegrityError:
|
||||
db.rollback() # another worker inserted first: it won
|
||||
return False
|
||||
db.rollback() # end the read; the UPDATE starts fresh
|
||||
result = db.execute(
|
||||
update(models.AppSetting)
|
||||
.where(models.AppSetting.key == CLAIM_KEY, models.AppSetting.updated_at <= cutoff)
|
||||
.values(updated_at=now)
|
||||
)
|
||||
db.commit()
|
||||
return result.rowcount == 1
|
||||
|
||||
|
||||
def run_if_due(session_factory: Callable[[], Session],
|
||||
fetch: Optional[Callable[[], "list[OktaUser]"]] = None,
|
||||
now: Optional[datetime] = None) -> Optional[SyncResult]:
|
||||
"""One timer tick: claim the cycle, and if it is ours, run the sync. Returns
|
||||
the SyncResult, or None when the cycle was not ours (or could not be claimed).
|
||||
Nothing here can turn a failure into a change: run_sync() owns that rule, and
|
||||
anything that goes wrong around it is logged and ends the tick."""
|
||||
try:
|
||||
with session_factory() as db:
|
||||
if not claim(db, interval_seconds(), now=now):
|
||||
return None
|
||||
with session_factory() as db:
|
||||
return run_sync(db, fetch=fetch)
|
||||
except Exception as exc:
|
||||
log.error("Okta sync tick failed, no account was changed: unexpected %s", type(exc).__name__)
|
||||
return None
|
||||
|
||||
|
||||
def start_scheduler(session_factory: Callable[[], Session]) -> "Optional[threading.Event]":
|
||||
"""Start the timer thread. Returns a stop Event (set it to end the thread), or
|
||||
None when the sync is off because OKTA_API_TOKEN or the org URL is missing.
|
||||
Logs one line either way, so an operator can see it in `docker compose logs api`."""
|
||||
log.info(describe())
|
||||
if not is_configured():
|
||||
return None
|
||||
stop = threading.Event()
|
||||
|
||||
def loop():
|
||||
if stop.wait(START_DELAY_SECONDS):
|
||||
return
|
||||
while not stop.is_set():
|
||||
run_if_due(session_factory)
|
||||
if stop.wait(TICK_SECONDS):
|
||||
return
|
||||
|
||||
threading.Thread(target=loop, name="okta-sync", daemon=True).start()
|
||||
log.info("Okta sync scheduled: every %d seconds (checked each %d s by every worker; "
|
||||
"one wins each cycle)", interval_seconds(), TICK_SECONDS)
|
||||
return stop
|
||||
@@ -254,27 +254,23 @@ def main():
|
||||
if ("el.textContent = '" in ln or "el.innerHTML = '" in ln)
|
||||
and "the project" not in ln and ("✓" in ln or "⚠" in ln or "↻" in ln or "✕" in ln)])
|
||||
|
||||
print("\n7. the analytics dwell listener still fires")
|
||||
print("\n7. autosave's own registration survives the retired analytics tracker")
|
||||
# This section used to also assert the per-browser usage-analytics dwell
|
||||
# listener (wp-usage.js) still fired alongside autosave, to catch autosave
|
||||
# accidentally replacing it. CR-019/T11.6 (2026-09-23) retired that tracker
|
||||
# on purpose - track() is now a documented no-op and wp-usage.js is gone -
|
||||
# so asserting it still records an event would be asserting the wrong
|
||||
# thing. What's still worth checking here is autosave's own registration,
|
||||
# which is what this section actually guards.
|
||||
boot(page, base, tok, "/work-package-suite.html?project=projA&tab=sop")
|
||||
time.sleep(0.8)
|
||||
chk("the wizard still has its analytics dwell tracker",
|
||||
page.eval("typeof trackStepDwell === 'function'"))
|
||||
chk("...and the wizard registered autosave too",
|
||||
chk("the wizard registered autosave",
|
||||
page.eval("typeof WPAutosave === 'object' && typeof sopDraftId === 'function'"))
|
||||
before = page.eval("""(() => {
|
||||
try { return (JSON.parse(localStorage.getItem('wp_suite_analytics_v1')||'{}').events||[]).length; }
|
||||
catch(e){ return -1; }
|
||||
})()""")
|
||||
# trackStepDwell only records a dwell longer than 400ms, so give it one.
|
||||
time.sleep(0.9)
|
||||
page.eval("typeof trackStepDwell==='function' && trackStepDwell()")
|
||||
time.sleep(0.4)
|
||||
after_n = page.eval("""(() => {
|
||||
try { return (JSON.parse(localStorage.getItem('wp_suite_analytics_v1')||'{}').events||[]).length; }
|
||||
catch(e){ return -1; }
|
||||
})()""")
|
||||
chk("calling it records an event (it was not replaced by the guard)",
|
||||
after_n > before, "%s -> %s" % (before, after_n))
|
||||
chk("...and the retired dwell tracker is a harmless no-op, not missing or throwing",
|
||||
page.eval("""(() => {
|
||||
if (typeof trackStepDwell !== 'function') return false;
|
||||
try { trackStepDwell(); return true; } catch (e) { return false; }
|
||||
})()"""))
|
||||
finally:
|
||||
page.close()
|
||||
browser.close()
|
||||
|
||||
252
tests/okta_sync_check.py
Normal file
@@ -0,0 +1,252 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Does the Okta Management API client keep its promises? - D18 / T13.2.
|
||||
|
||||
The one promise that matters: a failure is never an empty list. T13.3 reads an
|
||||
empty list as "nobody is active", so a client that returned [] when Okta was
|
||||
down would disable the whole org. Every failure below must raise OktaSyncError.
|
||||
|
||||
Also checked: the client uses its own credential (never the OIDC client secret),
|
||||
follows pagination, never sends the token to another host, never follows a
|
||||
redirect, and never lets the token reach a log line, an error message, or
|
||||
describe().
|
||||
|
||||
Self-contained: no server, no database, no network, no browser. httpx's
|
||||
MockTransport stands in for Okta.
|
||||
Exit 0 all passed, 1 a failure.
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
|
||||
import httpx # noqa: E402
|
||||
|
||||
from server import okta_auth, okta_sync # noqa: E402
|
||||
|
||||
TOKEN = "00tok-SECRET-api-token-xyz"
|
||||
CLIENT_SECRET = "oidc-client-SECRET-abc"
|
||||
ORG = "https://example.okta.com"
|
||||
|
||||
fails = 0
|
||||
|
||||
|
||||
def chk(label, ok, extra=""):
|
||||
global fails
|
||||
print(("PASS " if ok else "FAIL ") + label + (f" [{extra}]" if (extra and not ok) else ""))
|
||||
if not ok:
|
||||
fails += 1
|
||||
|
||||
|
||||
class Capture(logging.Handler):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.lines = []
|
||||
|
||||
def emit(self, record):
|
||||
self.lines.append(record.getMessage())
|
||||
|
||||
|
||||
cap = Capture()
|
||||
logging.getLogger("wpsuite").addHandler(cap)
|
||||
logging.getLogger("wpsuite").setLevel(logging.DEBUG)
|
||||
|
||||
|
||||
def reset_env():
|
||||
os.environ["OKTA_API_TOKEN"] = TOKEN
|
||||
os.environ["OKTA_CLIENT_SECRET"] = CLIENT_SECRET
|
||||
os.environ.pop("OKTA_ORG_URL", None)
|
||||
os.environ.pop("OKTA_SYNC_PROFILE_ATTRIBUTE", None)
|
||||
okta_auth.ISSUER = f"{ORG}/oauth2/default"
|
||||
okta_auth.IDENTITY_CLAIM = "preferred_username"
|
||||
|
||||
|
||||
def user(login, status="ACTIVE", **profile):
|
||||
p = {"login": login, "email": login}
|
||||
p.update(profile)
|
||||
return {"id": "00u" + login, "status": status, "profile": p}
|
||||
|
||||
|
||||
def client(handler):
|
||||
return httpx.Client(transport=httpx.MockTransport(handler))
|
||||
|
||||
|
||||
def raises(handler, label, **kw):
|
||||
"""Run fetch_users against `handler`; it must raise OktaSyncError, and the
|
||||
message must not hold the token."""
|
||||
reset_env()
|
||||
try:
|
||||
out = okta_sync.fetch_users(http=client(handler))
|
||||
except okta_sync.OktaSyncError as exc:
|
||||
msg = str(exc)
|
||||
clean = TOKEN not in msg and CLIENT_SECRET not in msg and "example.okta.com" not in msg
|
||||
chk(f"{label}: raises, message has no token or host", clean, msg)
|
||||
return msg
|
||||
except Exception as exc: # the wrong exception type
|
||||
chk(f"{label}: raises OktaSyncError", False, f"{type(exc).__name__}: {exc}")
|
||||
return ""
|
||||
chk(f"{label}: raises (did NOT raise, returned {out!r})", False)
|
||||
return ""
|
||||
|
||||
|
||||
def ok(body):
|
||||
return lambda req: httpx.Response(200, json=body)
|
||||
|
||||
|
||||
# ── 1. a good fetch ───────────────────────────────────────────────────────────
|
||||
print("\n1. a good fetch")
|
||||
reset_env()
|
||||
seen = []
|
||||
|
||||
|
||||
def good(req):
|
||||
seen.append(req)
|
||||
return httpx.Response(200, json=[user("alice", "ACTIVE"), user("bob", "suspended")])
|
||||
|
||||
|
||||
users = okta_sync.fetch_users(http=client(good))
|
||||
chk("returns every user", [u.identity for u in users] == ["alice", "bob"])
|
||||
chk("status is upper case", [u.status for u in users] == ["ACTIVE", "SUSPENDED"])
|
||||
chk("asks the users endpoint of the org", str(seen[0].url).startswith(f"{ORG}/api/v1/users"))
|
||||
chk("asks for the page maximum", seen[0].url.params.get("limit") == "200")
|
||||
chk("authenticates with the API token (SSWS)", seen[0].headers.get("authorization") == f"SSWS {TOKEN}")
|
||||
chk("never sends the OIDC client secret", CLIENT_SECRET not in str(seen[0].headers))
|
||||
|
||||
# ── 2. pagination ─────────────────────────────────────────────────────────────
|
||||
print("\n2. pagination")
|
||||
reset_env()
|
||||
calls = []
|
||||
|
||||
|
||||
def paged(req):
|
||||
calls.append(req)
|
||||
if "after=p2" not in str(req.url):
|
||||
return httpx.Response(200, json=[user("alice")],
|
||||
headers={"Link": f'<{ORG}/api/v1/users?limit=200&after=p2>; rel="next"'})
|
||||
return httpx.Response(200, json=[user("bob")])
|
||||
|
||||
|
||||
users = okta_sync.fetch_users(http=client(paged))
|
||||
chk("follows the next link and returns both pages", [u.identity for u in users] == ["alice", "bob"])
|
||||
chk("two requests, no more", len(calls) == 2)
|
||||
chk("page 2 also carries the token", calls[1].headers.get("authorization") == f"SSWS {TOKEN}")
|
||||
|
||||
# ── 3. the failure rule: an error is never an empty list ──────────────────────
|
||||
print("\n3. every failure raises (never returns an empty list)")
|
||||
for code in (401, 403, 404, 429, 500, 502, 503):
|
||||
raises(lambda req, c=code: httpx.Response(c, json={"errorCode": "x"}), f"HTTP {code}")
|
||||
raises(lambda req: httpx.Response(302, headers={"Location": "https://evil.example/"}), "a redirect (never followed)")
|
||||
raises(lambda req: (_ for _ in ()).throw(httpx.ConnectError("boom to https://example.okta.com/api/v1/users")), "connection error")
|
||||
raises(lambda req: (_ for _ in ()).throw(httpx.ReadTimeout("slow")), "timeout")
|
||||
raises(lambda req: httpx.Response(200, text="<html>not json</html>"), "body is not JSON")
|
||||
raises(ok({"errorCode": "E0000011"}), "body is an object, not a list")
|
||||
raises(ok("a string"), "body is a string")
|
||||
raises(ok([]), "an empty list (not credible for a real org)")
|
||||
raises(ok(["alice"]), "an item that is not an object")
|
||||
raises(ok([{"profile": {"login": "a"}}]), "an item with no status")
|
||||
raises(ok([{"status": "ACTIVE"}]), "an item with no profile")
|
||||
raises(ok([{"status": "ACTIVE", "profile": {}}]), "an item with no login")
|
||||
raises(ok([{"status": "ACTIVE", "profile": {"login": " "}}]), "an item with a blank login")
|
||||
raises(ok([user("alice"), {"status": "ACTIVE", "profile": {}}]), "one bad item among good ones (never skipped)")
|
||||
|
||||
# ── 4. misconfiguration raises ────────────────────────────────────────────────
|
||||
print("\n4. misconfiguration")
|
||||
reset_env()
|
||||
os.environ["OKTA_API_TOKEN"] = ""
|
||||
try:
|
||||
okta_sync.fetch_users(http=client(good))
|
||||
chk("no token raises", False)
|
||||
except okta_sync.OktaSyncError:
|
||||
chk("no token raises", True)
|
||||
reset_env()
|
||||
okta_auth.ISSUER = ""
|
||||
try:
|
||||
okta_sync.fetch_users(http=client(good))
|
||||
chk("no org URL raises", False)
|
||||
except okta_sync.OktaSyncError:
|
||||
chk("no org URL raises", True)
|
||||
reset_env()
|
||||
os.environ["OKTA_ORG_URL"] = "http://example.okta.com"
|
||||
sent = []
|
||||
try:
|
||||
okta_sync.fetch_users(http=client(lambda r: sent.append(r) or httpx.Response(200, json=[user("a")])))
|
||||
chk("an http org URL raises", False)
|
||||
except okta_sync.OktaSyncError:
|
||||
chk("an http org URL raises", True)
|
||||
chk("...and no request was made (token never sent over http)", sent == [])
|
||||
|
||||
# ── 5. never leaks the token to another host, never loops forever ─────────────
|
||||
print("\n5. next-link safety")
|
||||
reset_env()
|
||||
hosts = []
|
||||
|
||||
|
||||
def evil_link(req):
|
||||
hosts.append(req.url.host)
|
||||
return httpx.Response(200, json=[user("alice")],
|
||||
headers={"Link": '<https://evil.example/steal>; rel="next"'})
|
||||
|
||||
|
||||
raises(evil_link, "a next link to another host")
|
||||
chk("...the other host was never contacted", hosts == ["example.okta.com"])
|
||||
|
||||
count = []
|
||||
|
||||
|
||||
def endless(req):
|
||||
count.append(1)
|
||||
return httpx.Response(200, json=[user("alice")],
|
||||
headers={"Link": f'<{ORG}/api/v1/users?after={len(count)}>; rel="next"'})
|
||||
|
||||
|
||||
raises(endless, "an endless next chain (page cap)")
|
||||
chk(f"...stops at the cap ({okta_sync.MAX_PAGES} requests)", len(count) == okta_sync.MAX_PAGES, str(len(count)))
|
||||
|
||||
# ── 6. org URL and identity attribute ─────────────────────────────────────────
|
||||
print("\n6. org URL and identity attribute")
|
||||
reset_env()
|
||||
chk("org URL is the issuer's scheme and host", okta_sync.org_url() == ORG)
|
||||
os.environ["OKTA_ORG_URL"] = "https://corp.okta.com/"
|
||||
chk("OKTA_ORG_URL wins, trailing slash removed", okta_sync.org_url() == "https://corp.okta.com")
|
||||
reset_env()
|
||||
okta_auth.IDENTITY_CLAIM = "email"
|
||||
body = [{"status": "ACTIVE", "profile": {"login": "alice", "email": "alice@x.com"}}]
|
||||
chk("claim 'email' reads profile.email", okta_sync.fetch_users(http=client(ok(body)))[0].identity == "alice@x.com")
|
||||
okta_auth.IDENTITY_CLAIM = "sAMAccountName"
|
||||
body = [{"status": "ACTIVE", "profile": {"login": "alice@x.com", "sAMAccountName": "alice"}}]
|
||||
chk("a custom claim reads profile[<claim>]", okta_sync.fetch_users(http=client(ok(body)))[0].identity == "alice")
|
||||
os.environ["OKTA_SYNC_PROFILE_ATTRIBUTE"] = "login"
|
||||
chk("OKTA_SYNC_PROFILE_ATTRIBUTE overrides", okta_sync.fetch_users(http=client(ok(body)))[0].identity == "alice@x.com")
|
||||
|
||||
# ── 7. the token never leaks ──────────────────────────────────────────────────
|
||||
print("\n7. the token never leaks")
|
||||
reset_env()
|
||||
chk("describe() has no token", TOKEN not in okta_sync.describe() and "configured" in okta_sync.describe())
|
||||
os.environ["OKTA_API_TOKEN"] = ""
|
||||
chk("describe() says DISABLED without a token", "DISABLED" in okta_sync.describe())
|
||||
reset_env()
|
||||
every_log = " ".join(cap.lines)
|
||||
chk("no log line holds the API token", TOKEN not in every_log)
|
||||
chk("no log line holds the client secret", CLIENT_SECRET not in every_log)
|
||||
|
||||
# ── 8. the whole fetch is time-bounded (T13.4 relies on it) ──────────────────
|
||||
print("\n8. the whole fetch is time-bounded")
|
||||
reset_env()
|
||||
saved = okta_sync.TOTAL_DEADLINE_SECONDS
|
||||
okta_sync.TOTAL_DEADLINE_SECONDS = 0.0
|
||||
|
||||
|
||||
def slow_pages(req):
|
||||
return httpx.Response(200, json=[user("alice")],
|
||||
headers={"Link": f'<{ORG}/api/v1/users?after=x>; rel="next"'})
|
||||
|
||||
|
||||
raises(slow_pages, "a fetch that runs past the total deadline")
|
||||
okta_sync.TOTAL_DEADLINE_SECONDS = saved
|
||||
chk("the real deadline is under the shortest allowed interval",
|
||||
okta_sync.TOTAL_DEADLINE_SECONDS < okta_sync.INTERVAL_MIN_SECONDS)
|
||||
|
||||
print()
|
||||
print("ALL PASSED" if not fails else f"{fails} FAILED")
|
||||
sys.exit(0 if not fails else 1)
|
||||
218
tests/okta_sync_job_check.py
Normal file
@@ -0,0 +1,218 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Does the Okta deprovisioning sync fail closed? - D18 / T13.3.
|
||||
|
||||
The rule that must never be skipped or weakened: when Okta cannot be asked, or
|
||||
its answer cannot be trusted, the sync changes NO account and writes NO audit
|
||||
row. Treating "could not reach Okta" as "nobody is active" would lock the whole
|
||||
company out on one bad afternoon.
|
||||
|
||||
Also checked: the sync only ever disables (it never re-enables, under any
|
||||
input), writes one audit row per change with a non-human actor, leaves an
|
||||
already-disabled account alone, logs only changes, and keeps temporary Okta
|
||||
states (PASSWORD_EXPIRED, LOCKED_OUT, RECOVERY) from disabling anyone.
|
||||
|
||||
Self-contained: its own in-memory SQLite, no server, no network, no browser.
|
||||
Exit 0 all passed, 1 a failure.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
|
||||
import httpx # noqa: E402
|
||||
from sqlalchemy import create_engine, select # noqa: E402
|
||||
from sqlalchemy.orm import sessionmaker # noqa: E402
|
||||
from sqlalchemy.pool import StaticPool # noqa: E402
|
||||
|
||||
from server import models, okta_auth, okta_sync # noqa: E402
|
||||
from server.okta_sync import OktaSyncError, OktaUser, run_sync # noqa: E402
|
||||
|
||||
fails = 0
|
||||
|
||||
|
||||
def chk(label, ok, extra=""):
|
||||
global fails
|
||||
print(("PASS " if ok else "FAIL ") + label + (f" [{extra}]" if (extra and not ok) else ""))
|
||||
if not ok:
|
||||
fails += 1
|
||||
|
||||
|
||||
engine = create_engine("sqlite://", connect_args={"check_same_thread": False}, poolclass=StaticPool)
|
||||
models.Base.metadata.create_all(engine)
|
||||
Session = sessionmaker(bind=engine, autoflush=False, future=True)
|
||||
|
||||
|
||||
def seed(rows):
|
||||
"""rows: list of (username, is_active[, email]). Wipes users and audit rows first."""
|
||||
with Session() as db:
|
||||
db.query(models.AuditLog).delete()
|
||||
db.query(models.User).delete()
|
||||
for r in rows:
|
||||
name, active = r[0], r[1]
|
||||
db.add(models.User(id="u_" + name, username=name, is_active=active,
|
||||
email=(r[2] if len(r) > 2 else "")))
|
||||
db.commit()
|
||||
|
||||
|
||||
def state():
|
||||
with Session() as db:
|
||||
users = {u.username: u.is_active for u in db.scalars(select(models.User))}
|
||||
audits = [a.to_dict() for a in db.scalars(select(models.AuditLog))]
|
||||
return users, audits
|
||||
|
||||
|
||||
def fixed(*pairs):
|
||||
return lambda: [OktaUser(identity=i, status=s) for i, s in pairs]
|
||||
|
||||
|
||||
# ── 1. the failure rule ───────────────────────────────────────────────────────
|
||||
print("\n1. FAIL CLOSED: a failed or untrusted fetch changes nothing")
|
||||
BASE = [("alice", True), ("bob", True), ("carol", True)]
|
||||
|
||||
|
||||
def boom(exc):
|
||||
def f():
|
||||
raise exc
|
||||
return f
|
||||
|
||||
|
||||
failing = [
|
||||
("OktaSyncError (auth failure)", boom(OktaSyncError("HTTP 401"))),
|
||||
("RuntimeError", boom(RuntimeError("boom"))),
|
||||
("ConnectionError", boom(ConnectionError("down"))),
|
||||
("TimeoutError", boom(TimeoutError("slow"))),
|
||||
("ValueError", boom(ValueError("bad"))),
|
||||
("KeyError", boom(KeyError("x"))),
|
||||
("fetch returns an empty list", lambda: []),
|
||||
("fetch returns None", lambda: None),
|
||||
("fetch returns garbage (strings)", lambda: ["alice", "bob"]),
|
||||
("fetch returns garbage (dicts)", lambda: [{"identity": "alice"}]),
|
||||
]
|
||||
for label, fetch in failing:
|
||||
seed(BASE)
|
||||
before = state()
|
||||
with Session() as db:
|
||||
res = run_sync(db, fetch=fetch)
|
||||
after = state()
|
||||
chk(f"{label}: ok is False", res.ok is False)
|
||||
chk(f"{label}: zero accounts changed", after[0] == before[0] == {"alice": True, "bob": True, "carol": True})
|
||||
chk(f"{label}: zero audit rows", after[1] == [] and res.disabled == [])
|
||||
|
||||
print("\n ...and through the real client, not a stand-in")
|
||||
okta_auth.ISSUER = "https://example.okta.com/oauth2/default"
|
||||
os.environ["OKTA_API_TOKEN"] = "tok"
|
||||
for code in (401, 403, 429, 500):
|
||||
seed(BASE)
|
||||
c = httpx.Client(transport=httpx.MockTransport(lambda r, k=code: httpx.Response(k, json={})))
|
||||
with Session() as db:
|
||||
res = run_sync(db, fetch=lambda c=c: okta_sync.fetch_users(http=c))
|
||||
u, a = state()
|
||||
chk(f"real client, HTTP {code}: nothing changed, no audit rows",
|
||||
(not res.ok) and all(u.values()) and a == [])
|
||||
seed(BASE)
|
||||
os.environ["OKTA_API_TOKEN"] = ""
|
||||
with Session() as db:
|
||||
res = run_sync(db) # fetch=None: the default path, no token set
|
||||
u, a = state()
|
||||
chk("default fetch with no token: nothing changed, no audit rows", (not res.ok) and all(u.values()) and a == [])
|
||||
|
||||
print("\n ...and when saving fails")
|
||||
seed(BASE)
|
||||
with Session() as db:
|
||||
real_commit = db.commit
|
||||
|
||||
def bad_commit():
|
||||
raise RuntimeError("disk full")
|
||||
|
||||
db.commit = bad_commit
|
||||
res = run_sync(db, fetch=fixed(("alice", "ACTIVE"))) # bob and carol would be disabled
|
||||
u, a = state()
|
||||
chk("a failed commit keeps no change and no audit row", (not res.ok) and all(u.values()) and a == [])
|
||||
|
||||
print("\n ...and when no local account matches any Okta user (wrong identity attribute)")
|
||||
seed(BASE)
|
||||
with Session() as db:
|
||||
res = run_sync(db, fetch=fixed(("zed@x.com", "ACTIVE"), ("yan@x.com", "ACTIVE")))
|
||||
u, a = state()
|
||||
chk("zero matches is refused, not read as 'everyone left'", (not res.ok) and all(u.values()) and a == [])
|
||||
chk("...and the error says why", "identity attribute" in res.error)
|
||||
|
||||
# ── 2. what it does disable ───────────────────────────────────────────────────
|
||||
print("\n2. a good run disables the right accounts")
|
||||
seed([("alice", True), ("bob", True), ("carol", True), ("dave", True),
|
||||
("erin", True), ("frank", True), ("gina", True), ("hank", True)])
|
||||
okta = fixed(("alice", "ACTIVE"), ("bob", "SUSPENDED"),
|
||||
# carol is MISSING from the list
|
||||
("dave", "DEPROVISIONED"), ("erin", "PASSWORD_EXPIRED"),
|
||||
("frank", "LOCKED_OUT"), ("gina", "RECOVERY"), ("hank", "ACTIVE"))
|
||||
with Session() as db:
|
||||
res = run_sync(db, fetch=okta)
|
||||
users, audits = state()
|
||||
chk("run is ok", res.ok, res.error)
|
||||
chk("missing, SUSPENDED and DEPROVISIONED are disabled",
|
||||
sorted(res.disabled) == ["bob", "carol", "dave"] and
|
||||
[users[n] for n in ("bob", "carol", "dave")] == [False, False, False])
|
||||
chk("ACTIVE stays active", users["alice"] is True and users["hank"] is True)
|
||||
chk("PASSWORD_EXPIRED, LOCKED_OUT and RECOVERY do NOT disable (temporary states)",
|
||||
users["erin"] is True and users["frank"] is True and users["gina"] is True)
|
||||
chk("one audit row per disabled account", len(audits) == 3, str(len(audits)))
|
||||
row = next(a for a in audits if a["summary"] == "bob")
|
||||
chk("audit actor is the sync job, not a person", {a["actor"] for a in audits} == {"system:okta_sync"})
|
||||
chk("audit action is user_deprovisioned_by_sync", {a["action"] for a in audits} == {"user_deprovisioned_by_sync"})
|
||||
chk("audit row names the user and the Okta status found",
|
||||
row["entity_type"] == "user" and row["entity_id"] == "u_bob" and row["detail"].get("okta_status") == "SUSPENDED")
|
||||
chk("a missing user is recorded as MISSING",
|
||||
next(a for a in audits if a["summary"] == "carol")["detail"].get("okta_status") == "MISSING")
|
||||
|
||||
print("\n ...run it again: only changes are logged")
|
||||
with Session() as db:
|
||||
res2 = run_sync(db, fetch=okta)
|
||||
users2, audits2 = state()
|
||||
chk("second run is ok, disables nobody", res2.ok and res2.disabled == [])
|
||||
chk("second run writes no new audit rows (no duplicates)", len(audits2) == 3)
|
||||
chk("accounts unchanged by the second run", users2 == users)
|
||||
|
||||
# ── 3. never re-enables, never touches a disabled account ─────────────────────
|
||||
print("\n3. never re-enables, never touches an already-disabled account")
|
||||
seed([("alice", True), ("hank", False), ("gina", False)])
|
||||
with Session() as db:
|
||||
res = run_sync(db, fetch=fixed(("alice", "ACTIVE"), ("hank", "ACTIVE"))) # gina missing
|
||||
users, audits = state()
|
||||
chk("a disabled account that is ACTIVE in Okta stays disabled", users["hank"] is False)
|
||||
chk("a disabled account that is missing from Okta stays as it is", users["gina"] is False)
|
||||
chk("no audit row for either (nothing changed)", audits == [] and res.disabled == [])
|
||||
src = open(os.path.join(ROOT, "server", "okta_sync.py")).read()
|
||||
chk("no assignment of is_active to True anywhere in okta_sync.py",
|
||||
"is_active = True" not in src and "is_active=True" not in src.replace("is_active.is_(True)", ""))
|
||||
|
||||
# ── 4. matching ───────────────────────────────────────────────────────────────
|
||||
print("\n4. matching")
|
||||
seed([("ALICE", True), ("bob", True, "Bob@X.com"), ("carol", True)])
|
||||
with Session() as db:
|
||||
res = run_sync(db, fetch=fixed(("alice", "ACTIVE"), ("bob@x.com", "ACTIVE"), ("carol", "SUSPENDED")))
|
||||
users, _ = state()
|
||||
chk("username match ignores case", users["ALICE"] is True)
|
||||
chk("email match works, ignoring case", users["bob"] is True)
|
||||
chk("a matching SUSPENDED user is disabled", users["carol"] is False)
|
||||
|
||||
seed([("alice", True), ("bob", True), ("carol", True)])
|
||||
with Session() as db:
|
||||
run_sync(db, fetch=fixed(("alice", "ACTIVE"), ("bob", "ACTIVE"), ("bob", "SUSPENDED"),
|
||||
("carol", "SUSPENDED"), ("carol", "DEPROVISIONED")))
|
||||
users, _ = state()
|
||||
chk("an identity listed twice, one entry active: stays active", users["bob"] is True)
|
||||
chk("an identity listed twice, every entry disabling: disabled", users["carol"] is False)
|
||||
|
||||
# ── 5. the secret never reaches the audit row or the result ───────────────────
|
||||
print("\n5. no secret in what is stored or returned")
|
||||
seed(BASE)
|
||||
os.environ["OKTA_API_TOKEN"] = "SECRET-TOKEN-123"
|
||||
c = httpx.Client(transport=httpx.MockTransport(lambda r: httpx.Response(401, json={})))
|
||||
with Session() as db:
|
||||
res = run_sync(db, fetch=lambda: okta_sync.fetch_users(http=c))
|
||||
chk("the failure message has no token", "SECRET-TOKEN-123" not in res.error and "example.okta.com" not in res.error)
|
||||
|
||||
print()
|
||||
print("ALL PASSED" if not fails else f"{fails} FAILED")
|
||||
sys.exit(0 if not fails else 1)
|
||||
242
tests/okta_sync_schedule_check.py
Normal file
@@ -0,0 +1,242 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Does the Okta sync run on a schedule without doubling up? - D18 / T13.4.
|
||||
|
||||
The api container runs 2 gunicorn workers, and each starts its own timer. Checked
|
||||
here: exactly one of them wins each cycle (including a race on a real file
|
||||
database), a restart inside the interval does not start a second run, the
|
||||
interval is configurable and cannot go below the minimum, the timer thread starts
|
||||
and stops, nothing starts when the sync is not configured, and a failure inside
|
||||
the scheduled path still changes nothing.
|
||||
|
||||
Self-contained: a throwaway SQLite file, threads, no server, no network, no browser.
|
||||
Exit 0 all passed, 1 a failure.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
|
||||
from sqlalchemy import create_engine, select # noqa: E402
|
||||
from sqlalchemy.orm import sessionmaker # noqa: E402
|
||||
|
||||
from server import models, okta_auth, okta_sync # noqa: E402
|
||||
from server.okta_sync import OktaSyncError, OktaUser # noqa: E402
|
||||
|
||||
fails = 0
|
||||
|
||||
|
||||
def chk(label, ok, extra=""):
|
||||
global fails
|
||||
print(("PASS " if ok else "FAIL ") + label + (f" [{extra}]" if (extra and not ok) else ""))
|
||||
if not ok:
|
||||
fails += 1
|
||||
|
||||
|
||||
tmp = tempfile.mkdtemp(prefix="oktasync_")
|
||||
engine = create_engine(f"sqlite:///{os.path.join(tmp, 't.db')}",
|
||||
connect_args={"check_same_thread": False, "timeout": 20}, future=True)
|
||||
models.Base.metadata.create_all(engine)
|
||||
Session = sessionmaker(bind=engine, autoflush=False, future=True)
|
||||
INTERVAL = 900
|
||||
os.environ["OKTA_SYNC_INTERVAL_SECONDS"] = str(INTERVAL)
|
||||
T0 = datetime(2026, 10, 16, 12, 0, 0, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def wipe(users=()):
|
||||
with Session() as db:
|
||||
db.query(models.AuditLog).delete()
|
||||
db.query(models.User).delete()
|
||||
db.query(models.AppSetting).filter(models.AppSetting.key == okta_sync.CLAIM_KEY).delete()
|
||||
for name, active in users:
|
||||
db.add(models.User(id="u_" + name, username=name, is_active=active))
|
||||
db.commit()
|
||||
|
||||
|
||||
def users_state():
|
||||
with Session() as db:
|
||||
return ({u.username: u.is_active for u in db.scalars(select(models.User))},
|
||||
db.query(models.AuditLog).count())
|
||||
|
||||
|
||||
# ── 1. interval ───────────────────────────────────────────────────────────────
|
||||
print("\n1. interval")
|
||||
def iv(v):
|
||||
if v is None:
|
||||
os.environ.pop("OKTA_SYNC_INTERVAL_SECONDS", None)
|
||||
else:
|
||||
os.environ["OKTA_SYNC_INTERVAL_SECONDS"] = v
|
||||
return okta_sync.interval_seconds()
|
||||
|
||||
chk("default is 900 seconds (15 minutes)", iv(None) == 900 and iv("") == 900)
|
||||
chk("an env value is used", iv("600") == 600 and iv("3600") == 3600)
|
||||
chk("not a number: the default", iv("soon") == 900 and iv("1.5") == 900)
|
||||
chk("under the minimum is raised to 300 (a slow cycle can never overlap the next)",
|
||||
iv("60") == 300 and iv("0") == 300 and iv("-5") == 300)
|
||||
chk("a fetch is bounded below the minimum interval",
|
||||
okta_sync.TOTAL_DEADLINE_SECONDS < okta_sync.INTERVAL_MIN_SECONDS)
|
||||
iv(str(INTERVAL))
|
||||
|
||||
# ── 2. the claim ──────────────────────────────────────────────────────────────
|
||||
print("\n2. the claim gate")
|
||||
wipe()
|
||||
with Session() as db:
|
||||
chk("first ever claim wins (the insert is the claim)", okta_sync.claim(db, INTERVAL, now=T0) is True)
|
||||
with Session() as db:
|
||||
chk("a second claim in the same instant loses", okta_sync.claim(db, INTERVAL, now=T0) is False)
|
||||
with Session() as db:
|
||||
chk("one second before the interval is up: loses",
|
||||
okta_sync.claim(db, INTERVAL, now=T0 + timedelta(seconds=INTERVAL - 1)) is False)
|
||||
with Session() as db:
|
||||
chk("when the interval is up: wins",
|
||||
okta_sync.claim(db, INTERVAL, now=T0 + timedelta(seconds=INTERVAL)) is True)
|
||||
with Session() as db:
|
||||
chk("and then the next caller loses again",
|
||||
okta_sync.claim(db, INTERVAL, now=T0 + timedelta(seconds=INTERVAL + 1)) is False)
|
||||
|
||||
# ── 3. the race ───────────────────────────────────────────────────────────────
|
||||
print("\n3. exactly one of many workers wins (a real race on a file database)")
|
||||
|
||||
|
||||
def race(n=8):
|
||||
barrier = threading.Barrier(n)
|
||||
wins, errors = [], []
|
||||
|
||||
def worker():
|
||||
try:
|
||||
with Session() as db:
|
||||
barrier.wait()
|
||||
wins.append(okta_sync.claim(db, INTERVAL, now=datetime.now(timezone.utc)))
|
||||
except Exception as exc: # a lost race may surface as an error
|
||||
errors.append(type(exc).__name__)
|
||||
wins.append(False)
|
||||
|
||||
ts = [threading.Thread(target=worker) for _ in range(n)]
|
||||
[t.start() for t in ts]
|
||||
[t.join() for t in ts]
|
||||
return sum(1 for w in wins if w), errors
|
||||
|
||||
|
||||
wipe()
|
||||
won, errs = race()
|
||||
chk("no row yet, 8 workers at once: exactly 1 wins (the insert race)", won == 1, f"{won} {errs}")
|
||||
bad = []
|
||||
for i in range(15):
|
||||
with Session() as db: # age the claim so the next race is due again
|
||||
db.query(models.AppSetting).filter(models.AppSetting.key == okta_sync.CLAIM_KEY).update(
|
||||
{"updated_at": datetime.now(timezone.utc) - timedelta(seconds=INTERVAL + 5)})
|
||||
db.commit()
|
||||
won, errs = race()
|
||||
if won != 1:
|
||||
bad.append((i, won, errs))
|
||||
chk("row due, 8 workers at once, 15 rounds: exactly 1 wins every round", not bad, str(bad))
|
||||
|
||||
# ── 4. one tick ───────────────────────────────────────────────────────────────
|
||||
print("\n4. a tick runs the sync once per cycle")
|
||||
wipe([("alice", True), ("bob", True)])
|
||||
calls = []
|
||||
|
||||
|
||||
def fetch_ok():
|
||||
calls.append(1)
|
||||
return [OktaUser("alice", "ACTIVE"), OktaUser("bob", "SUSPENDED")]
|
||||
|
||||
|
||||
r1 = okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0)
|
||||
chk("first tick claims and runs", r1 is not None and r1.ok and r1.disabled == ["bob"])
|
||||
r2 = okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0 + timedelta(seconds=30))
|
||||
chk("a tick 30 seconds later does nothing (not due)", r2 is None and len(calls) == 1)
|
||||
r3 = okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0 + timedelta(seconds=INTERVAL))
|
||||
chk("a tick when due runs again", r3 is not None and r3.ok and len(calls) == 2)
|
||||
|
||||
print("\n ...a restart inside the interval does not start a second run")
|
||||
wipe([("alice", True), ("bob", True)])
|
||||
calls.clear()
|
||||
okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0)
|
||||
# "restart": a brand-new caller with nothing in memory, 2 minutes later
|
||||
r = okta_sync.run_if_due(Session, fetch=fetch_ok, now=T0 + timedelta(seconds=120))
|
||||
chk("the claim row outlives the process: no duplicate run", r is None and len(calls) == 1)
|
||||
|
||||
print("\n ...two workers tick in the same instant")
|
||||
wipe([("alice", True), ("bob", True)])
|
||||
calls.clear()
|
||||
results = []
|
||||
barrier = threading.Barrier(2)
|
||||
|
||||
|
||||
def tick():
|
||||
barrier.wait()
|
||||
results.append(okta_sync.run_if_due(Session, fetch=fetch_ok))
|
||||
|
||||
|
||||
ts = [threading.Thread(target=tick) for _ in range(2)]
|
||||
[t.start() for t in ts]
|
||||
[t.join() for t in ts]
|
||||
chk("Okta is asked once, not twice", len(calls) == 1, str(len(calls)))
|
||||
chk("exactly one worker got a result", sum(1 for r in results if r is not None) == 1)
|
||||
_, audits = users_state()
|
||||
chk("one audit row, not two", audits == 1, str(audits))
|
||||
|
||||
# ── 5. fail closed through the scheduled path ─────────────────────────────────
|
||||
print("\n5. FAIL CLOSED through the scheduled wrapper (not just run_sync)")
|
||||
for label, exc in (("OktaSyncError", OktaSyncError("HTTP 500")), ("RuntimeError", RuntimeError("x")),
|
||||
("TimeoutError", TimeoutError("t"))):
|
||||
wipe([("alice", True), ("bob", True)])
|
||||
|
||||
def failing(e=exc):
|
||||
raise e
|
||||
|
||||
r = okta_sync.run_if_due(Session, fetch=failing, now=T0)
|
||||
u, a = users_state()
|
||||
chk(f"{label} in the scheduled path: ok is False, zero changes, zero audit rows",
|
||||
r is not None and r.ok is False and all(u.values()) and a == 0)
|
||||
|
||||
wipe([("alice", True), ("bob", True)])
|
||||
|
||||
|
||||
def broken_factory():
|
||||
raise RuntimeError("database is down")
|
||||
|
||||
|
||||
chk("the database failing at claim time: the tick ends quietly, returns None",
|
||||
okta_sync.run_if_due(broken_factory, fetch=fetch_ok, now=T0) is None)
|
||||
u, a = users_state()
|
||||
chk("...and nothing changed", all(u.values()) and a == 0)
|
||||
|
||||
# ── 6. the thread ─────────────────────────────────────────────────────────────
|
||||
print("\n6. the timer thread")
|
||||
os.environ["OKTA_API_TOKEN"] = ""
|
||||
okta_auth.ISSUER = "https://example.okta.com/oauth2/default"
|
||||
chk("no token: nothing starts", okta_sync.start_scheduler(Session) is None)
|
||||
chk("...no thread named okta-sync", not any(t.name == "okta-sync" for t in threading.enumerate()))
|
||||
|
||||
os.environ["OKTA_API_TOKEN"] = "tok"
|
||||
wipe([("alice", True), ("bob", True)])
|
||||
okta_sync.START_DELAY_SECONDS = 0.05
|
||||
okta_sync.TICK_SECONDS = 0.05
|
||||
ran = []
|
||||
real_fetch = okta_sync.fetch_users
|
||||
okta_sync.fetch_users = lambda http=None: (ran.append(1) or
|
||||
[OktaUser("alice", "ACTIVE"), OktaUser("bob", "DEPROVISIONED")])
|
||||
stop = okta_sync.start_scheduler(Session)
|
||||
chk("configured: a stop Event comes back", stop is not None)
|
||||
time.sleep(1.2) # about 20 ticks
|
||||
u, a = users_state()
|
||||
chk("the thread ran the sync on its own (bob was disabled without a manual call)", u["bob"] is False)
|
||||
chk("many ticks, one run (the claim gate holds)", len(ran) == 1 and a == 1, f"ran={len(ran)} audits={a}")
|
||||
th = [t for t in threading.enumerate() if t.name == "okta-sync"]
|
||||
chk("one timer thread is alive", len(th) == 1 and th[0].is_alive())
|
||||
chk("the thread is a daemon (never blocks shutdown)", th[0].daemon)
|
||||
stop.set()
|
||||
th[0].join(timeout=3)
|
||||
chk("setting the stop Event ends the thread", not th[0].is_alive())
|
||||
okta_sync.fetch_users = real_fetch
|
||||
os.environ["OKTA_API_TOKEN"] = ""
|
||||
|
||||
print()
|
||||
print("ALL PASSED" if not fails else f"{fails} FAILED")
|
||||
sys.exit(0 if not fails else 1)
|
||||
153
tests/okta_sync_ui_check.py
Normal file
@@ -0,0 +1,153 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Does an auto-disable read clearly in the admin console? - D18 / T13.5.
|
||||
|
||||
T13.5 builds no new UI. It confirms that an account the Okta sync disabled shows
|
||||
up legibly in the surface admins already use (the Activity log card on
|
||||
admin.html), without special-casing, and that the disabled account is refused on
|
||||
its very next request. This runs the REAL sync against a throwaway database,
|
||||
then reads the real page in a headless browser at 1440px and 390px.
|
||||
|
||||
It does not rely on CR-019's Activity & usage card: only the Activity log card
|
||||
(AuditLog) is read.
|
||||
|
||||
Boots its own throwaway SQLite + uvicorn + headless browser; run it alone.
|
||||
Screenshots go to a folder printed at the end (WP_SHOTS_DIR overrides it).
|
||||
Exit 0 all passed, 1 a failure, 2 could not run.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import cdp # noqa: E402
|
||||
from browser_check import seed, start_server, chk, _PASS, _FAIL # noqa: E402
|
||||
from qa_gate_check import api # noqa: E402
|
||||
|
||||
|
||||
def ascii_(v, n=300):
|
||||
return re.sub(r"\s+", " ", str(v)).encode("ascii", "replace").decode()[:n]
|
||||
|
||||
|
||||
ROWS_JS = """JSON.stringify([...document.querySelectorAll('#audit-admin tbody tr')].map(
|
||||
tr => [...tr.children].map(td => td.textContent.trim())))"""
|
||||
|
||||
|
||||
def main():
|
||||
exe = cdp.find_browser()
|
||||
if not exe:
|
||||
print("no headless-capable browser found; set WP_BROWSER.")
|
||||
return 2
|
||||
|
||||
tmpdir = tempfile.mkdtemp(prefix="wpsuite-syncui-")
|
||||
shots = os.environ.get("WP_SHOTS_DIR") or os.path.join(tmpdir, "shots")
|
||||
db_path = os.path.join(tmpdir, "check.db")
|
||||
server = browser = None
|
||||
try:
|
||||
tok = seed(db_path)
|
||||
|
||||
print("\n1. run the real sync against the throwaway database")
|
||||
from server.db import SessionLocal
|
||||
from server import okta_sync
|
||||
from server.okta_sync import OktaUser
|
||||
okta = lambda: [OktaUser("root", "ACTIVE"), OktaUser("sue", "ACTIVE"),
|
||||
OktaUser("pat", "SUSPENDED"), OktaUser("mix", "ACTIVE"),
|
||||
OktaUser("sam", "ACTIVE"), OktaUser("legacy", "ACTIVE")] # bob is missing
|
||||
with SessionLocal() as db:
|
||||
res = okta_sync.run_sync(db, fetch=okta)
|
||||
chk("the sync disabled pat (SUSPENDED) and bob (missing)",
|
||||
res.ok and sorted(res.disabled) == ["bob", "pat"], ascii_((res.ok, res.error, res.disabled)))
|
||||
|
||||
port = cdp.free_port()
|
||||
base = "http://127.0.0.1:%d" % port
|
||||
server = start_server(port, db_path)
|
||||
if not server:
|
||||
print("the server did not start")
|
||||
return 2
|
||||
|
||||
print("\n2. a disabled account is refused on its very next request")
|
||||
st, _ = api(base, "/api/auth/me", tok["pat"])
|
||||
chk("pat (auto-disabled) is refused", st in (401, 403), st)
|
||||
st, _ = api(base, "/api/auth/me", tok["bob"])
|
||||
chk("bob (auto-disabled) is refused", st in (401, 403), st)
|
||||
st, _ = api(base, "/api/auth/me", tok["mix"])
|
||||
chk("mix (still active in Okta) still works", st == 200, st)
|
||||
|
||||
print("\n3. the audit rows through the API")
|
||||
st, audit = api(base, "/api/audit?entity_type=user&limit=50", tok["root"])
|
||||
mine = [e for e in (audit or []) if e.get("actor") == "system:okta_sync"]
|
||||
chk("an admin gets both rows", st == 200 and len(mine) == 2, ascii_((st, len(mine))))
|
||||
chk("each row names the action, the user, and the Okta status",
|
||||
{(e["summary"], e["detail"].get("okta_status")) for e in mine} == {("pat", "SUSPENDED"), ("bob", "MISSING")},
|
||||
ascii_([(e["summary"], e["detail"]) for e in mine]))
|
||||
st, audit_sue = api(base, "/api/audit?entity_type=user&limit=50", tok["sue"])
|
||||
chk("a project super user does not see them (they belong to no project)",
|
||||
st == 200 and not [e for e in audit_sue if e.get("actor") == "system:okta_sync"])
|
||||
|
||||
print("\n4. the admin console, as an admin")
|
||||
browser = cdp.Browser(exe)
|
||||
page = browser.page()
|
||||
for width, label in ((1440, "1440"), (390, "390")):
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["root"])
|
||||
page.viewport(width, 900, mobile=(width < 500))
|
||||
page.goto(base + "/admin.html")
|
||||
time.sleep(2.0)
|
||||
page.eval("void loadAudit()")
|
||||
time.sleep(1.2)
|
||||
rows = json.loads(page.eval(ROWS_JS) or "[]")
|
||||
sync_rows = [r for r in rows if "system:okta_sync" in r]
|
||||
chk("[%s] the Activity log lists both auto-disables" % label, len(sync_rows) == 2,
|
||||
ascii_(rows[:4]))
|
||||
if sync_rows:
|
||||
r = next((x for x in sync_rows if "pat" in x), sync_rows[0])
|
||||
# columns: When, Who, Action, Type, Item, Detail
|
||||
chk("[%s] Who reads 'system:okta_sync', not a person" % label, r[1] == "system:okta_sync", r)
|
||||
chk("[%s] Action reads 'user deprovisioned by sync'" % label,
|
||||
r[2] == "user deprovisioned by sync", r)
|
||||
chk("[%s] Type and Item name the account" % label, r[3] == "user" and r[4] == "pat", r)
|
||||
chk("[%s] Detail gives the Okta status found" % label,
|
||||
"SUSPENDED" in r[5] and "okta" in r[5].lower(), r)
|
||||
miss = next((x for x in sync_rows if "bob" in x), None)
|
||||
chk("[%s] a person missing from Okta reads 'MISSING'" % label,
|
||||
bool(miss) and "MISSING" in miss[5], miss)
|
||||
# filter + search work on it with no special-casing
|
||||
page.eval("document.getElementById('audit-search').value='okta_sync'; renderAudit();")
|
||||
rows = json.loads(page.eval(ROWS_JS) or "[]")
|
||||
chk("[%s] searching 'okta_sync' finds exactly the sync rows" % label,
|
||||
len(rows) == 2 and all("system:okta_sync" in r for r in rows), ascii_(rows))
|
||||
page.eval("document.getElementById('audit-search').value='';"
|
||||
"document.getElementById('audit-type').value='user'; renderAudit();")
|
||||
rows = json.loads(page.eval(ROWS_JS) or "[]")
|
||||
chk("[%s] the 'User accounts' filter keeps them" % label,
|
||||
len([r for r in rows if "system:okta_sync" in r]) == 2)
|
||||
page.eval("document.getElementById('audit-type').value=''; renderAudit();")
|
||||
over = page.eval("document.documentElement.scrollWidth - document.documentElement.clientWidth")
|
||||
chk("[%s] no sideways scroll on the page" % label, (over or 0) <= 0, over)
|
||||
card = page.eval("(()=>{const e=document.getElementById('audit-admin');"
|
||||
"e.scrollIntoView();return e.getBoundingClientRect().width>0})()")
|
||||
chk("[%s] the card is on screen" % label, bool(card))
|
||||
time.sleep(0.4)
|
||||
page.screenshot(os.path.join(shots, "admin-activity-log-%s.png" % label), full_page=False)
|
||||
errs = [e for e in page.js_errors() if "beforeunload" not in e]
|
||||
chk("no JavaScript errors", not errs, ascii_(errs[:3]))
|
||||
print("\n screenshots: " + shots)
|
||||
finally:
|
||||
if browser:
|
||||
browser.close()
|
||||
if server:
|
||||
server.terminate()
|
||||
|
||||
print("\n" + "-" * 54)
|
||||
print("%d/%d checks passed." % (len(_PASS), len(_PASS) + len(_FAIL)))
|
||||
for f in _FAIL:
|
||||
print(" - " + f)
|
||||
return 1 if _FAIL else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
236
tests/session_idle_check.py
Normal file
@@ -0,0 +1,236 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Does the sliding session with a hard ceiling work? - D18 / T13.1 (checked in T13.6).
|
||||
|
||||
T13.1 shipped with its done-when list unchecked and no test of its own; the old
|
||||
session tests had no idea about idle time. This is that test.
|
||||
|
||||
1. Continuous activity keeps a session alive past 30 minutes, but never past the
|
||||
absolute ceiling (AUTH_SESSION_HOURS) measured from the ORIGINAL sign-in.
|
||||
2. A session idle for more than 30 minutes is refused on its next request.
|
||||
3. The cookie is not rewritten on every request (the refresh is throttled).
|
||||
4. A token from before this shipped (no login_at) still gets a hard ceiling,
|
||||
from its iat.
|
||||
5. A refresh carries identity and login_at forward unchanged.
|
||||
6. The two variables are documented, and both defaults are flagged as proposed.
|
||||
|
||||
Self-contained: throwaway SQLite, in-process test client, no network, no browser.
|
||||
Exit 0 all passed, 1 a failure.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import tempfile
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
os.environ["DATABASE_URL"] = "sqlite:///" + os.path.join(tempfile.mkdtemp(prefix="wp-idle-"), "t.db").replace("\\", "/")
|
||||
os.environ["AUTH_SECRET_KEY"] = "session-idle-check-secret-not-for-production"
|
||||
os.environ.pop("AUTH_IDLE_MINUTES", None)
|
||||
os.environ.pop("AUTH_SESSION_HOURS", None)
|
||||
|
||||
import jwt # noqa: E402
|
||||
from fastapi.testclient import TestClient # noqa: E402
|
||||
|
||||
from server import app as appmod, auth, models # noqa: E402
|
||||
from server.db import Base, SessionLocal, engine # noqa: E402
|
||||
|
||||
if not hasattr(auth, "maybe_refresh_token"):
|
||||
print("SKIP: T13.1 (sliding session) is not on this branch. It waits for T14.1.")
|
||||
sys.exit(0)
|
||||
|
||||
fails = 0
|
||||
|
||||
|
||||
def chk(label, ok, extra=""):
|
||||
global fails
|
||||
print(("PASS " if ok else "FAIL ") + label + (f" [{extra}]" if (extra and not ok) else ""))
|
||||
if not ok:
|
||||
fails += 1
|
||||
|
||||
|
||||
Base.metadata.create_all(bind=engine)
|
||||
with SessionLocal() as db:
|
||||
db.add(models.User(id="u1", username="alice", email="a@x.test", role=auth.ROLE_ADMIN))
|
||||
db.commit()
|
||||
USER = db.get(models.User, "u1")
|
||||
db.expunge(USER)
|
||||
|
||||
# A route the auth gate covers. NOT /api/auth/me: everything under /api/auth/ is
|
||||
# exempt from the gate (auth._EXEMPT_PREFIXES), so no session refresh happens there
|
||||
# and a probe on it would pass without testing anything.
|
||||
PROBE = "/api/projects"
|
||||
IDLE = auth.IDLE_MINUTES
|
||||
CEIL_H = auth.SESSION_HOURS
|
||||
NOW = datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def mint(login_at=None, iat=None, exp=None, **extra):
|
||||
"""A token with exactly the claims given (None = leave the claim out)."""
|
||||
p = {"sub": "u1", "username": "alice", "role": auth.ROLE_ADMIN, "ver": 0}
|
||||
if iat is not None:
|
||||
p["iat"] = iat
|
||||
if login_at is not None:
|
||||
p["login_at"] = login_at.timestamp()
|
||||
if exp is not None:
|
||||
p["exp"] = exp
|
||||
p.update(extra)
|
||||
return jwt.encode(p, auth.SECRET_KEY, algorithm=auth.JWT_ALG)
|
||||
|
||||
|
||||
def raw_claims(tok):
|
||||
return jwt.decode(tok, auth.SECRET_KEY, algorithms=[auth.JWT_ALG], options={"verify_exp": False, "verify_iat": False, "verify_nbf": False})
|
||||
|
||||
|
||||
class FakeClock(datetime):
|
||||
_now = NOW
|
||||
|
||||
@classmethod
|
||||
def now(cls, tz=None):
|
||||
return cls._now
|
||||
|
||||
|
||||
real_dt = auth.datetime
|
||||
|
||||
|
||||
def at(t):
|
||||
FakeClock._now = t
|
||||
auth.datetime = FakeClock
|
||||
|
||||
|
||||
def real():
|
||||
auth.datetime = real_dt
|
||||
|
||||
|
||||
print("defaults: idle %d min, ceiling %d h" % (IDLE, CEIL_H))
|
||||
|
||||
# ── 1. continuous activity, then the ceiling ──────────────────────────────────
|
||||
print("\n1. continuous activity stays alive past 30 minutes, but is cut off at the ceiling")
|
||||
at(NOW)
|
||||
tok = auth.create_token(USER)
|
||||
c = raw_claims(tok)
|
||||
chk("a new token carries login_at", "login_at" in c)
|
||||
chk("a new token expires one idle window out, not at the ceiling",
|
||||
abs(c["exp"] - (NOW + timedelta(minutes=IDLE)).timestamp()) < 2)
|
||||
login_at = NOW
|
||||
cur = c
|
||||
alive_past_30 = False
|
||||
rewrites = 0
|
||||
last_exp = cur["exp"]
|
||||
t = NOW
|
||||
STEP = timedelta(minutes=1) # an active user: a request every minute
|
||||
end = NOW + timedelta(hours=CEIL_H + 2)
|
||||
cut_off_at = None
|
||||
while t < end:
|
||||
t += STEP
|
||||
at(t)
|
||||
if cur["exp"] <= t.timestamp(): # the session already lapsed
|
||||
cut_off_at = t
|
||||
break
|
||||
if t - NOW > timedelta(minutes=IDLE):
|
||||
alive_past_30 = True
|
||||
new = auth.maybe_refresh_token(cur)
|
||||
if new:
|
||||
rewrites += 1
|
||||
cur = raw_claims(new)
|
||||
last_exp = cur["exp"]
|
||||
real()
|
||||
chk("with a request every minute the session is alive well past 30 minutes", alive_past_30)
|
||||
chk("it is never extended past the ceiling (last exp <= login + %d h)" % CEIL_H,
|
||||
last_exp <= (login_at + timedelta(hours=CEIL_H)).timestamp() + 1,
|
||||
str(datetime.fromtimestamp(last_exp, timezone.utc)))
|
||||
chk("it IS cut off once the ceiling is reached, however active the session",
|
||||
cut_off_at is not None and cut_off_at <= login_at + timedelta(hours=CEIL_H) + timedelta(minutes=IDLE) + STEP,
|
||||
str(cut_off_at))
|
||||
chk("the claims after the last refresh still carry the ORIGINAL login_at",
|
||||
abs(cur["login_at"] - login_at.timestamp()) < 1)
|
||||
steps = int((CEIL_H * 60) / 1)
|
||||
chk("the cookie was rewritten far less often than once per request (%d rewrites for ~%d requests)" % (rewrites, steps),
|
||||
0 < rewrites < steps / 5, f"{rewrites}/{steps}")
|
||||
|
||||
# ── 2. idle ───────────────────────────────────────────────────────────────────
|
||||
print("\n2. a session idle for more than %d minutes is refused on its next request" % IDLE)
|
||||
client = TestClient(appmod.app)
|
||||
idle_tok = mint(login_at=NOW - timedelta(minutes=IDLE + 1), iat=NOW - timedelta(minutes=IDLE + 1),
|
||||
exp=NOW - timedelta(minutes=1))
|
||||
client.cookies.set(auth.COOKIE_NAME, idle_tok)
|
||||
r = client.get(PROBE)
|
||||
chk("an idle session gets 401", r.status_code == 401, r.status_code)
|
||||
live_tok = mint(login_at=NOW - timedelta(minutes=5), iat=NOW - timedelta(minutes=5),
|
||||
exp=NOW + timedelta(minutes=IDLE - 5))
|
||||
client.cookies.clear()
|
||||
client.cookies.set(auth.COOKIE_NAME, live_tok)
|
||||
chk("a session inside the window gets 200", client.get(PROBE).status_code == 200)
|
||||
|
||||
# ── 3. throttle ───────────────────────────────────────────────────────────────
|
||||
print("\n3. the cookie is not rewritten on every request")
|
||||
fresh = auth.create_token(USER)
|
||||
client.cookies.clear()
|
||||
client.cookies.set(auth.COOKIE_NAME, fresh)
|
||||
cookies_set = 0
|
||||
for _ in range(20):
|
||||
r = client.get(PROBE)
|
||||
if "set-cookie" in r.headers and auth.COOKIE_NAME in r.headers["set-cookie"]:
|
||||
cookies_set += 1
|
||||
chk("20 rapid requests on a fresh session: 0 cookie rewrites", cookies_set == 0, cookies_set)
|
||||
|
||||
aged = mint(login_at=NOW - timedelta(minutes=20), iat=NOW - timedelta(minutes=20),
|
||||
exp=NOW + timedelta(minutes=2)) # 2 minutes of idle window left
|
||||
client.cookies.clear()
|
||||
client.cookies.set(auth.COOKIE_NAME, aged)
|
||||
r = client.get(PROBE)
|
||||
chk("an aged session gets a refreshed cookie on its next request",
|
||||
r.status_code == 200 and auth.COOKIE_NAME in r.headers.get("set-cookie", ""))
|
||||
m = re.search(auth.COOKIE_NAME + r"=([^;]+)", r.headers.get("set-cookie", ""))
|
||||
newc = raw_claims(m.group(1)) if m else {}
|
||||
chk("the refreshed cookie keeps the original login_at",
|
||||
abs(newc.get("login_at", 0) - (NOW - timedelta(minutes=20)).timestamp()) < 1)
|
||||
client.cookies.clear()
|
||||
client.cookies.set(auth.COOKIE_NAME, m.group(1) if m else "")
|
||||
r2 = client.get(PROBE)
|
||||
chk("...and the very next request does not rewrite it again", auth.COOKIE_NAME not in r2.headers.get("set-cookie", ""))
|
||||
|
||||
# ── 4. tokens from before D18 ─────────────────────────────────────────────────
|
||||
print("\n4. a token with no login_at still gets a hard ceiling (via iat)")
|
||||
old = {"sub": "u1", "username": "alice", "role": auth.ROLE_ADMIN, "ver": 0,
|
||||
"iat": (NOW - timedelta(hours=CEIL_H + 1)).timestamp(),
|
||||
"exp": (NOW + timedelta(hours=3)).timestamp()} # the old flat 12 h exp, still valid
|
||||
chk("past its iat-based ceiling: no refresh, the old exp is not extended", auth.maybe_refresh_token(old) is None)
|
||||
young = {"sub": "u1", "username": "alice", "role": auth.ROLE_ADMIN, "ver": 0,
|
||||
"iat": (NOW - timedelta(hours=1)).timestamp(),
|
||||
"exp": (NOW + timedelta(minutes=2)).timestamp()}
|
||||
ref = auth.maybe_refresh_token(young)
|
||||
rc = raw_claims(ref) if ref else {}
|
||||
chk("inside the ceiling: refreshed, and login_at is taken from iat (not 'now')",
|
||||
bool(ref) and abs(rc["login_at"] - young["iat"]) < 1)
|
||||
chk("the refreshed token cannot outlive iat + ceiling",
|
||||
bool(ref) and rc["exp"] <= young["iat"] + CEIL_H * 3600 + 1)
|
||||
nothing = {"sub": "u1", "username": "alice", "role": auth.ROLE_ADMIN, "ver": 0,
|
||||
"exp": (NOW + timedelta(minutes=2)).timestamp()}
|
||||
chk("no login_at and no iat: no refresh, and no crash", auth.maybe_refresh_token(nothing) is None)
|
||||
junk = dict(young, login_at="not-a-number")
|
||||
chk("an unparseable login_at falls back to iat, no crash", auth.maybe_refresh_token(junk) is not None)
|
||||
|
||||
# ── 5. identity carried forward ───────────────────────────────────────────────
|
||||
print("\n5. a refresh carries the identity forward unchanged")
|
||||
src = {"sub": "u9", "username": "zed", "role": "project_admin", "ver": 7,
|
||||
"iat": (NOW - timedelta(minutes=20)).timestamp(),
|
||||
"login_at": (NOW - timedelta(minutes=20)).timestamp(),
|
||||
"exp": (NOW + timedelta(minutes=2)).timestamp()}
|
||||
rc = raw_claims(auth.maybe_refresh_token(src))
|
||||
chk("sub, username, role, ver are unchanged",
|
||||
(rc["sub"], rc["username"], rc["role"], rc["ver"]) == ("u9", "zed", "project_admin", 7))
|
||||
|
||||
# ── 6. documentation ──────────────────────────────────────────────────────────
|
||||
print("\n6. the two variables are documented, defaults flagged as proposed")
|
||||
env = open(os.path.join(ROOT, "server", ".env.example"), encoding="utf-8").read()
|
||||
dep = open(os.path.join(ROOT, "DEPLOYMENT.md"), encoding="utf-8").read()
|
||||
for name in ("AUTH_IDLE_MINUTES", "AUTH_SESSION_HOURS"):
|
||||
chk(f"{name} is in .env.example", name in env)
|
||||
chk(f"{name} is in DEPLOYMENT.md", name in dep)
|
||||
chk(".env.example calls the defaults proposed, not confirmed", re.search(r"propos", env, re.I) is not None)
|
||||
chk("DEPLOYMENT.md calls the defaults proposed, not confirmed", re.search(r"propos", dep, re.I) is not None)
|
||||
|
||||
print()
|
||||
print("ALL PASSED" if not fails else f"{fails} FAILED")
|
||||
sys.exit(0 if not fails else 1)
|
||||
@@ -1,192 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Is there exactly one analytics implementation, reported from admin? — D5, T7.10.
|
||||
|
||||
Usage analytics existed twice (creator + wizard), five of the nine colliding
|
||||
globals `creator-frame.md` counted, and the wizard's copy had no caller. One
|
||||
core survives in wp-usage.js; the pages keep thin track() wrappers; the report
|
||||
and its downloads live on the admin console behind the same role gate as the
|
||||
rest of that page. Keys are unchanged, so pre-move data still reads.
|
||||
|
||||
Self-contained: throwaway SQLite, its own uvicorn, headless Edge or Chrome.
|
||||
Exit 0 all passed, 1 a failure, 2 could not run.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import cdp # noqa: E402
|
||||
from browser_check import seed, start_server, chk, _PASS, _FAIL # noqa: E402
|
||||
from sections_check import set_sop # noqa: E402
|
||||
from stepper_check import dismiss_dialogs # noqa: E402
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
HTML = os.path.join(ROOT, "html")
|
||||
|
||||
|
||||
def ascii_(v, n=280):
|
||||
return re.sub(r"\s+", " ", str(v)).encode("ascii", "replace").decode()[:n]
|
||||
|
||||
|
||||
def settle(seconds=0.6):
|
||||
time.sleep(seconds)
|
||||
|
||||
|
||||
def strip_js(src):
|
||||
src = re.sub(r"/\*.*?\*/", "", src, flags=re.S)
|
||||
return "\n".join(re.sub(r"(?<![:'\"])//.*$", "", ln) for ln in src.split("\n"))
|
||||
|
||||
|
||||
def main():
|
||||
exe = cdp.find_browser()
|
||||
if not exe:
|
||||
print("no headless-capable browser found; set WP_BROWSER.")
|
||||
return 2
|
||||
|
||||
# ── 1. the grep half: one implementation, no leftover controls ───────────
|
||||
print("\n1. grep: one implementation, nothing unreferenced")
|
||||
files = {}
|
||||
for name in os.listdir(HTML):
|
||||
if name.endswith((".js", ".html")):
|
||||
files[name] = strip_js(open(os.path.join(HTML, name), encoding="utf-8").read())
|
||||
|
||||
core_defs = [n for n, src in files.items() if "window.WPUsage" in src]
|
||||
chk("the WPUsage core is defined in wp-usage.js and only there",
|
||||
core_defs == ["wp-usage.js"], ascii_(core_defs))
|
||||
chk("the pages record THROUGH it - no page touches the storage keys directly",
|
||||
all("wp_iwp_analytics_v1" not in src and "wp_suite_analytics_v1" not in src
|
||||
for n, src in files.items()
|
||||
if n not in ("wp-usage.js",) and n.endswith(".js")))
|
||||
leftovers = {n: re.findall(r"analyticsLoad|analyticsSave|downloadAnalytics|showAnalytics"
|
||||
r"|ANALYTICS_KEY|USAGE_KEY|usageLoad|downloadUsage", src)
|
||||
for n, src in files.items() if n != "wp-usage.js"}
|
||||
leftovers = {n: v for n, v in leftovers.items() if v}
|
||||
chk("none of the five colliding globals survives anywhere; grep confirms",
|
||||
not leftovers, ascii_(leftovers))
|
||||
chk("no analytics control remains on the creator or the wizard; grep confirms",
|
||||
"Usage data" not in files["wp-creation-index.html"]
|
||||
and "showAnalytics" not in files["work-package-suite.html"])
|
||||
chk("both storage keys survive, verbatim, in the core (data continuity)",
|
||||
"wp_iwp_analytics_v1" in files["wp-usage.js"]
|
||||
and "wp_suite_analytics_v1" in files["wp-usage.js"])
|
||||
|
||||
tmpdir = tempfile.mkdtemp(prefix="wpsuite-usage-")
|
||||
db_path = os.path.join(tmpdir, "check.db")
|
||||
server = None
|
||||
browser = None
|
||||
try:
|
||||
tok = seed(db_path)
|
||||
set_sop(db_path, {})
|
||||
port = cdp.free_port()
|
||||
base = "http://127.0.0.1:%d" % port
|
||||
server = start_server(port, db_path)
|
||||
|
||||
browser = cdp.Browser(exe)
|
||||
page = browser.page()
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["root"])
|
||||
page.viewport(1440, 900)
|
||||
|
||||
# ── 2. recording still works from both tools ─────────────────────────
|
||||
print("\n2. the tools still record")
|
||||
page.goto(base + "/wp-creation-index.html?project=projA")
|
||||
dismiss_dialogs(page)
|
||||
settle(2.0)
|
||||
# Plant a LEGACY-format event under the pre-move key: the done-when is
|
||||
# that data recorded before this task is still readable after it.
|
||||
page.eval("""(() => {
|
||||
const d = JSON.parse(localStorage.getItem('wp_iwp_analytics_v1')) || {events: []};
|
||||
d.events.unshift({ts: '2026-07-01T10:00:00Z', session: 's_legacy',
|
||||
event: 'legacy_probe_event', detail: null});
|
||||
localStorage.setItem('wp_iwp_analytics_v1', JSON.stringify(d));
|
||||
})()""")
|
||||
n0 = page.eval("WPUsage.load(WPUsage.KEYS.creator).events.length")
|
||||
page.eval("track('probe_event')")
|
||||
chk("the creator's track() still lands events under the old key",
|
||||
page.eval("WPUsage.load(WPUsage.KEYS.creator).events.length") == n0 + 1)
|
||||
|
||||
page.goto(base + "/work-package-suite.html?tab=sop")
|
||||
dismiss_dialogs(page)
|
||||
settle(2.0)
|
||||
n0 = page.eval("WPUsage.load(WPUsage.KEYS.wizard).events.length")
|
||||
page.eval("track('probe_event_wizard')")
|
||||
chk("the wizard's track() still lands events under its old key",
|
||||
page.eval("WPUsage.load(WPUsage.KEYS.wizard).events.length") == n0 + 1)
|
||||
wiz_errors = [e for e in page.js_errors() if "beforeunload" not in e]
|
||||
chk("...and the wizard page throws no errors without its old globals "
|
||||
"(the blocked-beforeunload console line is BL-020, filtered not hidden)",
|
||||
not wiz_errors, ascii_(wiz_errors[:2]))
|
||||
|
||||
# ── 3. the report, on admin, behind the admin gate ───────────────────
|
||||
print("\n3. the admin report")
|
||||
page.goto(base + "/admin.html")
|
||||
dismiss_dialogs(page)
|
||||
settle(2.0)
|
||||
page.eval("loadUsage()")
|
||||
settle(0.5)
|
||||
report = page.eval("(document.getElementById('usage-admin')||{textContent:''}).textContent")
|
||||
chk("usage data is reachable from admin.html, both tools reported",
|
||||
"Work package creator" in report and "SOP wizard" in report, ascii_(report, 160))
|
||||
chk("the pre-move legacy event is readable in the report",
|
||||
"legacy_probe_event" in report)
|
||||
chk("this session's fresh events are in it too",
|
||||
"probe_event" in report and "probe_event_wizard" in report)
|
||||
chk("each tool offers its download from the report",
|
||||
page.eval("[...document.querySelectorAll('#usage-admin button')].length") >= 2)
|
||||
|
||||
page.viewport(390, 844, mobile=True)
|
||||
settle(0.6)
|
||||
fits = page.eval("""(() => {
|
||||
const box = document.getElementById('usage-admin');
|
||||
return box && box.scrollWidth <= box.clientWidth + 2
|
||||
&& document.documentElement.scrollWidth <= 392;
|
||||
})()""")
|
||||
chk("the report is usable at 390px - no sideways scrolling", bool(fits))
|
||||
page.viewport(1440, 900)
|
||||
settle(0.4)
|
||||
|
||||
# The same role gate as the rest of the console: a non-admin sees the
|
||||
# denied card and no cards, this one included.
|
||||
page.clear_cookies()
|
||||
page.set_cookie("wp_session", tok["pat"])
|
||||
page.goto(base + "/admin.html")
|
||||
dismiss_dialogs(page)
|
||||
settle(2.0)
|
||||
chk("a non-administrator gets the denied notice, not the usage report",
|
||||
page.eval("""(() => {
|
||||
const denied = document.getElementById('admin-denied');
|
||||
const usage = document.getElementById('usage-admin');
|
||||
const visible = el => !!el && el.offsetParent !== null;
|
||||
return visible(denied) && !visible(usage);
|
||||
})()"""))
|
||||
|
||||
js_errors = [e for e in page.js_errors() if "beforeunload" not in e]
|
||||
chk("no JavaScript errors anywhere in this run", not js_errors,
|
||||
ascii_(js_errors[:2]))
|
||||
|
||||
finally:
|
||||
if browser is not None:
|
||||
try:
|
||||
browser.close()
|
||||
except Exception:
|
||||
pass
|
||||
if server is not None:
|
||||
try:
|
||||
server.terminate()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
print("\n" + "-" * 54)
|
||||
print("%d/%d checks passed." % (len(_PASS), len(_PASS) + len(_FAIL)))
|
||||
for f in _FAIL:
|
||||
print(" - " + f)
|
||||
return 1 if _FAIL else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||