Release: wave 11 + T13.2-T13.5 (D18 Okta sync), T13.1 held back until T14.1; mobile roadmap docs (BL-062..BL-069)
This commit is contained in:
@@ -696,3 +696,124 @@ deliberately deferred.
|
||||
- **Suggested wave or follow-up:** wave 14, alongside T14.1's badge changes:
|
||||
do not show the badge on pages with nothing to sync, and keep it clear of
|
||||
page controls.
|
||||
|
||||
<<<<<<< ours
|
||||
### BL-069 - The sync has no cap on one run and no last-admin guard
|
||||
|
||||
- **Found during:** T13.3 (D18)
|
||||
- **Where:** `server/okta_sync.py` `run_sync()`
|
||||
- **What:** a run disables every account the rules select, however many. If the
|
||||
identity attribute is partly wrong (it matches some accounts and not others),
|
||||
one run can disable most of the company, including every admin. The
|
||||
zero-match guard only stops the case where nothing matches. Disabled accounts
|
||||
cannot sign in, so re-enabling needs an admin who is still active, or the
|
||||
`manage_users.py` command line on the host.
|
||||
- **Why not now:** D18 does not ask for it, and the right limit is a product
|
||||
call: a count, a share of accounts, a rule that the last active admin never
|
||||
disables, or all three.
|
||||
- **Suggested wave or follow-up:** before the first live run, or in T13.6 if
|
||||
Matt decides. Needs Matt.
|
||||
|
||||
=======
|
||||
## Future build: mobile app and scaling
|
||||
|
||||
Logged 2026-10-12 from the meeting with Nick. The answers are in
|
||||
`docs/waves/decisions-2026-10-12.md` (Q2 to Q14). The path from here to a
|
||||
mobile version outside the VPN, across projects, is in
|
||||
`docs/waves/roadmap-mobile-scaling.md`. These are features for a future
|
||||
build. They have no wave and no `CR-`/`D-` id yet. Give each one an id when a
|
||||
wave is planned. Items marked "to confirm" have an open question listed in the
|
||||
decision file ("Still open"). Do not build them until it is answered.
|
||||
|
||||
Follow-up answers from Matt on 2026-10-12 are folded into the entries below.
|
||||
|
||||
Server work is needed for BL-062 to BL-066 (CLAUDE.md, "Frontend and backend
|
||||
boundary"). Do not use `localStorage` as the store of record.
|
||||
|
||||
### BL-062 - Offline Field View for crews
|
||||
|
||||
- **From:** Q2, Q3
|
||||
- **What:** Field View opens offline for up to **7 days** after the last sign-in
|
||||
(today: 12 hours). Only crews in Field View work offline. The creator has no
|
||||
offline mode. When the app opens online again and the account is disabled,
|
||||
wipe that account's local data.
|
||||
- **Why not now:** new scope. It needs an id, and it follows the outbox work in
|
||||
wave 14 (T14.1 to T14.3).
|
||||
- **Suggested wave or follow-up:** future mobile wave. Check it against the D18
|
||||
session rules (idle timeout, absolute ceiling): a 7 day offline window and a
|
||||
session ceiling must agree on what "signed in" means.
|
||||
|
||||
### BL-063 - Offline edit conflicts: merge, notify, log
|
||||
|
||||
- **From:** Q4
|
||||
- **What:** When an offline edit meets a newer server version, the app merges
|
||||
the two, shows a notification to the crew, writes a log entry, and updates the
|
||||
owner. If both sides changed the same field, **the field value wins**, and the
|
||||
app keeps a **history of changes**. This replaces the earlier default (keep
|
||||
both, crew chooses).
|
||||
- **To confirm:** who "the owner" is, how long the history is kept, and who can
|
||||
read it. See the decision file, "Still open" item 2.
|
||||
- **Suggested wave or follow-up:** same wave as BL-062.
|
||||
|
||||
### BL-064 - Field photo limits
|
||||
|
||||
- **From:** Q5
|
||||
- **What:** a limit of **5 MB per photo**. This equals the D8 limit that the
|
||||
server already enforces for drawings (`FILE_MAX_BYTES`), so photos can use the
|
||||
same upload path and the same D19 download headers.
|
||||
- **To confirm:** the count per package, and whether the app resizes a photo
|
||||
before upload (phone photos are often 3 MB to 8 MB). Decision file, "Still
|
||||
open" item 1.
|
||||
- **Suggested wave or follow-up:** same wave as BL-062.
|
||||
|
||||
### BL-065 - Batch print and merged PDF
|
||||
|
||||
- **From:** Q6, Q7, Q8, Q9
|
||||
- **What:** print many packages in one PDF.
|
||||
- Limit: **25 packages on phones**. On desktop the user can scale the filter
|
||||
up to 50 or 100. Open: who sets the larger value (each user, or an admin
|
||||
setting). Decision file, "Still open" item 5.
|
||||
- Drawings go **after the form**, in **upload order**, **scaled to fit**.
|
||||
- SVG drawings are **left out**. A page names each file that was left out.
|
||||
- Every page shows the **package number, the print date, and page X of Y**.
|
||||
- **Related:** BL-000d (merge or list attachments on export) asked the same
|
||||
question for CR-008. This entry answers the layout part of it.
|
||||
- **Suggested wave or follow-up:** future print wave.
|
||||
|
||||
### BL-066 - Keep drawings when a package is deleted
|
||||
|
||||
- **From:** Q13
|
||||
- **What:** deleting a package no longer deletes its drawings. Keep them for
|
||||
data retention. **The storage quota is freed** when the package is deleted.
|
||||
Storage will move to cloud storage in a future deployment. This replaces the
|
||||
earlier default (delete with the package).
|
||||
- **Today:** drawing bytes live in the Postgres `wp_files` table (CR-007, D8).
|
||||
The 2 GB per project limit counts them. Kept files must not count toward that
|
||||
limit, so the count has to change.
|
||||
- **To confirm:** retention period, who can delete for good, who can recover.
|
||||
See the decision file, "Still open" item 3.
|
||||
- **Suggested wave or follow-up:** roadmap Phase 3 (object storage). It affects
|
||||
D19 (safe download) paths too: recovered files must keep the attachment and
|
||||
sandbox headers.
|
||||
|
||||
### BL-067 - Ready for QA progress weight is 0.85
|
||||
|
||||
- **From:** Q12
|
||||
- **What:** change the progress weight for Ready for QA from 0 to **0.85**.
|
||||
Today a package falls from 75% to 0% when it reaches Ready for QA.
|
||||
- **Why not now:** it changes a shipped number on dashboards and roll-ups. Use
|
||||
its own item id and re-run the roll-up checks.
|
||||
- **Suggested wave or follow-up:** future wave. Small task.
|
||||
|
||||
### BL-068 - One cost-code list, kept in the SOP
|
||||
|
||||
- **From:** Q14
|
||||
- **What:** the creator has 38 cost codes, the SOP has 39, and 13 differ. Keep
|
||||
**one list, in the SOP**, and make the creator read it.
|
||||
- **Blocked on:** the correct codes. The **PM** supplies them (decided
|
||||
2026-10-12). The date is not set. This is a data question and not a code
|
||||
question.
|
||||
- **Related:** BL-019 (closed) made a stored code that has left the list stay
|
||||
as an option. Keep that behavior when the list changes.
|
||||
- **Suggested wave or follow-up:** future wave, after the codes arrive.
|
||||
>>>>>>> theirs
|
||||
|
||||
97
docs/waves/context-handoff-2026-10-12.md
Normal file
97
docs/waves/context-handoff-2026-10-12.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Context handoff: Work Package Suite
|
||||
|
||||
> Update 2026-10-16: waves 11 and 13 (T13.2 to T13.5) are on the release branch `release/waves-11-13-okta-sync`, with T13.1 held back. The branch and wave status tables below describe 2026-10-12. See `docs/waves/wave-13.md` for the current state.
|
||||
|
||||
Date: 2026-10-12
|
||||
Purpose: Give a new chat the facts about waves 11, 12, 13, and 16.
|
||||
This file is a handoff note. It is not a spec. The spec files stay the source of truth (`IMPLEMENTATION.md`, `docs/waves/*`, `CLAUDE.md`).
|
||||
It was checked against `git` on 2026-10-12. Earlier chat summaries about this repo are out of date. Use this file.
|
||||
|
||||
## 1. Branch state
|
||||
|
||||
| Branch | State |
|
||||
|---|---|
|
||||
| `main` | At `d42ae0c` (2026-10-06). Contains wave 12 (CR-020, bulk user editing) and wave 16 (D19, D20, D21), both merged by PR. |
|
||||
| `feat/waves-11-13` | Local and `origin` are the same commit, `da1325c`. It is pushed. It is not merged. It is 13 commits ahead of `main` and 19 behind. |
|
||||
| `notes/cr019-usage-metrics-d18-deprovisioning` | Holds the same wave 11 work plus the session notes. |
|
||||
|
||||
Correction to an earlier statement: the Gitea push is no longer blocked. The branch is on `origin`.
|
||||
|
||||
Merge rule from `wave-12.md`: T13.1 (idle timeout) must not reach `main` before T14.1. Do not merge `feat/waves-11-13` into `main` as a whole without a decision from the user. Wave 11 and T13.1 are on the same branch.
|
||||
|
||||
Working tree on `main` (2026-10-12) has two untracked files: `WorkPackageSuite_Consolidated_Change_Request_R2.docx` and `docs/waves/decisions-2026-08-25.md`. Do not delete them.
|
||||
|
||||
## 2. What is on `feat/waves-11-13`
|
||||
|
||||
| Commit | Task | Item |
|
||||
|---|---|---|
|
||||
| `75ac930` | Planning docs for CR-019, CR-020, D18 | CR-019, CR-020, D18 |
|
||||
| `0652fa7` | T11.1 UsageEvent model and migration | CR-019 |
|
||||
| `850b789` | T11.2 capture usage events | CR-019 |
|
||||
| `3584695`, `6cde6e3` | T13.1 idle timeout with an absolute ceiling | D18 |
|
||||
| `8e863ae` | T11.3 aggregation endpoint with filters | CR-019 |
|
||||
| `2de76d5` | T11.4 usage export (raw and sanitized CSV) | CR-019 |
|
||||
| `a6c3fbf`, `0b5ab59` | T11.5 admin console "Activity & usage" card, checked at 390px and 1440px | CR-019 |
|
||||
| `b2a083a` | T11.6 retire the per-browser Usage report | CR-019 |
|
||||
| `2842ec9` | Session notes for 2026-09-23 | none |
|
||||
| `4f3a94b` | T11.6 follow-up: fix `tests/autosave_check.py` | CR-019 |
|
||||
| `da1325c` | T11.7 full verification pass. Wave 11 closed on 2026-09-24. | CR-019 |
|
||||
|
||||
Notes:
|
||||
- `html/wp-usage.js` and `tests/usage_check.py` were deleted in T11.6. The `track()` functions in `wp-creation-app.js` and `work-package-suite-app.js` are now documented no-ops. About 45 call sites stay.
|
||||
- T11.7 found a real regression that T11.6 caused. `tests/autosave_check.py` still expected the retired tracker to write to `localStorage`. The test was fixed. It now passes 33 of 33. The cause was that T11.6 ran only `tests/baseline_shots.py`, not the full suite. Run the full suite for any frontend task.
|
||||
|
||||
## 3. T11.7 verification results (2026-09-24)
|
||||
|
||||
- Alembic: upgrade, downgrade, upgrade on SQLite passed. The SQL render for Postgres was clean (this catches SQLite-only DDL, the "BL-027 class" of defect).
|
||||
- Test scripts: 38 of 39 passed after the autosave fix.
|
||||
- Two known failures that are not regressions:
|
||||
- `token_check.py` exits 2. This is a CLI usage detail.
|
||||
- `generalinfo_check.py` finds an old color-literal gap in `wp-creation-styles.css`.
|
||||
|
||||
## 4. Wave status
|
||||
|
||||
| Wave | Items | Status |
|
||||
|---|---|---|
|
||||
| 10 | Okta OIDC sign-in (D15, D16) | Done. In production. |
|
||||
| 11 | CR-019 usage and activity metrics | Done on `feat/waves-11-13`. Not on `main`. |
|
||||
| 12 | CR-020 bulk user editing | Done. Merged to `main` on 2026-10-06 (T12.1 to T12.6). |
|
||||
| 13 | D18 Okta/AD deprovisioning sync | T13.1 done on `feat/waves-11-13`. T13.2 and later are not started. T13.3 must fail closed. Do not relax that. |
|
||||
| 14, 15 | Not checked in this handoff | T14.1 must land before T13.1 reaches `main`. Read the wave files before you plan. |
|
||||
| 16 | D19 safe downloads, D20 one SOP per project, D21 session decode fails closed | Done. Merged to `main` on 2026-10-06. |
|
||||
|
||||
Wave 16 decisions (`docs/waves/decisions-2026-10-02.md`): SEC-06 (session revocation) waits for the AWS work. Other SEC findings stay as backlog drafts in `docs/reference/security-review/backlog-drafts.md`.
|
||||
|
||||
## 5. Rules that apply here (`CLAUDE.md`)
|
||||
|
||||
- Every change traces to an item ID (`CR-`, `F`, `S`, `A`, `B`, `C`, `D`, task `T<wave>.<n>`). New scope gets a new ID.
|
||||
- One task per PR. Put the task ID and item IDs in the commit message and PR title.
|
||||
- Do not fix other problems that you see. Log them in `docs/waves/backlog.md`.
|
||||
- Do not reorder waves. Do not start a wave before the previous wave is merged, unless the task says it is independent.
|
||||
- Server work is needed for CR-004, CR-018, CR-007, CR-011, CR-014, CR-013, and B4. Do not use `localStorage` for these.
|
||||
- Do not change: Actual Hours in Closeout (CR-017), localization (A7), uppercase card headers in `console.css` (A5), the logged-override path `confirmEarlyRelease()` (A1), or hidden-not-deleted fields (CR-002, CR-016).
|
||||
- Token rule: one place defines color, spacing, and type. Do not add raw hex values to page stylesheets.
|
||||
- Accessibility (C1): use real `<button>`, `<a>`, or input elements. Add `aria-live` for status changes. Keep focus visible. Text contrast is 4.5:1 (3:1 for large text).
|
||||
- Frontend checks: run `uvicorn server.app:app` on a throwaway SQLite database. Check 390px and 1440px. Capture before and after screenshots. Run the smoke test.
|
||||
- If a done-when check cannot be run, do not mark the task complete. Say which check failed and why.
|
||||
|
||||
## 6. Environment on the user's PC
|
||||
|
||||
- Python was installed with `winget`. Tests use the repo `.venv`.
|
||||
- Browser tests (`tests/cdp.py` and about 40 `*_check.py` scripts) need Edge or Chrome. `cdp.py` starts from an elevated shell (wave 16 note). The Linux sandbox cannot run them.
|
||||
- See `C:\Users\m.mabrey\GIT\Field Value Driver\sde-meeting-toolkit\docs\context\environment-notes.md` for tool problems (PowerShell `$` removal, long git paths, the 180 second command limit).
|
||||
|
||||
## 7. Open items
|
||||
|
||||
1. Decide how `feat/waves-11-13` reaches `main`. The T13.1 and T14.1 order is the constraint.
|
||||
2. Plan the next wave. T13.2 and later need an explicit instruction.
|
||||
3. Future build list added 2026-10-12: `BL-062` to `BL-068` in `docs/waves/backlog.md` (mobile app, offline Field View, batch print, and related items). Answers are in `docs/waves/decisions-2026-10-12.md` (follow-ups from Matt included). The path to a mobile version outside the VPN is in `docs/waves/roadmap-mobile-scaling.md` (phases 0 to 5; confirmed: all users in the Okta group, multiple customers, company-managed devices; nine open questions in its section 7). These files are uncommitted edits on `main`.
|
||||
4. Work from the other chat (the SDE Meeting Toolkit) does not touch this repo. See the toolkit `docs/context/` folder.
|
||||
|
||||
## 8. Related files
|
||||
|
||||
- `IMPLEMENTATION.md` (sections 2 and 4)
|
||||
- `docs/waves/wave-11.md`, `wave-12.md`, `wave-13.md` (on `feat/waves-11-13`), `wave-16.md`
|
||||
- `docs/waves/decisions-2026-09-17.md` (CR-019, CR-020, D18), `decisions-2026-10-02.md` (D19, D20, D21)
|
||||
- `docs/waves/notes-2026-09-23.md` (on `feat/waves-11-13`)
|
||||
- `docs/waves/backlog.md`
|
||||
51
docs/waves/decisions-2026-10-12.md
Normal file
51
docs/waves/decisions-2026-10-12.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Decisions: October 12, 2026 (meeting with Nick)
|
||||
|
||||
Topic: mobile app (offline Field View), batch print, and related scaling questions.
|
||||
Recorded by Matt Mabrey from the meeting with Nick, 2026-10-12. Follow-up answers from Matt are in the section "Follow-up answers".
|
||||
|
||||
This file records the answers. It does not schedule any work. The features that come from these answers are logged in `docs/waves/backlog.md` as `BL-062` to `BL-068` for a future build. The path to a mobile version that works outside the VPN is in `docs/waves/roadmap-mobile-scaling.md`. None of these have a wave or a `CR-` or `D-` id yet. Assign the ids when a wave is planned.
|
||||
|
||||
Question numbers (`Q2` to `Q14`) come from the decision table that Matt received. `Q1`, `Q10`, and `Q11` were not in the table. They are not recorded here.
|
||||
|
||||
## Answers
|
||||
|
||||
| Q | Decision needed | Default that was proposed | Answer |
|
||||
|---|---|---|---|
|
||||
| Q2 | Days the app may open offline after the last sign-in (today: 12 hours) | 7 days. Wipe a disabled account's data on the next online open. | **7 days.** The default wipe rule stands. |
|
||||
| Q3 | Must planners edit in the creator offline, or only crews in Field View? | Field View only | **Only crews in the field.** The creator needs no offline mode. |
|
||||
| Q4 | An offline edit meets a newer server version. What does the crew see? | Keep both, show differences, crew chooses. Server wins if nobody acts. | **Merge with a notification. Log it. Update the owner.** If the same field changed on both sides, **the field value wins**, and the app keeps a history of changes. This replaces the default. |
|
||||
| Q5 | Photo limit per photo and per package | Resize to 1280 px, 50 per package | **5 MB per photo.** |
|
||||
| Q6 | Maximum packages in one batch print | 50, with a message to narrow the filter | **25 on phones.** On desktop the user can scale the filter up to 50 or 100. |
|
||||
| Q7 | Where drawings go in the PDF, and at what size | After the form, upload order, scaled to fit | **After the form, upload order, scaled to fit.** Same as the default. |
|
||||
| Q8 | SVG drawings in the merged PDF | Leave out, with a page that names the file | **Leave out, with a page that names the file.** Same as the default. |
|
||||
| Q9 | What goes on every page | Package no., print date, page X of Y | **Package no., print date, page X of Y.** Same as the default. |
|
||||
| Q12 | Progress weight for Ready for QA (today 0, so a package drops from 75% to 0%) | 0.85 | **0.85.** Same as the default. |
|
||||
| Q13 | What happens to a package's drawings when the package is deleted | Delete with the package | **Keep them (data retention). The storage quota is freed.** Storage will move to cloud storage in a future deployment. This replaces the default. |
|
||||
| Q14 | Which cost-code list is correct (creator has 38, SOP has 39, 13 differ) | One list in the SOP. Needs codes from Accounting or the PM. | **One list in the SOP.** The **PM** supplies the codes. |
|
||||
|
||||
## Follow-up answers (Matt, 2026-10-12)
|
||||
|
||||
These answer the six questions that were open after the first record.
|
||||
|
||||
1. **Q5:** the 5 MB limit is **per photo**. This matches the limit that already exists for drawings (D8, enforced in `server/app.py`: 5 MB a file, 2 GB per project).
|
||||
2. **Q6:** 25 on phones. Desktop users can scale up (50 or 100).
|
||||
3. **Q4:** on a conflict in the same field, **the field value wins**. The app keeps a history of changes.
|
||||
4. **Q13:** the quota is freed. Cloud storage will hold the kept files in a future deployment.
|
||||
5. **Q14:** the **PM** gives the codes.
|
||||
6. **Scaling:** Matt asked for a path to a mobile version that scales to multiple projects, outside VPN access, for field agents on phones and tablets on site. That path is in `docs/waves/roadmap-mobile-scaling.md`.
|
||||
7. **Who uses it:** all end users will eventually be in the Okta group.
|
||||
8. **Scale:** expect multiple **customers**. The data model needs a customer (organization) level above a project. See Phase 4 of the roadmap.
|
||||
9. **Devices:** company-managed.
|
||||
|
||||
## Still open
|
||||
|
||||
1. **Q5:** the count of photos per package, and whether the app resizes a photo before upload. Phone photos are often 3 MB to 8 MB. A 5 MB cap per photo with no resize will reject some normal photos.
|
||||
2. **Q4:** who "the owner" is (package owner, project owner, or the person who made the server change). Also: how long the history of changes is kept, and who can read it.
|
||||
3. **Q13:** how long a kept drawing is retained, who can delete it for good, and who can recover it. The quota is freed, so the retained bytes must not count toward the 2 GB project limit. Decide where they are counted instead.
|
||||
4. **Q14:** the date the PM gives the codes.
|
||||
5. **Q6:** who sets the larger desktop limit (each user, or an admin setting).
|
||||
6. The questions in `roadmap-mobile-scaling.md`, section 7 (nine open). The first two (one person on more than one customer, and the isolation model) change the data design. Answer them first.
|
||||
|
||||
## Source
|
||||
|
||||
Meeting with Nick. Status column in the original table: "Open. Team builds to the default" for every row, as of October 12. The answers above came after that status.
|
||||
85
docs/waves/notes-2026-09-23.md
Normal file
85
docs/waves/notes-2026-09-23.md
Normal file
@@ -0,0 +1,85 @@
|
||||
# Session notes — 2026-09-23
|
||||
|
||||
Working notes for the `feat/waves-11-13` line (CR-019, CR-020 reserved, D18),
|
||||
written up before merge. Not a spec document — `wave-11.md`, `wave-13.md` and
|
||||
`decisions-2026-09-17.md` are the source of truth for scope and acceptance
|
||||
criteria. This is the "what actually happened building it" record.
|
||||
|
||||
## What shipped today
|
||||
|
||||
**CR-019 (wave 11) — usage/activity metrics, T11.1 through T11.6 complete:**
|
||||
|
||||
- `UsageEvent` table + migration (T11.1), a capture endpoint wired into
|
||||
`auth-guard.js` so every protected page pings it once per load, plus a
|
||||
`login` event at Okta sign-in (T11.2).
|
||||
- `GET /api/usage/summary` (T11.3) — active users by day/week/month,
|
||||
per-user last-active, per-tool breakdown, filterable by date/project/
|
||||
user/tool, all combinable.
|
||||
- `GET /api/usage/export` (T11.4) — raw and sanitized CSV. Sanitized mode
|
||||
replaces the username with an HMAC-SHA256 pseudonym (keyed with
|
||||
`auth.SECRET_KEY`), stable per user across rows and across separate
|
||||
export calls, so an external tool (Power BI etc.) can still group by
|
||||
user without ever seeing a real name.
|
||||
- A new "Activity & usage" card in the admin console (T11.5): real filter
|
||||
controls, the summary tables, both export buttons. Client-side gated
|
||||
admin-only same as the rest of the console; the API underneath is
|
||||
independently gated server-side regardless.
|
||||
- Retired the old per-browser "Usage logs" panel and `wp-usage.js`
|
||||
entirely (T11.6) — it had no reader left and was never a real data
|
||||
source for the new report anyway. The scattered `track()` call sites in
|
||||
the creator and wizard were left in place calling a documented no-op,
|
||||
rather than deleting ~45 individual call sites for the same effect.
|
||||
|
||||
Only **T11.7 (final wave verification)** is left before wave 11 is fully
|
||||
closed out — everything under it has already been verified per-task, so
|
||||
this is a consolidation pass, not new work.
|
||||
|
||||
**D18 (wave 13) — Okta/AD deprovisioning, T13.1 only:**
|
||||
|
||||
- Session lifetime changed from one flat `AUTH_SESSION_HOURS` to a sliding
|
||||
idle timeout (`AUTH_IDLE_MINUTES`, default 30) capped by a hard ceiling
|
||||
from original sign-in (`AUTH_SESSION_HOURS`, default 8, meaning changed
|
||||
from "session length" to "absolute ceiling"). Both defaults are flagged
|
||||
in `.env.example` and `DEPLOYMENT.md` as proposed, not confirmed against
|
||||
the tenant's actual Okta SSO policy.
|
||||
- **T13.2 onward (the actual Okta Management API sync job) is paused** —
|
||||
explicit call from Matt: no Okta API credential yet, come back to it
|
||||
later. Not started, not blocked on anything code-side.
|
||||
|
||||
**CR-020 (wave 12, bulk user editing):** not started. Reserved, scoped in
|
||||
`wave-12.md`, no code touched.
|
||||
|
||||
## Environment work (not itself a task, but load-bearing)
|
||||
|
||||
- Fixed a CRLF/LF mismatch that was making every tracked file look modified
|
||||
to this session's git client (`core.autocrlf true`, repo-local, no file
|
||||
content changed).
|
||||
- This machine had no Python. Installed it via `winget` (`Python.Python.3.12`)
|
||||
and set up a `.venv` in the repo with `server/requirements.txt` installed,
|
||||
specifically so `tests/baseline_shots.py` could run locally — this
|
||||
sandbox has no headless-capable browser and can't download one (network
|
||||
allowlist), so the 390px/1440px screenshot verification CLAUDE.md asks
|
||||
for had to run on Matt's own machine instead, using the browser already
|
||||
installed there (Edge).
|
||||
- Established a repeatable local verification loop for every task: throwaway
|
||||
SQLite, fake-Okta sign-in, promote to admin, `seed_demo.py` +
|
||||
`smoketest.py` (27/27 passing throughout), plus `baseline_shots.py` for
|
||||
anything touching `html/`.
|
||||
|
||||
## Standing constraints, still in effect
|
||||
|
||||
- Everything stays local on this branch line. No `git push` at any point
|
||||
today.
|
||||
- One task per PR discipline was kept even though all of it landed on one
|
||||
branch — each commit corresponds to exactly one task ID, in wave order,
|
||||
each individually verified before the next started.
|
||||
|
||||
## Before merging
|
||||
|
||||
- Run T11.7 (full wave-11 verification pass) and record it in `wave-11.md`.
|
||||
- Decide where this branch actually merges to — `feat/waves-11-13` has all
|
||||
of today's commits already; this notes branch was cut from it so it can
|
||||
fast-forward back in, or merge as its own PR if the notes should be
|
||||
reviewed separately from the code.
|
||||
- T13.2+ and all of wave 12 remain explicitly out of scope until Matt says
|
||||
otherwise.
|
||||
178
docs/waves/roadmap-mobile-scaling.md
Normal file
178
docs/waves/roadmap-mobile-scaling.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# Roadmap: mobile version, multiple projects, access outside the VPN
|
||||
|
||||
Date: 2026-10-12
|
||||
Status: plan for a future build. It is not a spec. No wave, `CR-`, or `D-` id exists for it yet.
|
||||
Inputs: `docs/waves/decisions-2026-10-12.md` (Q2 to Q14), `BL-062` to `BL-068` in `docs/waves/backlog.md`, and the code as of `main` (`d42ae0c`).
|
||||
|
||||
Goal (from Matt, 2026-10-12): field agents use the app on phones and tablets on site, from outside the VPN, across more than one project.
|
||||
|
||||
Scope facts confirmed by Matt on 2026-10-12:
|
||||
- All end users will be in the Okta group. There are no contractors without an Okta account.
|
||||
- The product will scale to **multiple customers**, not only more projects for one customer.
|
||||
- The devices will be **company-managed**.
|
||||
|
||||
## 1. Where the product is today
|
||||
|
||||
| Area | Fact today | Source |
|
||||
|---|---|---|
|
||||
| Mobile shell | A PWA already exists: `manifest.webmanifest` (standalone, "Field View" shortcut) and `sw.js` (caches the app shell and uploaded drawings). | `html/manifest.webmanifest`, `html/sw.js` |
|
||||
| Offline writes | Writes queue in a sync outbox in `project-data.js`. Reads fall back to a `localStorage` cache. `/api/*` is never cached by the service worker. | `html/sw.js` header |
|
||||
| Offline drawings | Drawings for the user's assigned packages are prefetched and cached (CR-007, D8). | `html/sw.js`, `html/field.js` |
|
||||
| Files | The bytes live in the Postgres `wp_files` table. Limit: 5 MB a file, 2 GB per project. | `server/models.py` |
|
||||
| Sign-in | Okta OIDC. A signed cookie (`HttpOnly`, `SameSite=Lax`, `Secure` over HTTPS). Idle timeout (D18, T13.1) is on `feat/waves-11-13`, not on `main`. | `server/auth.py`, `docs/waves/wave-13.md` |
|
||||
| Projects | `Project` and `ProjectMember` exist. Access is by project role. There is no organization or customer level above the project. | `server/models.py` |
|
||||
| Hosting | Docker Compose on an internal host. NGINX serves the site. An external proxy ends TLS. The database and backup containers have no internet route. | `docker-compose.yml`, `nginx/conf.d/wp-suite.conf` |
|
||||
| Asset picker | Reads the Micron SQL Server catalog through `MICRON_DB_URL`. That database sits on the company network. | `server/assets_db.py` |
|
||||
| Abuse limits | No rate limiting or login throttling was found in `server/`. CSRF origin checks and security headers (CSP, HSTS) exist. | `server/app.py`, `wp-suite.conf` |
|
||||
| Security review | The 2026-10-01 review put session revocation (SEC-06) on hold until the AWS work. Wave 16 fixed SEC-01, SEC-02, SEC-07. | `docs/waves/decisions-2026-10-02.md` |
|
||||
|
||||
Result: the "mobile app" is mostly built as a PWA. The main work is not a new app. It is (a) finishing offline Field View, (b) making the service safe to put on the internet, and (c) removing the single-host limits.
|
||||
|
||||
## 2. Path in phases
|
||||
|
||||
Each phase has a gate. Do not start a phase before the gate of the one before it is met, unless this file says the phases are independent. Sizes are rough estimates (S under 1 week, M 1 to 3 weeks, L over 3 weeks). They are not commitments.
|
||||
|
||||
### Phase 0: Close the open decisions (S)
|
||||
|
||||
- Answer the questions in section 7 and the "Still open" list in `decisions-2026-10-12.md`.
|
||||
- Assign `CR-`/`D-` ids and write the wave files.
|
||||
- **Gate:** every item in `BL-062` to `BL-068` is free of "to confirm".
|
||||
|
||||
### Phase 1: Finish offline Field View, still inside the VPN (L)
|
||||
|
||||
Scope: `BL-062`, `BL-063`, `BL-064`.
|
||||
|
||||
1. Land the outbox work (wave 14, T14.1 to T14.3). `wave-12.md` says T13.1 must not reach `main` before T14.1.
|
||||
2. Set the offline window to 7 days (Q2). Reconcile it with the D18 idle timeout and absolute ceiling. Define what "signed in" means offline. Decide what the 7 day window does when Okta says the account is disabled (wipe on the next online open).
|
||||
3. Conflict handling (Q4): merge, field value wins on the same field, history of changes, a notification to the crew, a log entry, and an update to the owner.
|
||||
4. Photos: capture on the device, 5 MB per photo (Q5). Add resize before upload if Matt chooses it. Store photos like drawings (server-side, with D19 download headers).
|
||||
5. Add a phone and tablet test matrix (real devices): iOS Safari installed to the home screen, Android Chrome, one tablet.
|
||||
6. Test the weak-link case: no signal, then a slow signal, then a sync with a conflict.
|
||||
|
||||
Known platform risks to test on the devices (verify; do not assume):
|
||||
- iOS may delete site storage after a period of no use unless the PWA is installed to the home screen.
|
||||
- iOS Safari has no Background Sync. The outbox then sends only while the app is open. A crew member must open the app to push changes.
|
||||
- A lost device holds up to 7 days of project data. Web storage is not encrypted by the app. See Phase 2.
|
||||
|
||||
**Gate:** the crew completes a full shift offline on a test device, reconnects, and the server holds every change. The conflict case works as Q4 says. `tests/mobile_check.py` and the 390px/1440px checks pass.
|
||||
|
||||
### Phase 2: Make the service safe to reach from the internet (M to L)
|
||||
|
||||
Today the design assumes the firewall. Outside the VPN, the app becomes the perimeter. Do this before any real user connects from the internet.
|
||||
|
||||
| Control | Why |
|
||||
|---|---|
|
||||
| Rate limiting and login throttling on the API (none today) | Stops guessing and scraping. The toolkit (`sde-meeting-toolkit`) needed the same fix. |
|
||||
| A WAF or reverse proxy with request limits in front of NGINX | Filters bad requests before they reach the API. |
|
||||
| Real client IP handling (`X-Forwarded-For` from the trusted proxy only) | Limits and logs must use the real address, not the proxy. |
|
||||
| Okta policy for the app: MFA, session length, device trust | Okta is the only sign-in path (D15, D16). The policy is the control. Ask the Okta admin. |
|
||||
| Session revocation (SEC-06) | When a phone is lost, an admin must end that session now. This was put on hold for the AWS work. Bring it forward. |
|
||||
| A rule for lost devices | Devices are company-managed (decided). Require a device PIN and disk encryption through the device policy. Remote wipe through the device manager. Pair it with session revocation (above), because a wipe cannot reach an offline device. Confirm with IT that the policy covers every phone and tablet that runs the app, and that a lost device is reported quickly. |
|
||||
| Okta access rule | All end users are in the Okta group (decided). Gate the app on that group. Today the API creates the account on first sign-in; check that a person who is not in the group is refused, and that removal from the group ends access (this is what D18 and wave 13 do). Ask the Okta admin to add MFA and device-trust conditions for the outside path. |
|
||||
| Cookie flags on every cookie (the OIDC state cookie uses `https_only=False` in `server/app.py`) | Must be `Secure` on the public site. |
|
||||
| Review the 2026-10-01 findings that stayed in backlog drafts | Re-rank them for an internet-facing service. |
|
||||
| Outside review (a penetration test) | Required before the customer-IP data is reachable from the internet. |
|
||||
| Alerting on failed sign-ins and on odd upload sizes | Detect abuse early. |
|
||||
|
||||
**Gate:** the controls above are in place and an outside test finds no high findings open.
|
||||
|
||||
### Phase 3: Hosting that can leave the office network (L)
|
||||
|
||||
The AWS work is already planned (SEC-06 note). Proposed shape. The choice of services is open for Matt and the AWS owner.
|
||||
|
||||
- A public edge (load balancer, WAF, TLS) in front of the API and the static site.
|
||||
- API containers on a managed container service, with a managed Postgres database.
|
||||
- Files in object storage instead of the database (`BL-066`, Q13). Move the bytes out of `wp_files`. Keep the metadata in Postgres. Kept files from deleted packages go to a retention area with a lifecycle rule. The quota counts only live files.
|
||||
- Backups and restore tests on the new stack (the current backup container writes to a host folder).
|
||||
- A private link back to the Micron asset database, or a different design for the asset picker (the catalog is on the company network). Until this exists, the picker falls back to manual entry outside the VPN. Check that fallback works for a user who is online but cannot reach the database (the code treats "unconfigured" as a normal state, but not "unreachable").
|
||||
- Infrastructure as code, so the environment can be rebuilt.
|
||||
- Keep the internal deployment working until the cutover.
|
||||
|
||||
**Gate:** a full restore test passes, the migration runs on Postgres (render the SQL for the Postgres dialect first, BL-027), and a rollback plan is written and rehearsed.
|
||||
|
||||
### Phase 4: Many projects and many customers (L)
|
||||
|
||||
Matt confirmed multiple customers. A customer boundary is now part of the plan, not an option. The data model has `Project` and `ProjectMember` but nothing above a project, so this phase changes the base design. Plan for it early. Phase 3 choices (storage layout, database layout) must already allow it.
|
||||
|
||||
- Add an organization (customer) level above `Project`. Every project, user role, file, report, export, and search belongs to one organization. A user may belong to more than one organization (for example Prime Controls staff who work for several customers). Decide this rule first (section 7, question 1).
|
||||
- Isolation: choose the model before any code. Options: one database with an organization id on every row and checks on every query; a separate schema per customer; or a separate database or stack per customer. Matt and the AWS owner decide. The choice sets cost, backup, and the strength of the separation that a customer can be shown.
|
||||
- Files: one storage prefix per customer, with its own access rule and, if a customer asks, its own encryption key.
|
||||
- Customer-specific data: the cost-code list, the asset catalog (today Micron only), and the SOP templates must belong to the organization or the project. `BL-068` (one cost-code list in the SOP) must not become one list for all customers.
|
||||
- Okta: map Okta groups to organizations and projects, so the Okta admin controls who sees which customer.
|
||||
- Reports and the admin console: super users see only the organizations they are given. Check CR-019 usage reports, the user directory, and the bulk user tools from wave 12 against this rule.
|
||||
- Onboarding a customer: a runbook with the steps and a checklist.
|
||||
- Project onboarding: create a project, load its SOP, cost codes (`BL-068`, one list in the SOP, codes from the PM), people, and roles. Use the bulk user tools from wave 12.
|
||||
- Okta group to project membership mapping, so the Okta admin controls who joins a project.
|
||||
- Per-project storage and photo quotas (2 GB per project exists today) and per-project usage reports (CR-019 on `feat/waves-11-13`).
|
||||
- Performance work with a load test: many crews syncing at shift start. Check the outbox flush, the `wp_files` move, and the list endpoints on a large project.
|
||||
- Data retention rules per project.
|
||||
|
||||
**Gate:** two real customers run side by side. A test user in one customer cannot reach any data of the other through any screen, report, export, search, file link, or API call. A test that tries to cross the boundary on purpose passes. The load test meets targets that Matt sets.
|
||||
|
||||
### Phase 5: Native wrapper (only if needed) (M)
|
||||
|
||||
Wrap the PWA in a native shell (for example Capacitor) only if Phase 1 testing shows a platform limit that the PWA cannot meet: storage eviction, no background sync, camera control, or a company rule for app-store distribution. A wrapper adds app-store review, signing, and update cost. Do not start it by default.
|
||||
|
||||
## 3. Order and dependencies
|
||||
|
||||
```
|
||||
Phase 0 -> Phase 1 -> Phase 2 -> Phase 3 -> Phase 4
|
||||
\_ Phase 5 (decide after Phase 1 device tests)
|
||||
```
|
||||
|
||||
- Phase 2 may start in part while Phase 1 runs (rate limiting, Okta policy review). No crew uses the internet path before Phase 2 is done.
|
||||
- Phase 3 (storage move) and `BL-066` (retained drawings) are the same work. Do them together.
|
||||
- Phase 4 needs the Phase 2 and Phase 3 gates. Opening a second project to the internet first would repeat any gap twice.
|
||||
|
||||
## 4. Work that this path adds to the backlog
|
||||
|
||||
| Id | Item | Phase |
|
||||
|---|---|---|
|
||||
| BL-062 to BL-068 | Already logged | 1, 3, 4 |
|
||||
| New | API rate limiting and login throttling | 2 |
|
||||
| New | Real client IP handling behind the proxy | 2 |
|
||||
| New | Session revocation (SEC-06) brought forward | 2 |
|
||||
| New | Device policy check with IT (PIN, encryption, remote wipe) | 1, 2 |
|
||||
| New | Okta group gate and MFA/device-trust conditions for the outside path | 2 |
|
||||
| New | Object storage and lifecycle rules for files, one prefix per customer | 3 |
|
||||
| New | Asset picker when the Micron database is unreachable | 3 |
|
||||
| New | Organization (customer) level above `Project`, with the isolation model chosen first | 3, 4 |
|
||||
| New | Per-customer cost codes, asset catalog, and SOP templates | 4 |
|
||||
| New | Okta group to organization and project mapping | 4 |
|
||||
| New | Cross-customer access tests (a test that tries to cross the boundary) | 4 |
|
||||
| New | Customer onboarding runbook | 4 |
|
||||
| New | Load test and capacity targets | 4 |
|
||||
|
||||
Log these in `docs/waves/backlog.md` when Matt agrees with the plan. This file does not add them, so the backlog stays free of unconfirmed scope.
|
||||
|
||||
## 5. Risks
|
||||
|
||||
1. **Customer data on lost devices.** Offline data for 7 days on a phone or tablet. Devices are company-managed, which lowers this risk a lot, but only if the device policy (PIN, disk encryption, remote wipe) is real and covers every device. An offline device cannot receive a wipe until it connects. Keep the 7 day window only if IT confirms the policy.
|
||||
2. **Customer boundary added late.** The data model has no organization level. If Phases 1 to 3 are built without it, every later change touches every table and every query. Choose the isolation model in Phase 0 and design Phase 3 storage around it.
|
||||
3. **Contract terms.** Customer drawings and asset data may have rules about where they are stored and who can reach them. With several customers, each contract may differ. Check before Phase 3 (cloud storage) and before Phase 2 (internet access).
|
||||
4. **Okta.** The app has no password of its own. A weak Okta policy for the outside path is a weak app. The Okta admin owns that setting.
|
||||
5. **Single database host.** The current database is one container with a host-folder backup. Phase 3 fixes this. Until then, do not add projects that need high availability.
|
||||
6. **Sequence risk.** T13.1 (idle timeout) must wait for T14.1 (outbox). A short idle timeout can end a session while a crew is offline. Settle the rule in Phase 1.
|
||||
|
||||
## 6. What does not change
|
||||
|
||||
These decisions stay as recorded: Actual Hours stays (CR-017), localization stays (A7), uppercase card headers stay (A5), the logged-override path for predecessors stays (A1), removed fields are hidden and not deleted (CR-002, CR-016), and the token rule for CSS. Mobile work follows the accessibility rules (C1): real buttons, visible focus, `aria-live` for status, hover is not enough on tablets.
|
||||
|
||||
## 7. Questions that decide the plan
|
||||
|
||||
Answered on 2026-10-12:
|
||||
- Who are the outside users? **All end users will be in the Okta group.** No separate sign-in design is needed.
|
||||
- What does "multiple projects" mean? **Multiple customers.** See Phase 4.
|
||||
- Which devices? **Company-managed.**
|
||||
|
||||
Answer these before Phase 0 ends. The first four are new and follow from the answers above.
|
||||
|
||||
1. **Can one person work for more than one customer?** For example, a Prime Controls planner on two customers' projects. This sets how the organization level and the Okta groups map to each other.
|
||||
2. **Which isolation model?** One database with an organization id on every row, a schema per customer, or a database or stack per customer. Do any customers already require a separate stack in their contract?
|
||||
3. **Who manages the Okta groups?** One group for all users, or one group per customer or project? Does the Okta admin accept that mapping?
|
||||
4. **What does the device policy enforce today?** PIN, disk encryption, remote wipe, and a time limit for a device to check in. Ask IT.
|
||||
5. **Connectivity on site.** No signal for days, or weak signal that comes and goes? This sets how long the offline window must really be, and how big the photo queue gets.
|
||||
6. **AWS.** Is the AWS move approved? Who owns the account? Which region? Can the Micron database be reached from it?
|
||||
7. **Contract.** Can customer drawings and asset data be stored in cloud storage and reached from the internet? Does each customer have its own terms?
|
||||
8. **Distribution.** Is an installed PWA enough, or does the company need an app-store app? Company-managed devices can often install a web app from a device policy. Ask IT.
|
||||
9. **Targets.** How many customers, projects, and users, and how many crews syncing at the same time, in the first year?
|
||||
280
docs/waves/wave-11.md
Normal file
280
docs/waves/wave-11.md
Normal file
@@ -0,0 +1,280 @@
|
||||
# Wave 11 — Usage and activity metrics
|
||||
|
||||
**Items:** `CR-019`
|
||||
**Depends on:** wave 10 merged (it is; this wave does not wait on anything else)
|
||||
**Decision record:** `docs/waves/decisions-2026-09-17.md`
|
||||
|
||||
Seven tasks, one concern each, in build order. Do not start a task whose
|
||||
dependency is not merged. `CR-020` (bulk user editing) is reserved but not
|
||||
scoped — it does not belong in this wave.
|
||||
|
||||
---
|
||||
|
||||
### T11.1 — CR-019: `usage_events` table + migration
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** nothing (first task)
|
||||
- **Blocks:** T11.2
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/models.py`, `server/alembic/versions/`
|
||||
|
||||
**Do:** Add a `UsageEvent` model — append-only, same spirit as `AuditLog` but for
|
||||
navigation/feature-open events rather than business mutations. Suggested shape:
|
||||
`id`, `at` (indexed), `user_id` (or username — match whatever `AuditLog.actor`
|
||||
does today for consistency), `project_id` (nullable — not every event is
|
||||
project-scoped, e.g. opening the admin console), `tool` (e.g. `creator`,
|
||||
`wizard`, `field_view`, `dashboard`, `admin`, `directory`), `event` (e.g.
|
||||
`page_open`, `login`), `detail` (JSON, optional). Write the migration. Do not
|
||||
touch `AuditLog` — this is a new table, not an extension of it (see the
|
||||
decision record's reasoning).
|
||||
|
||||
**Do not:** fold this into `AuditLog`. They serve different questions and mixing
|
||||
them makes the existing audit trail noisier for its existing readers.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] `UsageEvent` exists with an indexed `at` column (this table will be scanned
|
||||
by date range constantly)
|
||||
- [ ] migration applies cleanly against both SQLite (dev) and Postgres (prod) —
|
||||
render `alembic upgrade --sql` for postgresql and read it before calling
|
||||
this done, per the class of defect `BL-027` logged
|
||||
- [ ] no change to `AuditLog`'s shape or behavior
|
||||
|
||||
---
|
||||
|
||||
### T11.2 — CR-019: capture the events
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.1
|
||||
- **Blocks:** T11.3
|
||||
- **Surface:** `server/` + `html/`
|
||||
- **Files:** `server/app.py` (new endpoint), `html/auth-guard.js`
|
||||
|
||||
**Problem:** Six pages need this and none should implement it separately — that
|
||||
is exactly how `S4`'s "no global nav on two pages" and the four parallel token
|
||||
systems (`S5`) happened. `auth-guard.js` is already loaded first, in the `<head>`,
|
||||
on all six protected pages (`index.html`, `field.html`, `users.html`,
|
||||
`wp-creation-index.html`, `work-package-suite.html`, `admin.html`) and already
|
||||
knows the verified user once the `wp-auth-ready` event fires. That is the one
|
||||
place this belongs.
|
||||
|
||||
**Do:** Add a small `POST /api/usage/ping`-style endpoint that writes one
|
||||
`UsageEvent` row per call, keyed to the session (server trusts the session, not
|
||||
anything the client claims about identity). Call it once from `auth-guard.js`
|
||||
after `wp-auth-ready`, tagging `tool` from the page's own path. Also write a
|
||||
`login` event at the point a session is actually established (reuse whatever
|
||||
`okta_callback` already does at sign-in — do not add a second source of truth
|
||||
for "did this person log in").
|
||||
|
||||
**Do not:** build a per-page capture call. If a page needs this and
|
||||
`auth-guard.js` does not cover it, fix `auth-guard.js`, not the page.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] one `page_open` event is recorded for a real sign-in on each of the six
|
||||
pages, verified per page
|
||||
- [ ] exactly one `login` event per Okta sign-in, not one per page load after
|
||||
it
|
||||
- [ ] the endpoint rejects a request with no valid session (this is server-
|
||||
enforced identity, not client-reported)
|
||||
- [ ] no page other than `auth-guard.js` calls this endpoint directly
|
||||
|
||||
---
|
||||
|
||||
### T11.3 — CR-019: aggregation endpoint with filters
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.2
|
||||
- **Blocks:** T11.4, T11.5
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/app.py`
|
||||
|
||||
**Do:** Build the read side: active-user counts by day/week/month, per-user
|
||||
last-active timestamp (derived from `UsageEvent`, not `User.last_login_at`,
|
||||
which only ever holds one value), and a per-tool usage breakdown. Accept query
|
||||
filters: date range, project, user, tool — combinable, per the decision record.
|
||||
This is server aggregation, the same principle `B4` established for the
|
||||
pipeline strip: the browser asks for a number, the server computes it from real
|
||||
rows, nothing is derived client-side from a partial cache.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] active-user counts are correct against a seeded fixture with known dates
|
||||
- [ ] filters combine correctly (verified: user + date range + tool together
|
||||
narrows correctly, not just each alone)
|
||||
- [ ] a project filter that matches nothing returns an empty result, not an
|
||||
error or the unfiltered total
|
||||
|
||||
---
|
||||
|
||||
### T11.4 — CR-019: export, raw and sanitized
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.3
|
||||
- **Blocks:** T11.5
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/app.py`
|
||||
|
||||
**Do:** A CSV export endpoint over the same filtered query T11.3 exposes.
|
||||
Two modes: raw (real usernames, the console's default) and sanitized. Sanitized
|
||||
mode replaces the actor field with a stable pseudonymous id — a per-user hash,
|
||||
consistent across rows in the same export and across separate export runs —
|
||||
so an external system (Power BI or similar) can still group and trend "by
|
||||
user" without ever receiving a real name. Do not simply drop the identity
|
||||
column; that breaks per-user grouping downstream, which defeats the point of
|
||||
an activity export.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] raw export contains real usernames
|
||||
- [ ] sanitized export never contains a real username or email anywhere in the
|
||||
file, including in a `detail` blob if one is included
|
||||
- [ ] the same real user maps to the same pseudonymous id within one export AND
|
||||
across two separate export runs (a hash of something stable, not a
|
||||
per-request random id)
|
||||
- [ ] both modes otherwise contain identical rows for the same filter
|
||||
|
||||
---
|
||||
|
||||
### T11.5 — CR-019: admin console Activity tab
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.3, T11.4
|
||||
- **Blocks:** T11.7
|
||||
- **Surface:** `html/`
|
||||
- **Files:** `html/admin.html`, `html/admin.js`
|
||||
|
||||
**Do:** New tab, same role gate as the User Directory. Filters (date range,
|
||||
project, user, tool) driving the tables from T11.3; export buttons (raw and
|
||||
sanitized) calling T11.4. Build accessible from the start per `C1` — this is a
|
||||
new component, not a legacy one carrying an old defect forward: real
|
||||
`<button>`/`<select>` controls, keyboard-reachable, `aria-live` on any
|
||||
count that updates without a page reload, focus visible throughout.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] the tab is reachable only by an admin (the card lives inside admin.html,
|
||||
already gated client-side by gateByRole(); the API underneath it is
|
||||
independently gated server-side by require_user_manager regardless)
|
||||
- [x] every filter is a real form control, keyboard-operable (date/select/text
|
||||
inputs and a `<button>`, no click-div)
|
||||
- [x] both export buttons produce the files T11.4 defines (verified against
|
||||
the live endpoint in T11.4's own checks, and present/wired here)
|
||||
- [x] works at 390px and 1440px — verified 2026-09-23 via
|
||||
`tests/baseline_shots.py --pages admin` run locally on Windows (this
|
||||
sandbox has no headless browser available; the script was run on the
|
||||
user's machine instead, after installing Python via winget since it
|
||||
wasn't present). Screenshots in `docs/reference/baseline/admin-390.png`
|
||||
/ `admin-1440.png`. No JS errors, no horizontal overflow at either
|
||||
width; the card rendered with real seeded data (events, by-tool,
|
||||
per-user-last-active tables) confirming the filters and summary read
|
||||
correctly, not just that the markup exists.
|
||||
|
||||
---
|
||||
|
||||
### T11.6 — CR-019: retire the per-browser Usage report
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.5
|
||||
- **Blocks:** T11.7
|
||||
- **Surface:** `html/`
|
||||
- **Files:** `html/admin.js` (the `usage-admin` panel, `admin.js:666-699`),
|
||||
`html/wp-usage.js` and its two call sites
|
||||
|
||||
**Do:** Remove the old per-browser `usage-admin` panel from `admin.js` now that
|
||||
the real one exists, per the 2026-09-17 decision. Decide what happens to
|
||||
`wp-usage.js`'s recording calls (wizard dwell-tracking, the creator's
|
||||
equivalent): they were never reliably tied to a real identity, so they are not
|
||||
a data source the new report can adopt. Default to removing the recorder too
|
||||
unless it is still doing something useful on its own (re-read what it actually
|
||||
records before deciding — do not assume from this file alone).
|
||||
|
||||
**Do not:** leave the old panel in place "just in case." Two activity reports
|
||||
showing two different numbers is worse than one.
|
||||
|
||||
**Decision (2026-09-23):** `wp-usage.js` is removed, not kept — it had no
|
||||
reader left once the admin panel above it was removed (the "download the
|
||||
full event log" button lived only in that panel), and per the original
|
||||
decision record it was never reliably tied to a real identity, so it was
|
||||
never a candidate source for the new report either. The file itself and its
|
||||
three `<script>` includes (`admin.html`, `wp-creation-index.html`,
|
||||
`work-package-suite.html`) are gone. Its two call sites
|
||||
(`wp-creation-app.js`, `work-package-suite-app.js`) keep a local `track()`
|
||||
function as a documented no-op rather than having each of their ~45
|
||||
individual `track('event', …)` call sites deleted one at a time — that
|
||||
would be a far larger, riskier diff for the same outcome (no more data is
|
||||
recorded either way), and it keeps each call site as a marker of what was
|
||||
worth recording if usage analytics are ever rebuilt server-side. The
|
||||
dwell-timer plumbing that only ever fed `track()` (`work-package-suite-app.js`'s
|
||||
`_stepEnter`/`trackStepDwell`) was left in place for the same reason — it is
|
||||
inert now, not broken, and touching it buys nothing.
|
||||
|
||||
Also removed: `tests/usage_check.py` (tested exactly the retired feature —
|
||||
D5/T7.10's per-browser analytics core and admin report) and its line in
|
||||
`docs/reference/file-map.md`, replaced with a note pointing at this decision.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] the old `usage-admin` panel and its markup are gone from `admin.html`/
|
||||
`admin.js`
|
||||
- [x] a decision on `wp-usage.js` itself is recorded (removed, or kept with a
|
||||
stated reason) — not left ambiguous — see above
|
||||
- [x] nothing else in the app references the removed code; grep confirms
|
||||
(only remaining hits are this file, the 2026-09-17 decision record, and
|
||||
the two explanatory code comments left at the retired call sites — all
|
||||
prose, not live references)
|
||||
|
||||
---
|
||||
|
||||
### T11.7 — CR-019: verification
|
||||
|
||||
- **Items:** `CR-019`
|
||||
- **Depends on:** T11.6
|
||||
- **Blocks:** nothing
|
||||
- **Surface:** `html/` + `server/`
|
||||
- **Files:** as touched above
|
||||
|
||||
**Do:** Full verification per `CLAUDE.md`: run the app locally, exercise the
|
||||
new tab at 390px and 1440px, before/after screenshots, run the existing smoke
|
||||
test and `seed_demo.py`, run the full suite.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] all `CR-019` acceptance criteria in `decisions-2026-09-17.md` are met or
|
||||
a failure is stated with a reason — all met
|
||||
- [x] screenshots committed — `admin-390.png`/`admin-1440.png` (and
|
||||
`creator`/`sop` at both widths, touched by T11.6's script-tag removal)
|
||||
- [x] smoke test and `seed_demo.py` both still pass — 27/27, verified
|
||||
2026-09-24 against the exact branch state about to be considered
|
||||
deployable
|
||||
- [x] full test suite passes — verified 2026-09-24: ran all 39
|
||||
`tests/*_check.py` scripts plus the `usage_events` migration
|
||||
round-trip. Result: 38/39 exit 0.
|
||||
- `token_check.py` exits 2 with no arguments by design (needs `--out`
|
||||
or `--compare`) — a CLI-invocation detail, not a test failure;
|
||||
already confirmed non-regression in wave 10.
|
||||
- `generalinfo_check.py` has one pre-existing failure (a raw colour
|
||||
literal in `html/wp-creation-styles.css`, a file no wave 11/13 task
|
||||
touched) — already diagnosed as pre-existing in wave 10.
|
||||
- `autosave_check.py` **did** regress from T11.6 (it asserted the
|
||||
retired analytics tracker still wrote to localStorage) — found by
|
||||
this run, fixed same day (commit `4f3a94b`), now 33/33.
|
||||
- Migration (T11.1): SQLite upgrade → downgrade → upgrade round-trip
|
||||
clean; `alembic upgrade ... --sql` against the `postgresql` dialect
|
||||
renders valid DDL (`TIMESTAMP WITH TIME ZONE`, `JSON`, no
|
||||
SQLite-only syntax) — no `BL-027`-class defect.
|
||||
|
||||
---
|
||||
|
||||
## Wave 11 exit criteria
|
||||
|
||||
- [x] real, server-side activity data exists per user, indefinitely retained
|
||||
- [x] the admin console shows it, filterable by date/project/user/tool
|
||||
- [x] export works in both raw and sanitized form
|
||||
- [x] the old per-browser report is gone, not duplicated
|
||||
- [x] `CR-019` fully accounted for, no open acceptance criteria
|
||||
|
||||
Wave 11 is closed out as of 2026-09-24. `CR-020` (wave 12) remains reserved
|
||||
and unstarted. `D18`/wave 13 (`T13.2` onward) remains paused pending an Okta
|
||||
API credential, per standing instruction — `T13.1`'s session changes were
|
||||
included in this same verification pass and did not regress anything above.
|
||||
330
docs/waves/wave-13.md
Normal file
330
docs/waves/wave-13.md
Normal file
@@ -0,0 +1,330 @@
|
||||
# Wave 13 — Okta/AD deprovisioning sync
|
||||
|
||||
**Items:** `D18`
|
||||
**Depends on:** wave 10 merged (it is). Not blocked by wave 11 or wave 12, but
|
||||
shares `server/app.py` account-state surface with wave 12 (`CR-020`) — sequence
|
||||
commits to avoid an avoidable conflict.
|
||||
**Decision record:** `docs/waves/decisions-2026-09-17.md`
|
||||
|
||||
Six tasks, in build order. **T13.3's fail-closed behavior is the most
|
||||
important done-when list in this wave — do not relax it to ship faster.**
|
||||
|
||||
---
|
||||
|
||||
### T13.1 — D18: idle timeout with an absolute ceiling
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** nothing (first task, independent of the rest)
|
||||
- **Blocks:** nothing
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/auth.py`, `server/app.py` (`auth_gate` middleware),
|
||||
`server/.env.example`, `DEPLOYMENT.md`
|
||||
|
||||
**Revised 2026-09-23** — originally just "shrink `AUTH_SESSION_HOURS`". Matt
|
||||
asked whether an idle timeout would be a better fit than a flat session
|
||||
length. It is, but not by itself — see the decision record's reasoning on why
|
||||
idle-with-no-ceiling is actually a worse fit for this item's own threat model
|
||||
than a flat expiry would have been. Build both.
|
||||
|
||||
**Do:**
|
||||
|
||||
- Add `login_at` to the JWT payload in `create_token()` — the original
|
||||
sign-in time, distinct from `iat`, which becomes "when THIS token was
|
||||
issued" once tokens start getting reissued. `login_at` never changes across
|
||||
reissues; it's what the absolute ceiling is measured from.
|
||||
- Add `AUTH_IDLE_MINUTES` (default 30). `AUTH_SESSION_HOURS` stays the name
|
||||
for the absolute ceiling, default changing from 12 to a proposed 8 — update
|
||||
its docstring/comment in `auth.py` and `.env.example`, since its MEANING is
|
||||
changing (session length -> hard ceiling on top of a sliding idle window),
|
||||
not just its value.
|
||||
- In `auth_gate` (`server/app.py`), after confirming a request is
|
||||
authenticated: if `now < login_at + AUTH_SESSION_HOURS` (the ceiling hasn't
|
||||
passed), compute `new_exp = min(now + AUTH_IDLE_MINUTES, login_at +
|
||||
AUTH_SESSION_HOURS)`. If `new_exp` is meaningfully later than the current
|
||||
token's `exp` (throttle this — do not reissue on every single request, only
|
||||
when enough time has passed to be worth a new cookie; a few minutes of
|
||||
slack is fine), mint a refreshed token carrying forward `sub`/`username`/
|
||||
`role`/`ver`/`login_at` unchanged, and set it on the response.
|
||||
- If the ceiling HAS passed, do not refresh — let the existing token expire
|
||||
on its own terms (it may already be invalid, or may tick over within the
|
||||
idle window; either way, no new one is issued past the ceiling).
|
||||
- The middleware should not need a DB round trip to do this — everything
|
||||
needed (`sub`, `username`, `role`, `ver`, `login_at`) is already in the
|
||||
validated claims. `get_current_user`'s existing per-request `is_active`/
|
||||
`token_version` check is unaffected and still runs separately.
|
||||
|
||||
**Do not:** reissue the cookie on every request unconditionally — that's a
|
||||
`Set-Cookie` header on every API call for no benefit over a coarser refresh.
|
||||
Do not let the ceiling check silently vanish for tokens issued before this
|
||||
change ships — a token with no `login_at` claim should fall back to treating
|
||||
its own `iat` as `login_at`, not bypass the ceiling entirely.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] a session with continuous activity stays alive past 30 minutes but is
|
||||
cut off at the `AUTH_SESSION_HOURS` ceiling regardless
|
||||
*(2026-10-16, T13.6: a request every minute for 10 hours: alive well past 30 minutes, never extended past login + 8 h, cut off at the ceiling, original `login_at` kept.)*
|
||||
- [x] a session with no activity for 30+ minutes is rejected on its next
|
||||
request
|
||||
*(2026-10-16, T13.6: an idle token gets 401 on a gated route; one inside the window gets 200.)*
|
||||
- [x] the cookie is not rewritten on every single request — verify the
|
||||
refresh is throttled, not unconditional
|
||||
*(2026-10-16, T13.6: 20 rapid requests on a fresh session: 0 rewrites; 480 requests over 8 h: about one rewrite per 10 minutes; a refreshed cookie is not rewritten again on the next request. Mutation test: removing the throttle fails 3 checks.)*
|
||||
- [x] a pre-existing token with no `login_at` claim (simulating a session
|
||||
issued before this shipped) still gets a hard ceiling, via the `iat`
|
||||
fallback
|
||||
*(2026-10-16, T13.6: a token with iat 9 h ago and a still-valid old exp is not extended; one with iat 1 h ago is refreshed with `login_at` taken from `iat`; no `login_at` and no `iat` is not refreshed and does not crash. Mutation test: giving it a fresh ceiling fails.)*
|
||||
- [x] `.env.example` and `DEPLOYMENT.md` explain both variables and flag both
|
||||
defaults as proposed, not confirmed against Okta's own session policy
|
||||
*(2026-10-16, T13.6: both variables are in both files, and both call the defaults proposed.)*
|
||||
- [x] existing session/auth tests updated for the new mechanism, not just a
|
||||
new number
|
||||
*(2026-10-16, T13.6: there were none. T13.1 shipped with no test of its own. `tests/session_idle_check.py` is that test (T13.6 added it).)*
|
||||
|
||||
---
|
||||
|
||||
**Found while writing that test (T13.6, 2026-10-16):**
|
||||
- Nothing under `/api/auth/` slides the session. `auth._EXEMPT_PREFIXES` covers
|
||||
that whole prefix, so `auth_gate` never refreshes there (including
|
||||
`/api/auth/me`). A page that only called `/api/auth/me` would not extend its
|
||||
session. The spec says "after confirming a request is authenticated", and these
|
||||
routes are not confirmed by the gate, so this is as written. Noted, not changed.
|
||||
- The wave's exit criteria below say `AUTH_SESSION_HOURS` defaults to **2**. T13.1
|
||||
(this section), `auth.py`, `.env.example` and `DEPLOYMENT.md` all say **8**. The
|
||||
code and the T13.1 text agree, so the exit line looks like a leftover from
|
||||
before the 2026-09-23 revision. Not changed here: it is a product number. Needs
|
||||
Matt to say which one he means.
|
||||
|
||||
### T13.2 — D18: Okta Management API client
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** nothing (independent of T13.1)
|
||||
- **Blocks:** T13.3
|
||||
- **Surface:** `server/`
|
||||
- **Files:** new `server/okta_sync.py` (or extend `server/okta_auth.py` —
|
||||
task's call), `server/.env.example`
|
||||
|
||||
**Do:** A small client for Okta's user-list endpoint, authenticated with a new,
|
||||
separate credential (e.g. `OKTA_API_TOKEN`) — not the OIDC client secret used
|
||||
for sign-in. Fetch the full user list (paginated per Okta's API) rather than
|
||||
one-by-one lookups per local user; this app has ~20-odd accounts today, and a
|
||||
list-and-diff is simpler and cheaper than N calls. Return each Okta user's
|
||||
identity-claim value (matching `OKTA_IDENTITY_CLAIM`, already confirmed live
|
||||
per wave 10) and status.
|
||||
|
||||
Follow the existing pattern for external credentials in this repo (`MICRON_DB_URL`,
|
||||
`SMTP_PASSWORD`): env-only, never logged, never returned to the browser in an
|
||||
error message.
|
||||
|
||||
**Do not:** reuse `OKTA_CLIENT_ID`/`OKTA_CLIENT_SECRET` for this. Sign-in and
|
||||
the management API are different trust boundaries with different scopes;
|
||||
conflating them means a compromise or rotation of one affects the other
|
||||
unnecessarily.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] the client authenticates with its own credential, distinct from the OIDC
|
||||
client
|
||||
*(2026-10-16: `OKTA_API_TOKEN`, sent as `Authorization: SSWS`. `tests/okta_sync_check.py` sets a different OIDC client secret and asserts it is never sent.)*
|
||||
- [x] it fetches the complete Okta user list, handling pagination
|
||||
*(2026-10-16: follows `Link: rel="next"`, 200 per page, 2-page case checked. A next link to another host is refused, a chain longer than 50 pages raises instead of cutting the list off, redirects are never followed.)*
|
||||
- [x] an auth failure or malformed response raises a clear, specific error
|
||||
rather than returning an empty list indistinguishable from "everyone was
|
||||
deprovisioned" — this distinction is what T13.3 depends on
|
||||
*(2026-10-16: `OktaSyncError` for HTTP 401/403/404/429/5xx, redirects, network errors, timeouts, non-JSON, non-list, a bad item (never skipped), and an empty list. 31 failure checks. Mutation test: making the client return `[]` on an error fails 10 of them.)*
|
||||
- [x] the credential is never logged or surfaced in any API response
|
||||
*(2026-10-16: error messages carry the class name and HTTP status only, never the URL or a header. `describe()` has no token. Nothing is wired to an API response yet; T13.4 wires the startup log.)*
|
||||
|
||||
---
|
||||
|
||||
### T13.3 — D18: the sync job
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** T13.2
|
||||
- **Blocks:** T13.4
|
||||
- **Surface:** `server/`
|
||||
- **Files:** `server/okta_sync.py`, `server/app.py` (or wherever `log_event`
|
||||
lives)
|
||||
|
||||
**Do:** Compare the Okta user list (T13.2) against local `users` rows. For any
|
||||
local `is_active=True` user whose Okta identity is missing from the list, or
|
||||
present with a non-active status, set `is_active=False` and write an
|
||||
`AuditLog` row (`actor="system:okta_sync"`, action e.g.
|
||||
`user_deprovisioned_by_sync`, detail naming the Okta status found). Never
|
||||
touch a user already `is_active=False`. Never re-enable anyone.
|
||||
|
||||
**The fail-closed rule, non-negotiable:** if T13.2's client raises an error of
|
||||
any kind (network, auth, malformed response, timeout), this task takes **no
|
||||
action on any account** for that run and logs the failure clearly (server log,
|
||||
at minimum). An error is never treated as "Okta returned zero active users."
|
||||
Write a test that asserts this directly: feed the sync a failing client and
|
||||
assert zero rows changed and zero `AuditLog` entries written.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] a user missing from Okta's list, or present but not active, is disabled
|
||||
with a correctly-detailed audit row
|
||||
*(2026-10-16: see "Decided 2026-10-16" below for which statuses count as "not active". Audit row: actor `system:okta_sync`, action `user_deprovisioned_by_sync`, `entity_type` user, `detail` `{okta_status, via}`; a missing user is recorded as `MISSING`.)*
|
||||
- [x] a user already disabled is left alone (no duplicate audit row each run)
|
||||
*(2026-10-16: a second run over the same data writes no new rows.)*
|
||||
- [x] an active Okta user already active locally produces no audit row (only
|
||||
changes are logged, not a clean bill of health every cycle)
|
||||
*(2026-10-16: verified in `tests/okta_sync_job_check.py`.)*
|
||||
- [x] **a simulated Okta API failure results in zero account changes and zero
|
||||
audit rows** — this is the one check that must never be skipped or
|
||||
weakened
|
||||
*(2026-10-16: 10 stand-in failures (OktaSyncError, RuntimeError, ConnectionError, TimeoutError, ValueError, KeyError, empty list, None, two kinds of garbage), 4 through the real client (HTTP 401/403/429/500), the default path with no token, a failed commit, and a wrong identity attribute. Each: `ok` is False, zero accounts changed, zero audit rows. Mutation test: removing the fail-closed branch and the zero-match guard fails 28 of these.)*
|
||||
- [x] the job never re-enables an account under any input
|
||||
*(2026-10-16: a disabled account that is ACTIVE in Okta stays disabled, and `okta_sync.py` contains no assignment of `is_active` to True.)*
|
||||
|
||||
---
|
||||
|
||||
**Decided 2026-10-16 (Matt, in chat):** which Okta statuses disable an account.
|
||||
The spec says "a non-active status". Read literally, that also catches
|
||||
`PASSWORD_EXPIRED`, `LOCKED_OUT` and `RECOVERY`, which people leave on their
|
||||
own, and this job never re-enables, so one AD password expiry would lock someone
|
||||
out of the app until an admin stepped in. Decided: an account disables when it is
|
||||
**missing** from Okta's list, or every Okta entry for it is **SUSPENDED** or
|
||||
**DEPROVISIONED** (`DISABLING_STATUSES` in `server/okta_sync.py`). Okta's default
|
||||
user list already leaves out DEPROVISIONED users, so those arrive as "missing".
|
||||
|
||||
**Added 2026-10-16, inside decision 4's "anything the sync can't confidently
|
||||
parse":** if there are active local accounts and **none** of them matches any
|
||||
Okta user, the run is refused and nothing changes. That pattern means the
|
||||
identity attribute is wrong (`OKTA_SYNC_PROFILE_ATTRIBUTE`), not that the whole
|
||||
company left. It is a guard, not a new feature. Raise it in the PR.
|
||||
|
||||
**Open, for the PR / Nick:** which Okta profile attribute equals the
|
||||
`preferred_username` claim in this tenant. The client assumes `profile.login`,
|
||||
Okta's default. Confirm before the first live run. A wrong value is caught by the
|
||||
zero-match guard above only when it matches nobody at all; a value that matches
|
||||
some accounts and not others would disable the rest. See `BL-069`.
|
||||
|
||||
### T13.4 — D18: run it on a schedule
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** T13.3
|
||||
- **Blocks:** T13.6
|
||||
- **Surface:** `server/`, `docker-compose.yml`
|
||||
- **Files:** `server/app.py` (startup hook) or a new sidecar per the `backup`
|
||||
container's pattern — task's call, per the decision record's noted
|
||||
alternative
|
||||
|
||||
**Do:** Wire T13.3 to run on an interval (proposed 15 minutes, env-overridable
|
||||
— e.g. `OKTA_SYNC_INTERVAL_SECONDS`, matching `BACKUP_INTERVAL_SECONDS`'s
|
||||
naming). Default choice is an in-process background task in the `api`
|
||||
container; if a separate container is chosen instead, follow the `backup`
|
||||
service's shape (own Dockerfile or script, `internal` network plus whatever
|
||||
egress reaching Okta requires — check whether `outbound` as currently defined
|
||||
is sufficient or Okta needs a distinct allowance).
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] the job runs automatically on the configured interval without manual
|
||||
invocation
|
||||
*(2026-10-16: a daemon timer thread starts from the app's lifespan hook (`server/app.py`), checks once a minute, and ran the sync on its own in `tests/okta_sync_schedule_check.py`. With `OKTA_API_TOKEN` unset it starts nothing and the startup log says `Okta sync DISABLED`. Booted the real app both ways; the thread starts and ends with the app.)*
|
||||
- [x] interval is env-configurable with a sane default
|
||||
*(2026-10-16: `OKTA_SYNC_INTERVAL_SECONDS`, default 900. A non-number uses the default. A value under 300 is raised to 300. Passed through `docker-compose.yml`.)*
|
||||
- [x] a container restart does not produce a duplicate/overlapping run, and a
|
||||
slow cycle does not stack with the next one
|
||||
*(2026-10-16: the api runs 2 gunicorn workers, so each starts its own timer. One row in `app_settings` (key `okta_sync`) is the gate: a worker claims a cycle with one atomic conditional UPDATE, so exactly one wins. Checked with 8 racing workers on a file database, 15 rounds plus the first-insert race: exactly 1 winner every time. The row outlives a restart, so a restart inside the interval starts nothing. A slow run cannot overlap the next: a fetch is capped at 120 s in total and the interval cannot go below 300 s. Inside one process the thread runs one cycle at a time. This needed no migration: `app_settings` already exists.)*
|
||||
|
||||
---
|
||||
|
||||
**Decided 2026-10-16 (task's call, per this task's "Do"):** in-process timer in the
|
||||
`api` container, as D18 proposed, not a sidecar. The `backup` pattern was the
|
||||
alternative; it would need its own image and would not remove the 2-worker
|
||||
duplicate question, because the sync reads the application database anyway.
|
||||
**Network check, as the task asked:** the `outbound` network the `api` service
|
||||
already joins is an ordinary bridge with a gateway, so Okta is reachable with no
|
||||
network change. The host firewall must allow https to the Okta org.
|
||||
**One change to T13.2's code:** `fetch_users()` now also stops after 120 seconds in
|
||||
total (`TOTAL_DEADLINE_SECONDS`), which is what lets the scheduler promise no
|
||||
overlap. Tested in `tests/okta_sync_check.py` section 8.
|
||||
|
||||
**T13.4 follow-up, 2026-10-16:** the sync's INFO log lines did not appear in
|
||||
`docker compose logs api`. Nothing in the app configures Python logging, and
|
||||
gunicorn configures only its own loggers, so INFO from any `wpsuite.*` logger is
|
||||
dropped. The T13.4 check booted the app in-process with logging switched on,
|
||||
which hid it. With the exact Dockerfile launch (2 gunicorn workers), a healthy
|
||||
start printed zero Okta lines. `okta_sync.py` now gives its own logger one stderr
|
||||
handler (only when the root logger has none). Re-run with the real gunicorn
|
||||
command: `configured` and `scheduled` (or `DISABLED`) now print once per worker.
|
||||
No other module's output changed. App-wide logging stays a separate question
|
||||
(BL-028 is the same family).
|
||||
|
||||
### T13.5 — D18: admin visibility
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** T13.3
|
||||
- **Blocks:** T13.6
|
||||
- **Surface:** `html/`
|
||||
- **Files:** `html/users.js` / `html/admin.js` (wherever audit history is
|
||||
already surfaced)
|
||||
|
||||
**Do:** Confirm an auto-disable reads clearly wherever admins already look at
|
||||
account history — the actor string (`system:okta_sync`) should be
|
||||
self-explanatory in context, not require reading server logs to understand.
|
||||
Do not build new UI beyond making sure the existing audit surface renders this
|
||||
actor sensibly. Email/notification to admins on auto-disable is a noted
|
||||
fast-follow (the decision record flags it as open, not required here) — do not
|
||||
build it in this task; log it instead if it's tempting to add.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [x] an auto-disabled account's audit entry is visible and legible in the
|
||||
existing admin UI without special-casing
|
||||
*(2026-10-16: run on the PC against the real page, 1440px and 390px, `tests/okta_sync_ui_check.py` 28/28. The admin Activity log card reads: Who `system:okta_sync`, Action `user deprovisioned by sync`, Type `user`, Item the username, Detail `okta_status: SUSPENDED, via: okta_sync` (a person missing from Okta reads `okta_status: MISSING`). Search for `okta_sync` and the User accounts filter both find the rows. No sideways scroll, no JavaScript errors. The disabled account is refused on its very next request; an account still active in Okta is not. A project super user does not see these rows: they belong to no project, so only admins do, the same as other user-management events. No code in `html/` changed.)*
|
||||
- [x] nothing here silently assumes `CR-019`'s activity view exists yet — this
|
||||
must work standalone
|
||||
*(2026-10-16: the check reads only the Activity log (AuditLog) card. The new Activity & usage card is on the page but is not read.)*
|
||||
|
||||
---
|
||||
|
||||
**No UI was built.** The existing card renders the new rows legibly as is. Two
|
||||
things you might expect and did not get, both on purpose: no admin email on an
|
||||
auto-disable (D18 calls it a fast-follow; not built, not logged as a new item
|
||||
because D18 already names it), and no per-user history view in the User
|
||||
Directory (it has none today, and the task says not to build one).
|
||||
|
||||
### T13.6 — D18: verification
|
||||
|
||||
- **Items:** `D18`
|
||||
- **Depends on:** T13.4, T13.5
|
||||
- **Blocks:** nothing
|
||||
- **Surface:** `server/`
|
||||
- **Files:** as touched above
|
||||
|
||||
**Do:** Full verification per `CLAUDE.md`. Beyond the usual run: specifically
|
||||
re-run T13.3's fail-closed test in isolation and confirm it still passes after
|
||||
T13.4's scheduling wrapper is in place — the scheduling layer must not
|
||||
introduce a path that swallows the client's error and proceeds anyway.
|
||||
|
||||
**Done when:**
|
||||
|
||||
- [ ] all `D18` acceptance criteria in `decisions-2026-09-17.md` are met or a
|
||||
failure is stated with a reason
|
||||
- [ ] the fail-closed behavior is verified end to end through the scheduled
|
||||
wrapper, not just the bare sync function
|
||||
- [ ] smoke test and `seed_demo.py` both still pass
|
||||
- [ ] full test suite passes
|
||||
|
||||
---
|
||||
|
||||
## Wave 13 exit criteria
|
||||
|
||||
- [ ] `AUTH_SESSION_HOURS` defaults to 2, documented
|
||||
- [ ] the sync job runs on a schedule and correctly disables accounts Okta no
|
||||
longer shows as active
|
||||
- [ ] every auto-disable is individually audited with a clearly non-human actor
|
||||
- [ ] an Okta API failure of any kind changes zero accounts — verified, not
|
||||
assumed
|
||||
- [ ] the sync never re-enables an account
|
||||
- [ ] `D18` fully accounted for, no open acceptance criteria
|
||||
|
||||
## Release note (2026-10-16)
|
||||
|
||||
T13.1 (commit `6cde6e3`) is held back from `main` until T14.1 lands, per the merge
|
||||
rule in `wave-12.md`. The release branch contains wave 11 and T13.2 to T13.5.
|
||||
`tests/session_idle_check.py` skips (exit 0) when `auth.maybe_refresh_token` is absent.
|
||||
Sessions keep the flat `AUTH_SESSION_HOURS` lifetime (default 12) until T13.1 lands.
|
||||
Reference in New Issue
Block a user