n.siegfried/Project-SDE-WP-Suite
3
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Productionize WP Suite: auth, security hardening, sync, dashboard, PWA, email

Browse Source 
Brings the Work Package Suite from a browser-local prototype to a
multi-tenant, SQL-backed deployment hardened for customer IP.

Auth & access control
- Local username/password login (bcrypt + JWT in an HttpOnly cookie),
  admin-managed users, per-project membership, and project-scoped API access.
- Admin console: change user roles, view the audit trail, manage settings.

Security hardening
- CSP / HSTS / X-Frame-Options / nosniff headers in nginx; Secure cookie via
  X-Forwarded-Proto; CSRF Origin check; attribute-safe output escaping.
- Login lockout, token_version session revocation, stronger password policy,
  fail-closed secret loading, encrypted (AES-256) database backups.

Persistence & schema
- SOPs and Work Packages are now DB-backed and shared across users, written
  through a durable client sync outbox that queues offline edits.
- Alembic migrations applied automatically on container start.

New capabilities
- Phase 2 dashboard (progress, gating, pagination, archive).
- Phase 3 PWA "Field View" with offline caching and auth fallback.
- WP owner assignment with OPTIONAL email notifications, OFF by default and
  toggled from the admin console. SMTP password is read only from the
  SMTP_PASSWORD env var (never stored); emails carry a WP number + deep link,
  never customer IP.

Also: IBM Carbon restyle, Help section, and DEPLOYMENT.md brought up to date.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Nick Siegfried
2026-07-15 17:51:15 -07:00
parent dd37f1f551
commit 39b48055ff
48 changed files with 2867 additions and 507 deletions
Download Patch File Download Diff File Expand all files Collapse all files

154
DEPLOYMENT.md
View File

@@ -51,9 +51,25 @@ Create a file named `.env` in the **project root** (same folder as
POSTGRES_DB=wpsuite
POSTGRES_USER=wpsuite
POSTGRES_PASSWORD=<strong-random-password>
# REQUIRED — signs login session cookies. If unset, `docker compose up` errors
# out and the API refuses to start. Generate once and keep it stable:
# openssl rand -base64 48
AUTH_SECRET_KEY=<strong-random-secret>
# Encrypts database backups at rest (AES-256). Set this BEFORE the DB holds
# customer IP. Keep the passphrase OFF this host — losing it makes dumps
# unrecoverable: openssl rand -base64 32
BACKUP_ENC_PASSPHRASE=<strong-random-passphrase>
# OPTIONAL — SMTP password for WP-assignment email notifications. Email is OFF
# by default and enabled from the Admin console; the host/port/from-address are
# configured there, but the password is only ever read from this variable (never
# stored in the DB or shown in the UI). Leave unset until you have SMTP details.
# SMTP_PASSWORD=<smtp-app-password>
```
That's it — the API now builds its own connection string from these three
The API builds its own DB connection string from the `POSTGRES_*`
values and **encodes the password automatically**, so a password with special
characters (`@ ! # : /` …) works without any manual escaping. `DATABASE_URL`
is **optional** and only needed if you want to point the API at some other
@@ -64,7 +80,8 @@ Generate a strong password with `openssl rand -base64 32`.
> **Portainer note:** for a Git-based stack these go in the stack's
> **Environment variables** section (Portainer doesn't read a local `.env`).
> Set `POSTGRES_DB` / `POSTGRES_USER` / `POSTGRES_PASSWORD` there.
> Set `POSTGRES_DB` / `POSTGRES_USER` / `POSTGRES_PASSWORD` / `AUTH_SECRET_KEY` /
> `BACKUP_ENC_PASSPHRASE` (and `SMTP_PASSWORD`, if you enable email) there.
These are the only credentials in the system, and they never appear in the
compose file or in git.
@@ -77,6 +94,14 @@ chosen hostname (e.g. `wp-suite.company.local`) to the `nginx_webserver`
container on that network. The container already proxies `/api/` to the `api`
service internally — no extra app config needed.
> **Serve it over HTTPS, and forward the scheme.** The bundled nginx sets the
> security response headers (CSP, HSTS, `X-Frame-Options`, `nosniff`) and passes
> `X-Forwarded-Proto: https` to the API, which is what makes the session cookie
> `Secure`. If you front the stack with your **own** proxy instead, make sure it
> terminates TLS and forwards `X-Forwarded-Proto: https` — otherwise the login
> cookie won't get the `Secure` flag. HSTS also assumes the site is only ever
> reached over HTTPS.
## 4. Bring it up
From the project root:
@@ -145,12 +170,10 @@ python3 server/seed_demo.py https://wp-suite.company.local --insecure
python3 server/seed_demo.py https://wp-suite.company.local --clean # remove it later
```
> **What shows where:** the DEMO **project** is API/SQL-backed, so it appears in
> the home-page project picker right away (this is the visible proof that the
> projects → SQL path works end-to-end). The DEMO **SOP and Work Packages** are
> written to SQL too, but the current front end still reads SOPs/WPs from the
> browser, so they won't render in the Creator/Dashboard until the Phase 2
> wiring. Inspect them at the SQL layer with `smoketest.py` or:
> **What shows where:** the DEMO **project**, its **SOP**, and its **Work
> Packages** are all API/SQL-backed, so they appear in the home-page project
> picker and render in the Creator/Dashboard as soon as any user opens the
> project. Inspect them at the SQL layer with `smoketest.py` or:
> ```bash
> docker compose exec db psql -U wpsuite -d wpsuite \
> -c "select number, subject, status from work_packages order by number;"
@@ -160,20 +183,21 @@ python3 server/seed_demo.py https://wp-suite.company.local --clean # remove it
## What is stored in SQL today
Be aware of the current persistence split — the API + Postgres are fully
deployed, and:
The API + Postgres are the system of record. Everything below is server-stored
and shared across every user who opens the project:
| Data | Stored in PostgreSQL today? |
|------|------------------------------|
| **Projects** | **Yes** — the front end is API-first (`/api/projects`), falling back to the browser only if the API is unreachable. |
| **Comments / feedback** | **Yes** — every feedback surface posts to `/api/feedback`. |
| **SOPs** | Endpoints exist (`/api/sops`); the front end still keeps the SOP in the browser (namespaced per project). Wiring it to the API is the remaining **Phase 2** step. |
| **Work Packages** | Same — `/api/wps` (+ issue/status/metrics) exist and are ready; the creator still saves to the browser per project. |
| **SOPs** | **Yes** — pulled from `/api/sops` on load and written through on every save. |
| **Work Packages** | **Yes** — same write-through to `/api/wps` (+ issue / status / archive / metrics), including the owner assignment (`assignee_id`). |
So a fresh deployment gives you **shared, server-stored projects and comments
immediately**. Moving SOPs and Work Packages off the browser and onto the API
(so they're shared across users too) is a front-end change only — the database
and endpoints are already in place.
Saves go through a **durable client-side sync outbox**: edits are written to the
API immediately, and if the device is offline they queue and retry when it
reconnects (4xx rejections are dropped rather than retried forever). The browser
cache is only an offline fallback that reconciles through that outbox — so two
users on the same project see the same server-stored SOP and Work Packages.
## Data model (PostgreSQL)
@@ -181,8 +205,13 @@ and endpoints are already in place.
|-------|-------|-------------|
| `projects` | top-level construction projects | `name`, `number`, `client`, `division`, `site`, `sample`, `data` |
| `sops` | project SOP baselines | `project_id` → projects, `name`, `number`, `complete`, `data` (full SOP JSON) |
| `work_packages` | individual IWPs | `project_id` → projects, `sop_id` → sops, `parent_id` (split instances), `number`, `subject`, `type`, `status`, `issued_at`, `data` (full WP JSON) |
| `work_packages` | individual IWPs | `project_id` → projects, `sop_id` → sops, `parent_id` (split instances), `number`, `subject`, `type`, `status`, `assignee_id` (owner), `issued_at`, `archived_at`, `data` (full WP JSON) |
| `comments` | feedback from any page | `source`, `sop_id`, `wp_id`, `step`, `author`, `text`, `extra` |
| `users` | login accounts | `username`, `password_hash` (bcrypt), `role`, `full_name`, `email`, `is_active`, login-lockout + `token_version` fields |
| `project_members` | per-project access control | `user_id` → users, `project_id` → projects |
| `audit_log` | append-only activity trail | `actor`, `action`, `entity_type`, `entity_id`, `project_id`, `summary`, `detail` |
| `notifications` | in-app record + email outbox | `user_id`, `kind`, `wp_id`, `subject`, `status` (pending / sent / failed / skipped) |
| `app_settings` | admin-configured settings (e.g. email) | `key`, `value` (JSON) |
The complete client document is stored verbatim in each row's `data` JSON
column; frequently-listed fields are promoted to real columns for filtering.
@@ -192,8 +221,13 @@ column; frequently-listed fields are promoted to real columns for filtering.
Projects `GET/POST /api/projects`, `GET/DELETE /api/projects/{id}` ·
SOPs `GET/POST /api/sops`, `GET /api/sops/latest`, `GET/DELETE /api/sops/{id}` ·
Work Packages `GET/POST /api/wps`, `GET/DELETE /api/wps/{id}`,
`POST /api/wps/{id}/issue`, `POST /api/wps/{id}/status`, `GET /api/wps/metrics` ·
Comments `POST /api/comments` (and `/api/feedback`), `GET /api/comments`.
`POST /api/wps/{id}/issue`, `POST /api/wps/{id}/status`, `POST /api/wps/{id}/archive`,
`GET /api/wps/metrics` ·
Comments `POST /api/comments` (and `/api/feedback`), `GET /api/comments` ·
Auth `POST /api/auth/login` / `logout`, `GET /api/auth/me`, admin user management
under `/api/auth/users` · Admin-only `GET/PUT /api/settings`,
`POST /api/settings/test-email`, `GET /api/notifications`,
`GET /api/projects/{id}/members`.
List/latest/metrics accept a `project_id` (and `sop_id`) filter. Full reference
and request shapes: `/api/docs` and [`server/README.md`](server/README.md).
@@ -209,27 +243,77 @@ docker compose up -d --build api # backend change (server/)
## Backups & retention
The whole dataset is in the `pgdata` volume — back it up on a schedule:
A **`backup` sidecar** (in `docker-compose.yml`) runs `pg_dump` on a schedule and
writes gzipped, timestamped dumps to `./backups/` on the host. It starts with the
stack — no cron to set up.
```bash
# Backup (run from project root)
docker compose exec -T db pg_dump -U wpsuite wpsuite > backup-$(date +%F).sql
- **Cadence / retention:** daily, keeping the newest 14 dumps. Override in `.env`
with `BACKUP_INTERVAL_SECONDS` (seconds between dumps) and `BACKUP_KEEP` (how many
to keep).
- **Encryption at rest:** set `BACKUP_ENC_PASSPHRASE` in `.env` and dumps are
written AES-256-encrypted as `*.sql.gz.enc`. **Do this before any customer IP
goes in** — without it the dumps (and every offsite copy) are plaintext. Store
the passphrase somewhere other than this host; if you lose it the backups can't
be restored.
- **Ad-hoc backup now:** `docker compose exec backup sh /scripts/db-backup.sh`
- **Restore (destructive — overwrites current data):**
`docker compose exec backup sh /scripts/db-restore.sh /backups/wpsuite-YYYYMMDD-HHMMSSZ.sql.gz.enc`
- **Offsite — do this:** the dumps live in `./backups/` on the host; if the host/volume
dies, so do they. Sync that folder offsite from the **host** (e.g. a cron running
`rclone`/`aws s3 sync`). The `db`/`backup` containers are on an egress-less
`internal` network on purpose, so offsite must be pushed from the host.
- **Test restores quarterly:** load the latest dump into a throwaway database and
confirm it applies. An untested backup is not a backup.
# Restore
docker compose exec -T db psql -U wpsuite -d wpsuite < backup-YYYY-MM-DD.sql
```
## Field devices & data at rest
## Schema migrations (important)
The field view (PWA) caches a project's Work Packages/SOP in the browser's
localStorage so it works offline — i.e. **customer IP sits on the device**.
localStorage is not encrypted and is not a security boundary. Signing out clears
the cached project data, but for any tablet/phone that opens customer-IP projects:
Tables are auto-created on API startup (`Base.metadata.create_all`). This
creates **missing tables**, but it does **not** alter existing ones. The
multi-project work added the `projects` table and new columns
(`sops.project_id`, `work_packages.project_id` / `parent_id` / `issued_at`):
- **Require full-disk encryption** (BitLocker / FileVault / Android FBE / iOS is
encrypted by default) and a device passcode.
- **Enrol field devices in MDM** so a lost device can be remotely wiped, and keep
the browser profile per-user on shared devices.
- Users should **sign out** when handing off a shared device (clears the cache).
- On a **fresh** database these appear automatically — nothing to do.
- On a database that **already has data** from an older schema, add the new
columns with a migration (introduce **Alembic**) or apply them manually with
`ALTER TABLE` before deploying — don't rely on `create_all` for column changes.
## Email notifications (optional)
Work-package **owner assignment** works out of the box (in-app only). Optional
**email** on assignment is **OFF by default** and is turned on from the **Admin
console → Notifications & email** card, where an admin sets the SMTP host / port /
TLS / From address and flips the master toggle.
- The **SMTP password is never stored in the database.** It is read only from the
`SMTP_PASSWORD` environment variable (see the `.env` block in step 2 and the
`api` service in `docker-compose.yml`). The UI shows only whether it is set.
- Email stays effectively off until **all** of: the toggle is on, SMTP host + From
are configured, and `SMTP_PASSWORD` is present. Until then, assignments are
still recorded in-app (status `skipped`); nothing is sent.
- Notification emails carry only a **WP number and a deep link** — never the work
package contents — so customer IP stays behind the login.
- Use the card's **Send test email** button to confirm SMTP before enabling.
## Schema migrations (Alembic)
Schema is managed by **Alembic** (`server/alembic/`). The API container runs
`alembic upgrade head` on startup (see the `Dockerfile` CMD), so **deploys apply
pending migrations automatically**.
- The **baseline** migration is idempotent: on a fresh database it creates every
table; on a database whose tables already exist (made by the old `create_all`)
it adopts the schema as-is — no manual `alembic stamp` needed.
- Local dev on SQLite still auto-creates tables for a zero-config run; Postgres is
migrations-only.
- **To change the schema:** edit `server/models.py`, then generate and review a
migration before committing:
```bash
# from the project root (against your dev SQLite or a staging DB)
python -m alembic -c server/alembic.ini revision --autogenerate -m "describe the change"
python -m alembic -c server/alembic.ini upgrade head # apply locally to test
```
The next `docker compose up -d --build api` applies it in production on startup.
## Local trial without Postgres