Files
Project-SDE-WP-Suite/DEPLOY-runbook-2026-08-04.md
n.siegfried 928ab8c900 Archive projects, auto-add default members, rebuild the admin console
Three things asked for together, plus the migration they share (a7c31f9e5b02 —
additive, with database defaults for existing rows, so unlike the users.role
rewrite it is safe under a code-only rollback).

ARCHIVE A PROJECT. A finished job leaves every picker, switcher and search, and
freezes read-only, without losing anything. Hiding is free: GET /api/projects
defaults to archived=exclude, so the home picker and the app-bar switcher drop it
without either of them changing. Freezing is require_project_writable(), which
every write that lands on a project now goes through — SOP and WP upserts (both
ends, so a package can be moved neither into nor out of an archived job), deletes,
issue, status, WP archive, and comments on its WPs/SOPs. It answers 409, not 403:
nobody lacks a permission, the project's state is the objection, and the browser
outbox in project-data.js retires 4xx ops instead of retrying them against a job
that will never accept them. Unarchive and delete stay allowed on purpose —
unarchive is the one write an archived project must take, and archive-then-delete
is a normal sequence.

DEFAULT MEMBERS ON NEW PROJECTS. users.auto_add_projects / auto_add_role flag the
people who belong on every job, so an admin says it once instead of remembering it
at each project creation. It runs on the is_new branch of upsert_project, which is
the single road into project creation, so the home page, the sample project and the
demo seeder are all covered and an update never re-runs it. Note the interaction
with the existing creator-grant: that row commits first and add_default_members
never overwrites an existing membership, so the creator grant now carries the
creator's own auto_add_role — otherwise someone flagged "Project Admin on every
job" would land as a plain member on the one job they started themselves.

ADMIN CONSOLE. The user table had outgrown .wrap{max-width:860px}: nine columns in
an 860px card meant every cell wrapped, so one user occupied a ~100px band, the
action buttons stacked, and the table spilled outside its own white card. Now
1240px, with wide tables scrolling inside .tscroll so the page itself never scrolls
sideways, and one spacing/control scale across all twelve cards. Truncation hangs
off a span inside the cell rather than max-width on the td, which table-layout:auto
treats as advisory — the usual reason cell ellipsis works in the stylesheet and not
on the page.

Found in review and fixed here rather than later:

- Stored XSS in the new Projects card, reachable by any signed-in user, landing in
  an admin's session. The uesc(v).replace(/'/g,"\'") idiom this file already used
  in eight places escapes in the wrong order — uesc leaves backslashes alone, so a
  stored name containing \' closes the JS string literal and the rest executes.
  jsq() does backslash, then quote, then HTML, and all thirteen handler bindings go
  through it. The same bug, unescaped entirely, was in the SOP builder's custom
  constraint names (escHandlerArg there). Three of seven test payloads escaped the
  literal under the old idiom — one of them a plain name ending in a backslash, so
  it was breaking buttons for innocent input too.
- _save_comment resolved wp_id and sop_id with if/elif but stored both, so a
  payload naming a WP you may touch and a SOP you may not was authorised on the WP
  alone and still wrote into the other project's thread. Both are checked now.
- Promoting an account to admin left its default-member flag set but invisible,
  ready to take effect again on demotion — cleared, as set_user_auto_add already
  does for the role.

smoketest.py and the console's own smoke test both assert the archive round trip:
out of the default list, present with archived=all, writes refused with 409, and
all of it undone by unarchiving.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 13:48:43 -07:00

10 KiB
Raw Blame History

Deploy runbook — WP Suite

For: IT / whoever administers the Docker host and Portainer From: n.siegfried@prime-controls.com Revised: 2026-08-05 — this replaces the 2026-08-04 version. Same procedure, but the deploy now carries a second database migration and a new admin screen. If you already have the earlier copy, work from this one instead. Expected duration: 1015 minutes, including the backup Expected downtime: under a minute, while containers are recreated


Fill these in before handing this over

Thing Value
Docker host (SSH target) ________________
Stack name in Portainer ________________
Site URL https://________________
Stack directory on the host (holds docker-compose.yml / backups/) ________________

Container names are fixed by the compose file and are the same on every host: nginx_webserver, wp_api, wp_db, wp_db_backup.


What this deploy changes

Front-end and nginx changes, a new admin screen, plus pending database migrations that run automatically. Three things make it more than a routine restart:

  1. The nginx config and the entire html/ directory are baked into the container image at build time. A plain restart deploys nothing — the stack must be re-pulled and re-built.
  2. A pending migration rewrites existing rows in the users.role column (b41c7ae90d52, values userproject_user). That is why step 1 is a backup and not optional. If a previous deploy already applied it, it will not run again — step 0 tells you which of these you are actually about to run.
  3. A second migration adds new columns (a7c31f9e5b02: an archive timestamp on projects, and two default-membership fields on users). This one is additive and has database defaults for existing rows, so it does not rewrite anything.

For the people using the app, the visible changes are: projects can now be archived from the Admin Console (they disappear from the pickers and go read-only, and can be brought back), certain users can be set to join every new project automatically, and the Admin Console has been rebuilt so the user table fits on screen.

Migrations run themselves when the wp_api container starts. There is nothing to type and no new environment variables — do not change the stack's environment variables.


Step 0 — Record the current state (needed for rollback)

SSH to the Docker host and run:

docker exec wp_api alembic -c server/alembic.ini current
docker inspect nginx_webserver --format 'nginx image: {{.Image}}'
docker inspect wp_api          --format 'api image:   {{.Image}}'

Copy the output into your ticket. Also note the Git commit the Portainer stack is currently on (Portainer → the stack → the Git reference / last-updated commit). Without these, rollback is guesswork.

The first command prints the migration the database is currently on. Use it to see which migrations this deploy will actually run:

alembic current shows What will run What that means
c93f2b1d7e04 or earlier both migrations The users.role rewrite is included — the backup in step 1 matters most in this case.
d15b8c4ef207 only a7c31f9e5b02 The users.role rewrite already happened on an earlier deploy. This one is additive only.
a7c31f9e5b02 nothing The database is already up to date; this is a code-only deploy.

Take the backup either way.


Step 1 — Back up the database

On the Docker host:

docker exec wp_db_backup /scripts/db-backup.sh

This triggers the stack's existing backup sidecar once, on demand. Expected output ends with a line like:

[db-backup] wrote 1.4M /backups/wpsuite-20260804-141233Z.sql.gz.enc

Confirm the file is on the host (substitute the stack directory):

ls -lt <stack-dir>/backups | head -3

Record that filename. Do not continue until you have seen the wrote … line and the file in that listing.

  • A .sql.gz.enc extension means backups are encrypted — expected and correct.
  • A .sql.gz extension plus a WARNING: BACKUP_ENC_PASSPHRASE not set line means backups are unencrypted. Not a blocker for this deploy; report it back.
  • No SSH access? Portainer → Containerswp_db_backupConsole → connect with /bin/sh, then run /scripts/db-backup.sh. Same result: the dump lands on the host, because /backups is a bind mount.

Step 2 — Redeploy the stack in Portainer

  1. Portainer → Stacks → select the stack.
  2. Pull and redeploy — with re-pull / re-build enabled.
  3. Wait for it to report success.

A plain "restart" or "stop/start" will not deploy this change. See "What this deploy changes" above.


Step 3 — Confirm the containers came up

docker ps --filter name=nginx_webserver --filter name=wp_api --filter name=wp_db

All three must be Up, and wp_db should show (healthy). Then check the API applied its migrations cleanly:

docker logs wp_api --tail 40

You are looking for Alembic Running upgrade … lines followed by gunicorn starting up, and no traceback. The last one should end at a7c31f9e5b02. The API deliberately refuses to start if a migration fails, so a restarting wp_api container means the migration failed — go to Rollback.

Confirm the database landed on the new revision:

docker exec wp_api alembic -c server/alembic.ini current

Expected: a7c31f9e5b02 (head).

Then verify nginx's own view of its config:

docker exec nginx_webserver nginx -t

Expected: syntax is ok / test is successful.


Step 4 — Confirm the response headers

curl -sI https://<site-url>/work-package-suite.html | grep -Ei 'cache-control|content-security-policy'

Add -k if the site uses an internal or self-signed certificate.

Both lines must come back. Expected, approximately:

cache-control: no-cache, must-revalidate
content-security-policy: default-src 'self'; script-src 'self' 'unsafe-inline'; ...

If the content-security-policy line is missing while cache-control is present, the deploy is bad — go to Rollback and send me the nginx log. (This is the specific regression this deploy fixes; the two headers must coexist.)

Also confirm the API is reachable through the proxy:

curl -s https://<site-url>/api/health      # → {"ok": true}

Step 5 — Hard-reload once in a browser

Open the site and press Ctrl+Shift+R (Cmd+Shift+R on macOS) once. The app uses a service worker; a normal reload can serve the previous version and make a good deploy look broken.

Sanity checks — all four should take under a minute:

  1. Log in. The home page offers to select or create a project.
  2. Open Admin Console (the link is on the home page; you need an admin account). The user table should read as one line per user — if rows are three lines tall and the table spills outside its white card, you are still on the old cached files: hard-reload again.
  3. Two new cards are present and load: Projects, and Default members on new projects. Both should list rows, not an error.
  4. In the Projects card, click Archive on a project you don't mind hiding (a DEMO- one if there is one), confirm the prompt, then tick Show archived — it should reappear marked archived. Click Unarchive to put it back. That round trip proves the new migration and the new endpoint are both live.

Deploy complete. Please report back: the step 0 output (including which migrations ran), the backup filename, and the two header lines from step 4.


Rollback

Pick the case that matches.

Case A — nginx won't start, or the CSP header is missing

The database is untouched by this, so this is a code-only rollback. In Portainer, redeploy the stack pinned to the previous Git commit recorded in step 0 (Portainer → the stack → change the Git reference to that commit → Pull and redeploy). Then re-run step 3 and step 4.

Before you do: grab the log, because it is what I need to fix this.

docker logs nginx_webserver --tail 100

Send me that output. If the container is in a restart loop the log still works.

Case B — wp_api is restarting / a migration failed

docker logs wp_api --tail 100

Send me that output. Do not restore the database and do not roll the API back without contacting me first. Which migration got as far as committing decides what is safe, and they are not the same:

  • a7c31f9e5b02 (the new columns) is additive. If only this one ran, rolling the API back to the previous image is safe on its own — the old code simply ignores the extra columns. Nothing needs converting.
  • b41c7ae90d52 (the users.role rewrite) is not. If that one committed, rolling the API back without converting those values back will break logins. That conversion is a one-line command, but it has to match what actually ran.

The alembic current output from step 0, plus the Running upgrade … lines in the log above, are exactly what tells us which case you are in — please include both.

Reach me at n.siegfried@prime-controls.com.

Case C — restoring the backup (only if I ask for it)

Destructive: this drops and recreates the current schema and data. For an encrypted dump, on the Docker host, in the backups directory:

export BACKUP_ENC_PASSPHRASE='<the passphrase — from the stack env vars>'
openssl enc -d -aes-256-cbc -pbkdf2 -pass env:BACKUP_ENC_PASSPHRASE \
  -in wpsuite-<timestamp>.sql.gz.enc \
  | gunzip \
  | docker exec -i wp_db psql -U wpsuite -d wpsuite
unset BACKUP_ENC_PASSPHRASE

For an unencrypted dump, drop the openssl stage and pipe gunzip straight into psql. Substitute the real values if POSTGRES_USER / POSTGRES_DB are not wpsuite.


Notes

  • Do not add or change environment variables for this deploy.
  • Do not run docker compose down -v — the -v flag deletes the pgdata volume and with it the entire database.
  • docker compose … commands are avoided throughout this runbook on purpose: for a Portainer-managed Git stack the compose project lives under Portainer's own data directory, so docker compose from an SSH session usually can't find it. The docker exec <container-name> form used here works from any directory.
  • Full background documentation: DEPLOYMENT.md in the repository.