Project-SDE-WP-Suite/DEPLOY-runbook-2026-08-04.md

Deploy runbook — WP Suite

For: IT / whoever administers the Docker host and Portainer From: n.siegfried@prime-controls.com Revised: 2026-08-05 — this replaces the 2026-08-04 version. Same procedure, but the deploy now carries a second database migration and a new admin screen. If you already have the earlier copy, work from this one instead. Expected duration: 10–15 minutes, including the backup Expected downtime: under a minute, while containers are recreated

---

Fill these in before handing this over

Thing	Value
Docker host (SSH target)	________________
Stack name in Portainer	________________
Site URL	https://________________
Stack directory on the host (holds docker-compose.yml / backups/)	________________

Container names are fixed by the compose file and are the same on every host: nginx_webserver, wp_api, wp_db, wp_db_backup.

---

What this deploy changes

Front-end and nginx changes, a new admin screen, plus pending database migrations that run automatically. Three things make it more than a routine restart:

1. The nginx config and the entire html/ directory are baked into the container image at build time. A plain restart deploys nothing — the stack must be re-pulled and re-built.
2. A pending migration rewrites existing rows in the users.role column (b41c7ae90d52, values user → project_user). That is why step 1 is a backup and not optional. If a previous deploy already applied it, it will not run again — step 0 tells you which of these you are actually about to run.
3. A second migration adds new columns (a7c31f9e5b02: an archive timestamp on projects, and two default-membership fields on users). This one is additive and has database defaults for existing rows, so it does not rewrite anything.

For the people using the app, the visible changes are: projects can now be archived from the Admin Console (they disappear from the pickers and go read-only, and can be brought back), certain users can be set to join every new project automatically, and the Admin Console has been rebuilt so the user table fits on screen.

Migrations run themselves when the wp_api container starts. There is nothing to type and no new environment variables — do not change the stack's environment variables.

---

Step 0 — Record the current state (needed for rollback)

SSH to the Docker host and run:

docker exec wp_api alembic -c server/alembic.ini current
docker inspect nginx_webserver --format 'nginx image: {{.Image}}'
docker inspect wp_api          --format 'api image:   {{.Image}}'

Copy the output into your ticket. Also note the Git commit the Portainer stack is currently on (Portainer → the stack → the Git reference / last-updated commit). Without these, rollback is guesswork.

The first command prints the migration the database is currently on. Use it to see which migrations this deploy will actually run:

alembic current shows	What will run	What that means
c93f2b1d7e04 or earlier	both migrations	The users.role rewrite is included — the backup in step 1 matters most in this case.
d15b8c4ef207	only a7c31f9e5b02	The users.role rewrite already happened on an earlier deploy. This one is additive only.
a7c31f9e5b02	nothing	The database is already up to date; this is a code-only deploy.

Take the backup either way.

---

Step 1 — Back up the database

On the Docker host:

docker exec wp_db_backup /scripts/db-backup.sh

This triggers the stack's existing backup sidecar once, on demand. Expected output ends with a line like:

[db-backup] wrote 1.4M /backups/wpsuite-20260804-141233Z.sql.gz.enc

Confirm the file is on the host (substitute the stack directory):

ls -lt <stack-dir>/backups | head -3

Record that filename. Do not continue until you have seen the wrote … line and the file in that listing.

• A .sql.gz.enc extension means backups are encrypted — expected and correct.
• A .sql.gz extension plus a WARNING: BACKUP_ENC_PASSPHRASE not set line means backups are unencrypted. Not a blocker for this deploy; report it back.
• No SSH access? Portainer → Containers → wp_db_backup → Console → connect with /bin/sh, then run /scripts/db-backup.sh. Same result: the dump lands on the host, because /backups is a bind mount.

---

Step 2 — Redeploy the stack in Portainer

1. Portainer → Stacks → select the stack.
2. Pull and redeploy — with re-pull / re-build enabled.
3. Wait for it to report success.

A plain "restart" or "stop/start" will not deploy this change. See "What this deploy changes" above.

---

Step 3 — Confirm the containers came up

docker ps --filter name=nginx_webserver --filter name=wp_api --filter name=wp_db

All three must be Up, and wp_db should show (healthy). Then check the API applied its migrations cleanly:

docker logs wp_api --tail 40

You are looking for Alembic Running upgrade … lines followed by gunicorn starting up, and no traceback. The last one should end at a7c31f9e5b02. The API deliberately refuses to start if a migration fails, so a restarting wp_api container means the migration failed — go to Rollback.

Confirm the database landed on the new revision:

docker exec wp_api alembic -c server/alembic.ini current

Expected: a7c31f9e5b02 (head).

Then verify nginx's own view of its config:

docker exec nginx_webserver nginx -t

Expected: syntax is ok / test is successful.

---

Step 4 — Confirm the response headers

curl -sI https://<site-url>/work-package-suite.html | grep -Ei 'cache-control|content-security-policy'

Add -k if the site uses an internal or self-signed certificate.

Both lines must come back. Expected, approximately:

cache-control: no-cache, must-revalidate
content-security-policy: default-src 'self'; script-src 'self' 'unsafe-inline'; ...

If the content-security-policy line is missing while cache-control is present, the deploy is bad — go to Rollback and send me the nginx log. (This is the specific regression this deploy fixes; the two headers must coexist.)

Also confirm the API is reachable through the proxy:

curl -s https://<site-url>/api/health      # → {"ok": true}

---

Step 5 — Hard-reload once in a browser

Open the site and press Ctrl+Shift+R (Cmd+Shift+R on macOS) once. The app uses a service worker; a normal reload can serve the previous version and make a good deploy look broken.

Sanity checks — all four should take under a minute:

1. Log in. The home page offers to select or create a project.

2. Open User Directory (the Users link in the top-right menu, or the tile on the home page). The table should read as one line per user — if rows are three lines tall and the table spills outside its white card, you are still on the old cached files: hard-reload again.

   Changed since this runbook was written: user accounts moved out of the Admin Console into users.html when the Project Super User role was added, so that a project admin can create accounts on their own job. If you are deploying a build from before that change, read this step as "Admin Console → the user table".

3. Open Admin Console (admin account required). Two new cards are present and load: Projects, and Default members on new projects. Both should list rows, not an error.

4. In the Projects card, click Archive on a project you don't mind hiding (a DEMO- one if there is one), confirm the prompt, then tick Show archived — it should reappear marked archived. Click Unarchive to put it back. That round trip proves the new migration and the new endpoint are both live.

Deploy complete. Please report back: the step 0 output (including which migrations ran), the backup filename, and the two header lines from step 4.

---

Rollback

Pick the case that matches.

Case A — nginx won't start, or the CSP header is missing

The database is untouched by this, so this is a code-only rollback. In Portainer, redeploy the stack pinned to the previous Git commit recorded in step 0 (Portainer → the stack → change the Git reference to that commit → Pull and redeploy). Then re-run step 3 and step 4.

Before you do: grab the log, because it is what I need to fix this.

docker logs nginx_webserver --tail 100

Send me that output. If the container is in a restart loop the log still works.

Case B — wp_api is restarting / a migration failed

docker logs wp_api --tail 100

Send me that output. Do not restore the database and do not roll the API back without contacting me first. Which migration got as far as committing decides what is safe, and they are not the same:

• a7c31f9e5b02 (the new columns) is additive. If only this one ran, rolling the API back to the previous image is safe on its own — the old code simply ignores the extra columns. Nothing needs converting.
• b41c7ae90d52 (the users.role rewrite) is not. If that one committed, rolling the API back without converting those values back will break logins. That conversion is a one-line command, but it has to match what actually ran.

The alembic current output from step 0, plus the Running upgrade … lines in the log above, are exactly what tells us which case you are in — please include both.

Reach me at n.siegfried@prime-controls.com.

Case C — restoring the backup (only if I ask for it)

Destructive: this drops and recreates the current schema and data. For an encrypted dump, on the Docker host, in the backups directory:

export BACKUP_ENC_PASSPHRASE='<the passphrase — from the stack env vars>'
openssl enc -d -aes-256-cbc -pbkdf2 -pass env:BACKUP_ENC_PASSPHRASE \
  -in wpsuite-<timestamp>.sql.gz.enc \
  | gunzip \
  | docker exec -i wp_db psql -U wpsuite -d wpsuite
unset BACKUP_ENC_PASSPHRASE

For an unencrypted dump, drop the openssl stage and pipe gunzip straight into psql. Substitute the real values if POSTGRES_USER / POSTGRES_DB are not wpsuite.

---

Notes

• Do not add or change environment variables for this deploy.
• Do not run docker compose down -v — the -v flag deletes the pgdata volume and with it the entire database.
• docker compose … commands are avoided throughout this runbook on purpose: for a Portainer-managed Git stack the compose project lives under Portainer's own data directory, so docker compose from an SSH session usually can't find it. The docker exec <container-name> form used here works from any directory.
• Full background documentation: DEPLOYMENT.md in the repository.