Project-SDE-WP-Suite/DEPLOY-runbook-2026-08-04.md

Deploy runbook — WP Suite, 2026-08-04

For: IT / whoever administers the Docker host and Portainer From: n.siegfried@prime-controls.com Expected duration: 10–15 minutes, including the backup Expected downtime: under a minute, while containers are recreated

---

Fill these in before handing this over

Thing	Value
Docker host (SSH target)	________________
Stack name in Portainer	________________
Site URL	https://________________
Stack directory on the host (holds docker-compose.yml / backups/)	________________

Container names are fixed by the compose file and are the same on every host: nginx_webserver, wp_api, wp_db, wp_db_backup.

---

What this deploy changes

Front-end and nginx changes, plus pending database migrations that run automatically. Two things make it more than a routine restart:

1. The nginx config and the entire html/ directory are baked into the container image at build time. A plain restart deploys nothing — the stack must be re-pulled and re-built.
2. A pending migration rewrites existing rows in the users.role column (values user → project_user). That is why step 1 is a backup and not optional.

Migrations run themselves when the wp_api container starts. There is nothing to type and no new environment variables — do not change the stack's environment variables.

---

Step 0 — Record the current state (needed for rollback)

SSH to the Docker host and run:

docker exec wp_api alembic -c server/alembic.ini current
docker inspect nginx_webserver --format 'nginx image: {{.Image}}'
docker inspect wp_api          --format 'api image:   {{.Image}}'

Copy the output into your ticket. Also note the Git commit the Portainer stack is currently on (Portainer → the stack → the Git reference / last-updated commit). Without these, rollback is guesswork.

---

Step 1 — Back up the database

On the Docker host:

docker exec wp_db_backup /scripts/db-backup.sh

This triggers the stack's existing backup sidecar once, on demand. Expected output ends with a line like:

[db-backup] wrote 1.4M /backups/wpsuite-20260804-141233Z.sql.gz.enc

Confirm the file is on the host (substitute the stack directory):

ls -lt <stack-dir>/backups | head -3

Record that filename. Do not continue until you have seen the wrote … line and the file in that listing.

• A .sql.gz.enc extension means backups are encrypted — expected and correct.
• A .sql.gz extension plus a WARNING: BACKUP_ENC_PASSPHRASE not set line means backups are unencrypted. Not a blocker for this deploy; report it back.
• No SSH access? Portainer → Containers → wp_db_backup → Console → connect with /bin/sh, then run /scripts/db-backup.sh. Same result: the dump lands on the host, because /backups is a bind mount.

---

Step 2 — Redeploy the stack in Portainer

1. Portainer → Stacks → select the stack.
2. Pull and redeploy — with re-pull / re-build enabled.
3. Wait for it to report success.

A plain "restart" or "stop/start" will not deploy this change. See "What this deploy changes" above.

---

Step 3 — Confirm the containers came up

docker ps --filter name=nginx_webserver --filter name=wp_api --filter name=wp_db

All three must be Up, and wp_db should show (healthy). Then check the API applied its migrations cleanly:

docker logs wp_api --tail 40

You are looking for Alembic Running upgrade … lines followed by gunicorn starting up, and no traceback. The API deliberately refuses to start if a migration fails, so a restarting wp_api container means the migration failed — go to Rollback.

Then verify nginx's own view of its config:

docker exec nginx_webserver nginx -t

Expected: syntax is ok / test is successful.

---

Step 4 — Confirm the response headers

curl -sI https://<site-url>/work-package-suite.html | grep -Ei 'cache-control|content-security-policy'

Add -k if the site uses an internal or self-signed certificate.

Both lines must come back. Expected, approximately:

cache-control: no-cache, must-revalidate
content-security-policy: default-src 'self'; script-src 'self' 'unsafe-inline'; ...

If the content-security-policy line is missing while cache-control is present, the deploy is bad — go to Rollback and send me the nginx log. (This is the specific regression this deploy fixes; the two headers must coexist.)

Also confirm the API is reachable through the proxy:

curl -s https://<site-url>/api/health      # → {"ok": true}

---

Step 5 — Hard-reload once in a browser

Open the site and press Ctrl+Shift+R (Cmd+Shift+R on macOS) once. The app uses a service worker; a normal reload can serve the previous version and make a good deploy look broken.

Sanity check: log in, and confirm the home page offers to select or create a project.

Deploy complete. Please report back: the step 0 output, the backup filename, and the two header lines from step 4.

---

Rollback

Pick the case that matches.

Case A — nginx won't start, or the CSP header is missing

The database is untouched by this, so this is a code-only rollback. In Portainer, redeploy the stack pinned to the previous Git commit recorded in step 0 (Portainer → the stack → change the Git reference to that commit → Pull and redeploy). Then re-run step 3 and step 4.

Before you do: grab the log, because it is what I need to fix this.

docker logs nginx_webserver --tail 100

Send me that output. If the container is in a restart loop the log still works.

Case B — wp_api is restarting / a migration failed

docker logs wp_api --tail 100

Send me that output. Do not restore the database and do not roll the API back without contacting me first — if the users.role migration already committed, rolling the API back to the previous image without converting those values back will break logins. That conversion is a one-line command, but it has to match what actually ran.

Reach me at n.siegfried@prime-controls.com.

Case C — restoring the backup (only if I ask for it)

Destructive: this drops and recreates the current schema and data. For an encrypted dump, on the Docker host, in the backups directory:

export BACKUP_ENC_PASSPHRASE='<the passphrase — from the stack env vars>'
openssl enc -d -aes-256-cbc -pbkdf2 -pass env:BACKUP_ENC_PASSPHRASE \
  -in wpsuite-<timestamp>.sql.gz.enc \
  | gunzip \
  | docker exec -i wp_db psql -U wpsuite -d wpsuite
unset BACKUP_ENC_PASSPHRASE

For an unencrypted dump, drop the openssl stage and pipe gunzip straight into psql. Substitute the real values if POSTGRES_USER / POSTGRES_DB are not wpsuite.

---

Notes

• Do not add or change environment variables for this deploy.
• Do not run docker compose down -v — the -v flag deletes the pgdata volume and with it the entire database.
• docker compose … commands are avoided throughout this runbook on purpose: for a Portainer-managed Git stack the compose project lives under Portainer's own data directory, so docker compose from an SSH session usually can't find it. The docker exec <container-name> form used here works from any directory.
• Full background documentation: DEPLOYMENT.md in the repository.