Project-SDE-WP-Suite/DEPLOYMENT.md

# Deployment

Audience: the IT admin standing this up inside the firewall. This covers the
**SQL-backed deployment** — NGINX serving the static front end and a Python API
backed by **PostgreSQL**.

The repo already contains everything needed to run it as a Docker stack:
`Dockerfile`, `docker-compose.yml`, the `nginx/` config, the front end in
`html/`, and the API in `server/`. The detailed container reference (endpoints,
password rotation, day-to-day commands) lives in
[`server/README.md`](server/README.md) — this doc is the start-to-finish guide.

```
                         [ your TLS reverse proxy / traefik ]   ← HTTPS terminates here
                                        │  (external "proxy" network)
                                   ┌────▼────┐   internal network   ┌──────────┐   ┌────────────┐
   browser ───────────────────────│  nginx  │ ───── /api/ ───────> │   api    │ → │  postgres  │
                                   │ (html/) │                      │ FastAPI  │   │   (db)     │
                                   └─────────┘                      └──────────┘   └────────────┘
```

Everything runs inside your firewall; the app makes **no outbound internet
calls** (logo and scripts are local).

> **Architecture note:** all static files live under **`html/`** and are *baked
> into the nginx image* at build time (not bind-mounted). So after any front-end
> change you rebuild the `webserver` image (see *Updating* below). The API image
> is built from the root `Dockerfile`.

---

## 1. Prerequisites

- A Linux host with **Docker** and **Docker Compose v2** (`docker compose …`).
- An external Docker network named `proxy` that your TLS-terminating reverse
  proxy also sits on (the compose file marks it `external: true`):
  ```bash
  docker network create proxy
  ```
  If you don't run a separate reverse proxy, you can instead publish the nginx
  container's port 80 directly (see the note in step 4) and terminate TLS there.
- The repository checked out on the host.

## 2. Create the database credentials (`.env`)

Create a file named `.env` in the **project root** (same folder as
`docker-compose.yml`). It is git-ignored and must never be committed.

```bash
# .env  — project root
POSTGRES_DB=wpsuite
POSTGRES_USER=wpsuite
POSTGRES_PASSWORD=<strong-random-password>
```

That's it — the API now builds its own connection string from these three
values and **encodes the password automatically**, so a password with special
characters (`@ ! # : /` …) works without any manual escaping. `DATABASE_URL`
is **optional** and only needed if you want to point the API at some other
database; if you do set it, you must URL-encode the password yourself, and it's
ignored whenever the three `POSTGRES_*` values are present.

Generate a strong password with `openssl rand -base64 32`.

> **Portainer note:** for a Git-based stack these go in the stack's
> **Environment variables** section (Portainer doesn't read a local `.env`).
> Set `POSTGRES_DB` / `POSTGRES_USER` / `POSTGRES_PASSWORD` there.

These are the only credentials in the system, and they never appear in the
compose file or in git.

## 3. Point your reverse proxy at the nginx container

The nginx container listens on port **80** on the `proxy` network and expects
TLS to be terminated upstream (by your reverse proxy / traefik). Route your
chosen hostname (e.g. `wp-suite.company.local`) to the `nginx_webserver`
container on that network. The container already proxies `/api/` to the `api`
service internally — no extra app config needed.

## 4. Bring it up

From the project root:

```bash
docker compose up -d --build      # builds the api + nginx images, starts all three containers
docker compose ps                 # confirm nginx_webserver, wp_api, wp_db are running/healthy
docker compose logs -f api        # watch the API start (Ctrl-C to stop following)
```

The database schema is **created automatically** on first API start — no manual
`CREATE TABLE`. The Postgres data lives in the named volume `pgdata` and
survives `docker compose down` (only `down -v` deletes it).

> No separate reverse proxy? Publish nginx directly by adding a `ports:` mapping
> to the `webserver` service (e.g. `"8080:80"`) and terminate TLS at whatever
> sits in front of it. The internal `api`/`db` containers should **never** be
> published.

## 5. Verify

```bash
# API liveness (from the host, through the proxy hostname)
curl https://wp-suite.company.local/api/health        # → {"ok": true}

# Interactive API docs
#   https://wp-suite.company.local/api/docs
```

Then load the site in a browser: the home page should prompt to **select or
create a project**. Create one, complete an SOP, and confirm a row appears:

```bash
docker compose exec db psql -U wpsuite -d wpsuite -c "select id, name from projects;"
```

### Automated smoke test

`server/smoketest.py` exercises the whole stack end-to-end (health → project →
SOP → Work Package → the AWP issue gate → status → metrics → comments → cascade
cleanup). Stdlib only — no pip/jq.

```bash
# Through the proxy (use --insecure for a self-signed internal cert):
python3 server/smoketest.py https://wp-suite.company.local --insecure

# Or from inside the api container (hits FastAPI directly):
docker compose exec api python /app/server/smoketest.py http://localhost:8000

# Add --keep to leave a demo project in the DB so you can open it in the UI.
```

Exit code 0 and "ALL PASS" means the API, the Python logic, and SQL are all
working. It cleans up after itself (the test project and its SOP/WPs are
deleted via cascade); a single tagged test comment remains (there's no comment
delete endpoint).

### Loadable demo project

`server/seed_demo.py` populates a realistic **DEMO** project (a complete SOP plus
a spread of Work Packages: issued, gated, a multi-discipline master with split
instances, an overdue one, an over-threshold draft) so there's data to look at.

```bash
python3 server/seed_demo.py https://wp-suite.company.local --insecure
python3 server/seed_demo.py https://wp-suite.company.local --clean   # remove it later
```

> **What shows where:** the DEMO **project** is API/SQL-backed, so it appears in
> the home-page project picker right away (this is the visible proof that the
> projects → SQL path works end-to-end). The DEMO **SOP and Work Packages** are
> written to SQL too, but the current front end still reads SOPs/WPs from the
> browser, so they won't render in the Creator/Dashboard until the Phase 2
> wiring. Inspect them at the SQL layer with `smoketest.py` or:
> ```bash
> docker compose exec db psql -U wpsuite -d wpsuite \
>   -c "select number, subject, status from work_packages order by number;"
> ```

---

## What is stored in SQL today

Be aware of the current persistence split — the API + Postgres are fully
deployed, and:

| Data | Stored in PostgreSQL today? |
|------|------------------------------|
| **Projects** | **Yes** — the front end is API-first (`/api/projects`), falling back to the browser only if the API is unreachable. |
| **Comments / feedback** | **Yes** — every feedback surface posts to `/api/feedback`. |
| **SOPs** | Endpoints exist (`/api/sops`); the front end still keeps the SOP in the browser (namespaced per project). Wiring it to the API is the remaining **Phase 2** step. |
| **Work Packages** | Same — `/api/wps` (+ issue/status/metrics) exist and are ready; the creator still saves to the browser per project. |

So a fresh deployment gives you **shared, server-stored projects and comments
immediately**. Moving SOPs and Work Packages off the browser and onto the API
(so they're shared across users too) is a front-end change only — the database
and endpoints are already in place.

## Data model (PostgreSQL)

| Table | Holds | Key columns |
|-------|-------|-------------|
| `projects` | top-level construction projects | `name`, `number`, `client`, `division`, `site`, `sample`, `data` |
| `sops` | project SOP baselines | `project_id` → projects, `name`, `number`, `complete`, `data` (full SOP JSON) |
| `work_packages` | individual IWPs | `project_id` → projects, `sop_id` → sops, `parent_id` (split instances), `number`, `subject`, `type`, `status`, `issued_at`, `data` (full WP JSON) |
| `comments` | feedback from any page | `source`, `sop_id`, `wp_id`, `step`, `author`, `text`, `extra` |

The complete client document is stored verbatim in each row's `data` JSON
column; frequently-listed fields are promoted to real columns for filtering.

### Endpoints (summary)

Projects `GET/POST /api/projects`, `GET/DELETE /api/projects/{id}` ·
SOPs `GET/POST /api/sops`, `GET /api/sops/latest`, `GET/DELETE /api/sops/{id}` ·
Work Packages `GET/POST /api/wps`, `GET/DELETE /api/wps/{id}`,
`POST /api/wps/{id}/issue`, `POST /api/wps/{id}/status`, `GET /api/wps/metrics` ·
Comments `POST /api/comments` (and `/api/feedback`), `GET /api/comments`.
List/latest/metrics accept a `project_id` (and `sop_id`) filter. Full reference
and request shapes: `/api/docs` and [`server/README.md`](server/README.md).

---

## Updating after a change

```bash
git pull
docker compose up -d --build webserver   # front-end change (html/) — rebuild the baked image
docker compose up -d --build api         # backend change (server/)
```

## Backups & retention

The whole dataset is in the `pgdata` volume — back it up on a schedule:

```bash
# Backup (run from project root)
docker compose exec -T db pg_dump -U wpsuite wpsuite > backup-$(date +%F).sql

# Restore
docker compose exec -T db psql -U wpsuite -d wpsuite < backup-YYYY-MM-DD.sql
```

## Schema migrations (important)

Tables are auto-created on API startup (`Base.metadata.create_all`). This
creates **missing tables**, but it does **not** alter existing ones. The
multi-project work added the `projects` table and new columns
(`sops.project_id`, `work_packages.project_id` / `parent_id` / `issued_at`):

- On a **fresh** database these appear automatically — nothing to do.
- On a database that **already has data** from an older schema, add the new
  columns with a migration (introduce **Alembic**) or apply them manually with
  `ALTER TABLE` before deploying — don't rely on `create_all` for column changes.

## Local trial without Postgres

For a quick local look, the API falls back to a SQLite file when `DATABASE_URL`
is unset (`sqlite:///./wpsuite.db`) — see [`server/README.md`](server/README.md)
§ *Local dev*. The front end alone can also be served statically from `html/`
(it falls back to browser storage when the API isn't reachable).