adding gitea actions
This commit is contained in:
@@ -181,7 +181,8 @@ Two settings live outside `src/env.ts`:
|
||||
### Deployment
|
||||
|
||||
- `Dockerfile`: a multi-stage build on `node:24-bookworm-slim`. The runtime stage holds `build/`, production `node_modules` (`better-sqlite3`, `zod`; everything else is bundled) and `drizzle/`, runs as `node`, and keeps the database at `/data/sync.db`.
|
||||
- `docker-compose.yml` is the Portainer stack: one service, the `sync-data` volume, required settings enforced with `${VAR:?}`, and `stop_grace_period: 2m` so a run in progress can finish.
|
||||
- `.gitea/workflows/ci.yml`: every pull request and push to main runs lint, type check, tests and build. On main, a second job that needs the first builds the image and pushes it to the Gitea registry, tagged `latest` and with the short commit hash, then calls the Portainer webhook if one is set. Since CI builds the image, `ORIGIN` is a Gitea Actions variable.
|
||||
- `docker-compose.yml` is the Portainer stack: one service using the CI image with `pull_policy: always` (a redeploy never reuses a stale image, and `IMAGE` pins a commit tag for rollback), the `sync-data` volume, required settings enforced with `${VAR:?}`, and `stop_grace_period: 2m` so a run in progress can finish. Its `build:` section is for local `docker compose up --build`.
|
||||
- `GET /healthz` backs the image's `HEALTHCHECK`.
|
||||
- On startup, runs left without an outcome (the process was killed mid-run) are closed as failed. The next run carries on from the saved item state.
|
||||
- Operations are covered in [runbook.md](runbook.md).
|
||||
@@ -226,6 +227,7 @@ src/lib/styles/app.css # design tokens + shared styles
|
||||
src/routes/ # projects, project status + history, admin, healthz
|
||||
drizzle/ # generated SQL migrations (npm run db:generate)
|
||||
Dockerfile docker-compose.yml
|
||||
.gitea/workflows/ci.yml # tests on PRs and main; image build + push on main
|
||||
```
|
||||
|
||||
## Open questions (resolved by spikes)
|
||||
@@ -246,4 +248,4 @@ Dockerfile docker-compose.yml
|
||||
6. **ApsDocumentTarget** + ACC folder URL resolver, including the fallbacks chosen in step 1.
|
||||
7. **Wiring (done):** coordinator, scheduler, composition root, `init` hook, history pruning, simulated mode.
|
||||
8. **UI (done):** projects list, project status (title block, issues with retry, file tree), 7-day history with per-run detail, Run now with cooldown, admin login, add/edit/resume/reset/remove.
|
||||
9. **Ship (done):** Docker image, Portainer stack file, runbook. The image was tested locally. A first Portainer deploy is still to do.
|
||||
9. **Ship (done):** Docker image, Portainer stack file, Gitea Actions CI, runbook. The image was tested locally. The first CI run and Portainer deploy are still to do.
|
||||
|
||||
@@ -4,7 +4,7 @@ How to deploy, update, back up and troubleshoot the sync server.
|
||||
|
||||
## What runs
|
||||
|
||||
- **One container**, built from the `Dockerfile`. Never run more than one copy: the locks that stop two syncs of the same project overlapping live inside the process.
|
||||
- **One container**, from the image CI builds (see [Continuous integration](#continuous-integration)). Never run more than one copy: the locks that stop two syncs of the same project overlapping live inside the process.
|
||||
- **One volume**, `sync-data`, mounted at `/data`. It holds `sync.db`, the SQLite database with the projects, the record of which ACC item belongs to which SharePoint item, and the last 7 days of history.
|
||||
- **Port 3000** inside the container. `HOST_PORT` sets the port on the server (default 3000).
|
||||
- **Health check:** `GET /healthz` answers `{"status":"ok"}` once the server has started. Docker checks it every 30 seconds, and Portainer shows the result as healthy or unhealthy.
|
||||
@@ -15,7 +15,7 @@ How to deploy, update, back up and troubleshoot the sync server.
|
||||
Decide this before the first deploy, because it changes how the image is built.
|
||||
|
||||
- **Behind an HTTPS reverse proxy** (recommended). The proxy must pass the original `Host` header on, which most do by default. Leave `ORIGIN` empty.
|
||||
- **Plain HTTP** on the VPN, for example `http://sync-server:3000`. Set `ORIGIN` to exactly the address people will type, with no trailing slash. `ORIGIN` is fixed when the image is built, so the app only works at that one address: not by IP if `ORIGIN` uses a name, and not on a different port.
|
||||
- **Plain HTTP** on the VPN, for example `http://sync-server:3000`. Set `ORIGIN` to exactly the address people will type, with no trailing slash. `ORIGIN` is fixed when the image is built, so the app only works at that one address: not by IP if `ORIGIN` uses a name, and not on a different port. Because CI builds the image, `ORIGIN` is a variable in Gitea, not a Portainer setting (see below).
|
||||
|
||||
If `ORIGIN` is wrong or missing over plain HTTP, pages still load, but every button that submits something fails with "Cross-site POST form submissions are forbidden".
|
||||
|
||||
@@ -23,42 +23,70 @@ If `ORIGIN` is wrong or missing over plain HTTP, pages still load, but every but
|
||||
|
||||
Set these in the Portainer stack's **Environment variables**, or in a `.env` file next to `docker-compose.yml` when using the command line. Never put them in the repository.
|
||||
|
||||
| Variable | Required | Default | What it is |
|
||||
| ----------------------------- | --------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `ADMIN_USERNAME` | yes | | Login for the admin pages |
|
||||
| `ADMIN_PASSWORD` | yes | | Password for the admin pages |
|
||||
| `SESSION_SECRET` | yes | | Signs admin logins. Any long random string, e.g. the output of `node -e "console.log(crypto.randomBytes(32).toString('hex'))"`. Changing it logs admins out |
|
||||
| `ORIGIN` | see above | | The app's address when served over plain HTTP. Build time only |
|
||||
| `HOST_PORT` | no | `3000` | Port on the server |
|
||||
| `SYNC_BACKEND` | no | `live` | `simulated` runs the in-memory demo instead (nothing is saved) |
|
||||
| `TZ` | no | `America/Chicago` | Time zone the status pages show times in |
|
||||
| `POLL_INTERVAL_SECONDS` | no | `180` | How often every project is checked for changes |
|
||||
| `MANUAL_RUN_COOLDOWN_SECONDS` | no | `30` | Minimum gap between "Run now" presses for one project |
|
||||
| `MAX_ATTEMPTS` | no | `3` | Failed tries before a file is reported as an issue |
|
||||
| `HISTORY_RETENTION_DAYS` | no | `7` | How long sync history is kept |
|
||||
| Variable | Required | Default | What it is |
|
||||
| ----------------------------- | -------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `ADMIN_USERNAME` | yes | | Login for the admin pages |
|
||||
| `ADMIN_PASSWORD` | yes | | Password for the admin pages |
|
||||
| `SESSION_SECRET` | yes | | Signs admin logins. Any long random string, e.g. the output of `node -e "console.log(crypto.randomBytes(32).toString('hex'))"`. Changing it logs admins out |
|
||||
| `HOST_PORT` | no | `3000` | Port on the server |
|
||||
| `IMAGE` | no | latest from CI | Image to run. Set it to a commit tag to roll back (see [Rolling back](#rolling-back)) |
|
||||
| `SYNC_BACKEND` | no | `live` | `simulated` runs the in-memory demo instead (nothing is saved) |
|
||||
| `TZ` | no | `America/Chicago` | Time zone the status pages show times in |
|
||||
| `POLL_INTERVAL_SECONDS` | no | `180` | How often every project is checked for changes |
|
||||
| `MANUAL_RUN_COOLDOWN_SECONDS` | no | `30` | Minimum gap between "Run now" presses for one project |
|
||||
| `MAX_ATTEMPTS` | no | `3` | Failed tries before a file is reported as an issue |
|
||||
| `HISTORY_RETENTION_DAYS` | no | `7` | How long sync history is kept |
|
||||
|
||||
The Microsoft Graph and APS credentials will be added to this table when the live connections are built.
|
||||
|
||||
## Continuous integration
|
||||
|
||||
`.gitea/workflows/ci.yml` runs on Gitea Actions:
|
||||
|
||||
- **Every pull request and every push to main:** lint, type check, tests and a production build. A pull request whose checks fail shouldn't be merged.
|
||||
- **Pushes to main, once those pass:** builds the Docker image and pushes it to this Gitea server's container registry as `primegit.primecontrols-dev.com/c.west/sharepoint-acc-sync`, tagged `latest` and with the commit's short hash (for example `1a2b3c4`). It then asks Portainer to redeploy, if a webhook is set.
|
||||
|
||||
One-time setup, in the repository's **Settings → Actions**:
|
||||
|
||||
| Name | Kind | Required | What it is |
|
||||
| ----------------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `REGISTRY_TOKEN` | secret | yes | Gitea access token with the `write:package` scope, from **User settings → Applications**. Used to push the image |
|
||||
| `REGISTRY_USERNAME` | variable | no | The account that owns `REGISTRY_TOKEN`. Defaults to the repository owner |
|
||||
| `ORIGIN` | variable | no | Only for plain HTTP (see [above](#choose-how-people-will-reach-it)). Changing it takes effect from the next image built |
|
||||
| `PORTAINER_WEBHOOK_URL` | secret | no | The stack's webhook URL (see [First deploy](#first-deploy-portainer)). Without it, the image is pushed but the stack has to be redeployed by hand |
|
||||
|
||||
The repository also needs Actions turned on (**Settings → Repository → Actions**) and a runner that can run Docker commands. Gitea's own runner does by default. If the image job fails at `docker login` with "command not found" or "Cannot connect to the Docker daemon", ask whoever looks after the runner.
|
||||
|
||||
## First deploy (Portainer)
|
||||
|
||||
1. **Stacks → Add stack**, give it a name such as `sharepoint-acc-sync`, and choose **Repository**.
|
||||
2. Enter the repository URL and the branch (`refs/heads/main`). The compose path is `docker-compose.yml`. If the repository is private, turn on authentication and use a read-only access token.
|
||||
3. Under **Environment variables**, add the settings above.
|
||||
4. **Deploy the stack.** Portainer builds the image from the `Dockerfile`, which takes a few minutes the first time.
|
||||
5. Check it's working:
|
||||
1. Push to main and wait for the CI run to finish, so there's an image to pull.
|
||||
2. If the image isn't public, add the registry under **Registries → Add registry → Custom registry**: URL `primegit.primecontrols-dev.com`, with a Gitea access token that has the `read:package` scope.
|
||||
3. **Stacks → Add stack**, give it a name such as `sharepoint-acc-sync`, and choose **Repository**.
|
||||
4. Enter the repository URL and the branch (`refs/heads/main`). The compose path is `docker-compose.yml`. If the repository is private, turn on authentication and use a read-only access token.
|
||||
5. Under **Environment variables**, add the settings above.
|
||||
6. To redeploy automatically after each CI run, turn on **GitOps updates**, choose **Webhook** and turn on **Re-pull image**. Copy the webhook URL into the `PORTAINER_WEBHOOK_URL` secret in Gitea.
|
||||
7. **Deploy the stack.**
|
||||
8. Check it's working:
|
||||
- the container shows as **healthy** in Portainer;
|
||||
- the logs show `Sync server started` with the expected `backend`;
|
||||
- the projects page opens at the chosen address, and the admin login works.
|
||||
|
||||
From the command line on the Docker host, the equivalent is `docker compose up -d --build` in a clone of the repository, with the settings in `.env`.
|
||||
From the command line on the Docker host, the equivalent is `docker compose up -d` in a clone of the repository, with the settings in `.env`. `docker compose up -d --build` builds the image locally instead of pulling it.
|
||||
|
||||
## Updating
|
||||
|
||||
1. In Portainer, open the stack and use **Pull and redeploy**. Make sure the image is rebuilt, not reused: after the first update, check that the change is actually live.
|
||||
2. On the command line: `git pull`, then `docker compose up -d --build`.
|
||||
Merge to main. Once CI passes, the new image is pushed and, with the webhook set, Portainer redeploys it. Without the webhook, open the stack in Portainer and use **Pull and redeploy**. Every deploy pulls the newest image, so a redeploy never runs an old one.
|
||||
|
||||
On the command line: `git pull`, then `docker compose up -d`.
|
||||
|
||||
When the container stops, a sync in progress gets up to 2 minutes to finish. If it doesn't finish in time, it's marked in the history as stopped by a restart, and the next sync picks up whatever was left. Nothing needs cleaning up by hand.
|
||||
|
||||
## Rolling back
|
||||
|
||||
Every image CI pushes is kept under its commit's short hash. Find the hash of the last good commit in Gitea (or under **Packages** on the repository owner's page), set the stack's `IMAGE` variable to `primegit.primecontrols-dev.com/c.west/sharepoint-acc-sync:<hash>`, and redeploy. Remove `IMAGE` again to go back to following `latest`.
|
||||
|
||||
A rollback doesn't undo database changes, and an older image may not work with a database a newer one has changed. Before deploying a release that adds a migration (a new file in `drizzle/`), take a [backup](#backups), and restore it if you roll back past that release.
|
||||
|
||||
## Backups
|
||||
|
||||
Back up `sync.db`. Losing it means the tool forgets which ACC files it created. It would then treat every SharePoint file as new, and could not tidy up the ACC copies it made before.
|
||||
@@ -74,11 +102,14 @@ To restore one: stop the stack, put the backup in the volume as `sync.db`, delet
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause and fix |
|
||||
| ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Deploy fails with "required variable … is missing a value" | A required setting isn't set. Add it to the stack's environment variables |
|
||||
| Buttons fail with "Cross-site POST form submissions are forbidden" | Plain HTTP without the right `ORIGIN`, or the app was opened at a different address. Set `ORIGIN` and rebuild |
|
||||
| Admin login appears to work but doesn't stick | Plain HTTP without `ORIGIN`: the login cookie is marked HTTPS-only, so the browser drops it. Set `ORIGIN` and rebuild |
|
||||
| Container is unhealthy or keeps restarting | Check the logs. Bad settings are reported by name at startup |
|
||||
| Times on the pages are off by some hours | `TZ` doesn't match the office's time zone |
|
||||
| A project shows **Paused** | Its SharePoint folder was deleted or couldn't be found. Nothing was deleted in ACC. Fix the folder, then **Resume** it on the admin page |
|
||||
| Symptom | Likely cause and fix |
|
||||
| ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Deploy fails with "required variable … is missing a value" | A required setting isn't set. Add it to the stack's environment variables |
|
||||
| Buttons fail with "Cross-site POST form submissions are forbidden" | Plain HTTP without the right `ORIGIN`, or the app was opened at a different address. Set the `ORIGIN` variable in Gitea and rerun CI |
|
||||
| Admin login appears to work but doesn't stick | Plain HTTP without `ORIGIN`: the login cookie is marked HTTPS-only, so the browser drops it. Set the `ORIGIN` variable in Gitea and rerun CI |
|
||||
| Container is unhealthy or keeps restarting | Check the logs. Bad settings are reported by name at startup |
|
||||
| Deploy fails pulling the image ("unauthorized" or "not found") | The registry isn't added in Portainer, its token lacks `read:package`, or CI hasn't pushed an image yet. See First deploy |
|
||||
| CI's image job fails at "Log in to the registry" | `REGISTRY_TOKEN` is missing, expired or lacks `write:package`, or `REGISTRY_USERNAME` doesn't match the token's owner |
|
||||
| A merge passed CI but the old version is still running | No webhook is set, or it failed (see the last step of the CI run). Use **Pull and redeploy** in Portainer |
|
||||
| Times on the pages are off by some hours | `TZ` doesn't match the office's time zone |
|
||||
| A project shows **Paused** | Its SharePoint folder was deleted or couldn't be found. Nothing was deleted in ACC. Fix the folder, then **Resume** it on the admin page |
|
||||
|
||||
Reference in New Issue
Block a user