Deployment

deploy.ps1 at the repository root deploys code and applies migrations. sync-docs.ps1 publishes the documentation — see Publishing the Docs. The two are deliberately separate.

.\deploy.ps1 -DryRun   # print the plan, write nothing
.\deploy.ps1           # the everyday deploy

Requires PowerShell 7. PowerShell 5.1 mangles UTF-8 in this repository's scripts into what look like parse errors.

A plain run does not touch data

.\deploy.ps1 pulls, builds, applies pending migrations and restarts containers. Replacing production data from a dump is opt-in and needs two flags: -RestoreDump -ConfirmReplace sis.

Three instances on one hostname

Instance	Path	Database	Backend service	Notes
sis	/sdasis	sis	sdasis-backend	The primary. The only one ever restored from a dump
demo	/demo	sis_demo	demo-backend	Sanitized, FERPA-safe. DEMO_MODE=true
records	/records	sis_prod	records-backend	True production, WRITE_MODE=restricted, EMAIL_DELIVERY=capture

They are described in one $INSTANCES registry in deploy.ps1, which also records each instance's UI Text overlay directory and compose profile. That used to be two structures 1,400 lines apart, so adding an instance meant remembering both.

Secondaries are only ever migrated, never restored. Every backend runs the same Go binary, so schema drift breaks a secondary exactly as it would break sis.

records sits behind profiles: ["records"], and the script issues compose commands with no --profile and no --remove-orphans. Compose therefore ignores it entirely on an ordinary run — not built, not recreated, not stopped, not reported as an orphan — so production keeps serving its own image while the testing tier is redeployed underneath it. Include it explicitly with -Include records.

Routing and the API prefix

Caddy serves the same site on two hostnames — sdasis.puc.edu and forge.puc.edu, the first being a CNAME for the second. Automatic HTTPS holds a separate certificate for each. It routes by path prefix and strips nothing:

Path	Target	Reachable from
/sdasis, /sdasis/*	sdasis-frontend:80	10.0.0.0/8
/mailpit*	mailpit:8025, behind basic auth	10.0.0.0/8
/records, /records/*	records-frontend:80	anywhere
/demo, /demo/*	demo-frontend:80	anywhere
/demomail*	mailpit-demo:8025, no auth	anywhere
/docs, /docs/*	docs:80	anywhere
/limesurvey*	redirect to survey.puc.edu	anywhere
/omr, /omr/*	omr-auth:4180 (oauth2-proxy in front of the OMR scantron portal)	anywhere
/pay/api/v1/aci/postback	aci-api:8100, prefix stripped	anywhere
/pay/* (everything else)	aci-api:8100 for /pay/api/*, else aci-web:80, prefix stripped; a bare /pay redirects to /pay/	10.0.0.0/8
/ on forge.puc.edu	static "Welcome to the FORGE Demo server" page linking /demo, /demomail, /docs and each guide	anywhere
/ on sdasis.puc.edu	/sdasis/ from 10/8, /demo/ otherwise	anywhere

Changing the Caddyfile. deploy.ps1 reloads Caddy whenever the pull changed anything under caddy/ (up -d alone never picks up a Caddyfile edit). For a Caddyfile-only change, such as rewording the forge.puc.edu welcome page, use the one-step script instead:

.\sync-caddy.ps1 -Message "Forge welcome page: reword intro"   # validate, commit caddy/, push, pull, reload
.\sync-caddy.ps1 -DryRun -Message "..."                       # validate and show the plan only

It validates the file in a local caddy:2-alpine container before committing, commits only caddy/, and refuses when unpushed commits touch anything else (use deploy.ps1, or -AllowOtherCommits). A config Caddy cannot load is refused by the reload and the running one stays live. The reload prints "Unnecessary header_up" and "not formatted" warnings; both are expected.

10.0.0.0/8 covers on-campus clients and off-campus ones arriving through Zscaler Private Access alike — the ZPA App Connector originates the connection from inside that range, so there is nothing Zscaler-specific in the Caddyfile. Who may reach the app over ZPA is a ZPA policy question.

Anything restricted, asked for from outside 10/8, is answered with a 403 explanation page saying the address needs the campus network or ZPA, and linking to /demo and /docs. It deliberately does not redirect into /demo: the demo is a pixel-identical copy of the application running on sanitized data, so silently landing someone there invites them to read demo records as real ones. The page is static by design — it must never echo the requested path back, because Caddy does not HTML-escape placeholders and this page is served to the open internet.

/sdasis/api/* and /records/api/* get a plain 403 instead of the page, so that a laptop which loaded the app on campus and then roamed reports a clean error rather than parsing an HTML page as JSON. A bare / still 302s to /demo/ — arriving at the hostname is not an access failure.

The gate is an allow-list of public paths, so a new public path is 10/8-only until it is added to @restricted in the Caddyfile.

/records is campus-only

/records is the sis_prod instance — real student records. It was public from Sep 2026 until 2026-11-03, when it was gated again: it now answers only from 10/8 (on campus or through ZPA), exactly like /sdasis, and /records/api/* is in @restrictedAPI. Off-campus students and proxies without ZPA cannot reach it. Do not add it back to the public list without asking.

/omr is public on purpose, too

/omr is the OMR scantron-grading portal, a separate compose project (~/apps/omr) that joins this network the way limesurvey does; deploy.ps1 does not build or restart it. Since Sep 2026 it is reachable from the open internet so faculty can grade from home without ZPA. Entra sign-in is the only boundary, and a real one: the app registration requires assignment to the faculty group. What Caddy reaches is omr-auth (oauth2-proxy), never the portal, which trusts the X-Forwarded-Email header the proxy sets and so must not be reachable directly. The /omr prefix is not stripped — oauth2-proxy emits its own /omr/oauth2/start links.

Opening a path takes two edits, and the second is easy to miss: add it to @restricted, and make sure none of its endpoints is in @restrictedAPI, which is answered first with a plain 403 from outside 10/8. /omr's polling endpoints used to be listed there; leaving them would have let an off-campus user load the portal and watch a grading run hang forever on a 403'd progress poll.

/pay: only the ACI postback is public

/pay is the ACI card-payment portal, another separate compose project (~/apps/aci) on this network. ACI's own server calls /pay/api/v1/aci/postback from outside 10/8 by definition, so that one path is in @restricted; the rest of /pay — the payer's browser, including the return from ACI to /pay/api/v1/aci/return — stays behind the gate. It is deliberately one path, not /pay/api/*: until ACI supplies its source ranges (ACI_POSTBACK_ALLOWED_IPS), the postback accepts a result from anyone who knows a payment uid. Opening the rest of /pay is the go-live step.

Unlike /omr, the prefix is stripped, and Caddy overwrites X-Forwarded-For on the API: the postback's source-address allowlist reads its first element, so an appended header would let a caller name its own address. @payAPI must stay before @pay, since this route runs in written order and /pay/* matches the API too.

The gate is a Caddy rule, not a firewall rule

It only covers what passes through Caddy. Anything a container publishes on a host port answers the internet directly, skipping both the gate and any basic_auth — which is why mailpit binds its published ports to 127.0.0.1 in docker-compose.prod.yml. Keep new port publications loopback-bound unless you mean otherwise.

Stripping happens one layer down, in each frontend's nginx, which is a template with __BASE_PATH__ substituted at image build time. /sdasis/api/v1/x reaches the Go backend as /api/v1/x.

The backend cannot infer its own prefix

Because nginx removes the prefix before proxying, the backend never sees it — so PUBLIC_API_BASE_PATH must be set per instance to match that instance's VITE_API_BASE_URL (/sdasis/api/v1, /demo/api/v1, /records/api/v1). It is what server-built URLs (photo and avatar src) are prefixed with. Get it wrong and images 404 while everything else works.

COOKIE_PATH is per instance for the same reason — three instances sharing a hostname would otherwise overwrite each other's session. SAML_SP_BASE_URL must keep the prefix.

Cookies are host-only (COOKIE_DOMAIN is unset), so sdasis.puc.edu and forge.puc.edu hold entirely separate sessions — you log in once per hostname. For the same reason SSO started on forge.puc.edu finishes on sdasis.puc.edu; see the comment above CORS_ALLOWED_ORIGINS in docker-compose.prod.yml for why adding forge to that list is a trap rather than the fix it looks like.

There is no /api route at the Caddy level, by design.

Changing the Caddyfile

It lives at caddy/Caddyfile, and docker-compose.prod.yml mounts the directory./caddy at /etc/caddy — not the file. Apply a change with:

cd ~/apps/sdasis && git pull --ff-only && ./scripts/caddy-reload.sh

That validates, checks the container is seeing the file you just pulled, and reloads gracefully. No restart, no dropped connections, the other instances never notice.

Do not mount the Caddyfile as a single file

It used to be ./Caddyfile:/etc/caddy/Caddyfile, and that is a trap worth stating plainly, because it fails silently in both directions.

Docker binds a single file by inode. git pull replaces the file rather than rewriting it, so after a pull the container kept serving the previous config — and caddy validate and caddy reload both reported success, because they were reading the same stale inode. The running config was live, valid, and out of date, and nothing anywhere said so. Only recreating the container picked the change up, which is why deploy.ps1 masked this for as long as it did: a full deploy recreates containers, so the bug only bit config-only changes applied by hand.

A directory mount resolves each file on access, so a pulled change is visible immediately. scripts/caddy-reload.sh compares the host and container checksums and refuses to reload if they differ, so if anyone converts this back to a file mount it says so instead of quietly applying the old config.

What a run does

1. SSH preflight
2. Find the latest local dump (only relevant to a restore)
3. git pull on the server
4. Ensure .env.local is present, and diff it name-only against every ${VAR} the compose file references — including a "defined but empty" list, since compose treats empty as unset
5. Decide what work is needed (build detection, restore detection)
6. Start the staged database restore, if requested — in the background
7. Build the Go migration binaries
8. docker compose build
9. Collect the staged restore and swap it into place
10. Apply pending migrations to every instance's live database
11. docker compose up -d
12. Report UI Text overrides this release orphaned or left out of date
13. Verify: container health, search_path, backend version, mailpit, crontab drift

The staged restore

A restore lands in sis_staging and is renamed into place with ALTER DATABASE RENAME, concurrently with the image build. sis_old is only the momentary parking name for the outgoing database during the swap. Both are dropped WITH (FORCE) before use, so an interrupted run cannot leave one behind eating disk — the server runs at about 85% full.

No swap happens if the build fails.

Migrations

Pending migrations are applied through the ledger, core.schema_migrations. Because the ledger lives inside the database, it travels inside pg_dump, and a restored database inherits its source's applied set.

.\scripts\db-migrate.ps1                      # what is pending (read-only)
.\scripts\db-migrate.ps1 -Apply               # apply, in order
.\scripts\db-migrate.ps1 -Target Prod -Status # the same question about production
.\scripts\db-migrate.ps1 -Baseline            # once per database: record on-disk files as applied

Pick a migration number from the ledger, not from disk

A ledger row whose file is absent from disk is a claimed number — another session may be working in a parallel tree. A reused number blocks the next deploy. Never replay history from 001; it cannot succeed.

See Migrations for the full rules.

Useful flags

Flag	Use
-DryRun	Print the plan, write nothing
-Include demo / -Include records	Bring a secondary instance into the run
-RestoreDump -ConfirmReplace sis	Replace the primary's data from a dump
-ForceBuild	Build even when change detection says it is unnecessary
-NoBuildCache	Build with --no-cache
-SkipMigrations	Deploy code without applying migrations
-PruneLocaleOverrides	Drop orphaned UI Text overrides, behind a history snapshot
-Trace	Verbose

"CACHED" plus "/app/dist not found" is cache poisoning

That specific pair is a poisoned BuildKit cache. The fix is -NoBuildCache, not-ForceBuild — forcing a build reuses the same poisoned layers.

Related

• Publishing the Docs
• Migrations
• Sanitizing the Database — how the demo tier's data is produced
• Scheduled Jobs — the tracked production crontab