Scheduled Jobs

Ten jobs run from the production host's crontab. Each one is wrapped so that it logs and mails its result, because for most of this system's life it did neither usefully.

The schedule

When	Job	What it does
Sun 02:00	reset-demo	Reloads sis_demo from its sanitized artifact, undoing a week of visitors, then applies the migrations the artifact predates. Reload only — producing a new artifact is a manual workstation job (Sanitizing the Database).
04:30	omr-backup	Data snapshot for the OMR scantron portal (a separate compose project in ~/apps/omr), ahead of the database backup so the two never contend for disk. Uses this repo's cron-run.sh.
05:30	backup	Database dumps, deliberately before the import, so the daily restore point is the state prior to that day's delta. 14-day retention, ~220 MB per database.
Sun 05:45	migrate-weekly.sis_prod	Full (non-delta) run of the sections, enrollments and instructors steps into production. A delta never re-reads a credit that did not change, and a section Colleague renames changes the section, not its students' credits, so only a full enrollments read moves them (ODS Data Migration). The per-database flock keeps it and the delta from interleaving.
Sun 06:15	migrate-weekly.sis	Same, into the testing tier.
06:30	migrate-daily.sis_prod	Nightly ODS delta import into production (ODS Data Migration).
07:00	migrate-daily.sis	Same, into the testing tier.
07:30	sync-photos.sis_prod	Official photo repository sweep, after both imports so newly imported people are there to match against.
07:45	sync-photos.sis	Same, staggered so the two do not hit the SMB share at once.
08:00	omr-secret-expiry	Checks the OMR portal's Entra client secret. Runs --quiet and mails only when the secret expires within 30 days, has expired, or has no date set — the one entry here that does not mail every run, because its honest answer is "nothing to do" on nearly every day.

The times dodge the API's own in-process jobs — role derivation at 03:00 and the student-status recompute at 04:00, both in cmd/api/main.go. Those are not covered by any of this: they log inside the container and mail nothing. Nor is the staff-email outbox worker, which runs continuously in every API process (a 5-second tick, throttled to EMAIL_SEND_PER_MINUTE).

The two OMR entries were first added to the host crontab by hand; they are tracked now so the deploy's drift check stops reporting them and install-crontab.sh does not delete them.

The demo is absent from the photo sweep on purpose. Its people are fabricated and it must never touch the official share.

Installing it

scripts/records/crontab.example is the tracked copy of what is installed. Install it with:

cd ~/apps/sdasis
./scripts/install-crontab.sh

It snapshots the existing crontab into logs/crontab.backup.<stamp> first (crontab <file> replaces outright, with no confirmation and no undo), installs the tracked copy, and reads it back to confirm. Running it twice is running it once. ./scripts/install-crontab.sh --check compares without installing.

Do not install it by hand. That is not style advice. The crontab was a hand-run crontab -e until 2026-09-02, and it drifted the first chance it got: the mail wrapper below shipped on 2026-08-30 with every tracked entry rewritten to use it, and three days later the host was still running the pre-wrapper crontab. The jobs ran, the logs filled, and not one mail had ever been sent.

deploy.ps1 now hashes the installed crontab against the tracked one after every release and warns if they differ, so the same drift reports itself.

Reading the mail

Every run mails its result to mailpit (basic auth), via scripts/cron-run.sh. Subjects carry the outcome:

[OK] sdasis migrate-daily.sis_prod (4m18s)
[FAIL rc=1] sdasis backup (12s)

The body carries host, job, command, start and finish times, duration, exit code and log path, followed by the run's captured output. Output longer than 256 KB is split head-and-tail with an elision marker rather than truncated, because the two useful ends of a long run are opposite ones: what it set out to do, and what it died of.

Every run mails, not only the failures — so silence means cron itself has stopped. (omr-secret-expiry is the deliberate exception above.) That is the point of the policy rather than an oversight in it. A failures-only channel cannot tell a healthy job from one that is no longer scheduled at all, and this host has been in the second state twice: it had no crontab whatsoever when /records went live, and it ran an unwrapped one for three days after the wrapper shipped. Both looked exactly like success.

Mailpit's inbox persists across container recreates (MP_DATABASE on the mailpit_data volume). /demomail — the demo's separate, password-less mailbox — deliberately does not.

The wrapper

scripts/cron-run.sh <job-name> -- <command> [args...]

It appends to logs/<job-name>.log, live, so a long import still shows progress in the file while it runs. <job-name> is deliberately the log's basename, so nothing that reads those files had to change when the wrapper was introduced. Runs are bracketed in the log:

=== 2026-09-02T05:30:01-07:00 cron-run START backup: ./scripts/backup-db.sh
=== 2026-09-02T05:31:47-07:00 cron-run END backup [OK] (1m46s)

Grepping a log for cron-run is therefore the quickest way to tell whether a job is running under the wrapper at all.

There is no MTA on the host, and installing one would be a host dependency living outside this repo, so the wrapper speaks SMTP straight to mailpit over bash's /dev/tcp on the 127.0.0.1:1025 that mailpit already publishes. No new binary and no build step — so it still works on the night the Go toolchain or Docker is what broke.

Losing the mail never turns a successful backup into a failed one: a send failure is reported into the log as a cron-run WARNING line and the job's own exit status is still what the wrapper returns.

Configuration, all optional, read from the crontab's own environment rather than by Docker Compose (defaults in the script): CRON_MAIL_TO, CRON_MAIL_FROM, CRON_MAIL_HOST, CRON_MAIL_PORT, CRON_MAIL_TIMEOUT, CRON_MAIL_MAX_BYTES, CRON_MAIL_HEAD_BYTES. CRON_MAIL_TO defaults to cron@sdasis.puc.edu, which is undeliverable by design — mailpit captures everything and delivers nothing, so the address only has to be stable enough to filter on.

Troubleshooting

Logs are being written but no mail arrives. The crontab is almost certainly not the tracked one. Check with crontab -l | grep -c cron-run — zero means the jobs are running unwrapped. Fix with ./scripts/install-crontab.sh.

Nothing at all in the inbox, and the crontab looks right. Check grep 'cron-run WARNING' logs/*.log for send failures, and that mailpit is up with 1025 reachable (docker ps, ss -lntp | grep 1025). deploy.ps1 checks both after every release.

Test without waiting for 05:30. ./scripts/cron-run.sh smoketest -- echo hi should produce an [OK] sdasis smoketest mail; -- false should produce [FAIL rc=1] and exit 1. Delete logs/smoketest.log afterwards.