Photo Repository

Permission nodes: PEOPLE.PHOTO_SYNC, PEOPLE.PHOTO_SUBMISSION_QUEUE, PEOPLE.PHOTO_APPROVAL_QUEUEAPI: /api/v1/people/photo-sync, /api/v1/people/photo-submissions

The institution's official photo repository is an SMB share on a Windows Server, and it is the source of truth. SIS holds read-only credentials for it and mirrors it; nothing in the codebase writes to the share.

Official photos are submitted through an external Microsoft Form that requires its own authentication. SIS cannot make that submission on anyone's behalf, which is the single fact that shapes everything below.

Where the bytes live

Files, not database rows. Under PHOTO_STORAGE_PATH:

$PHOTO_STORAGE_PATH/
  official/{first 3 digits}/{7-digit ID}_{thumb|display|original}.jpg
  uploads/{photo_id}/{thumb|display|original}.jpg

core.person_photos is a catalogue of what is on disk, not a store of images.

Three sizes are generated at ingest, never per request: a 64 px thumbnail (~1.5 KB), a 400 px display image (~15 KB), and a normalised original (~60 KB). Rosters ask for ?size=thumb, which is what makes a 200-student photo roster move ~300 KB instead of tens of megabytes.

Everything is re-encoded to JPEG, so the extension on disk is always a lowercase .jpg even where the share holds .JPG or .png. Re-encoding also bakes in EXIF orientation — necessary, because dropping the EXIF block without rotating the pixels would leave phone photos sideways.

The official/ tree is a cache: delete it and the next sweep rebuilds it. The uploads/ tree is not — it holds photos that exist nowhere else yet.

Which photo a person shows

A person can have two live rows at once, one per origin, because the share may hold a photo for them while an upload is in flight. Which one wins is resolved per request, in this order:

#	Row	Why it wins
1	Local upload, approved, not yet submitted	The most recent decision made inside SIS
2	Local upload, submitted, not yet superseded	See below — submitting must not blank the photo
3	Official repository photo	The normal case
4	Local upload, pending review	Visible only to the uploader and to reviewers

A rejected photo never wins.

The hand-off back to the share

When an approved upload is submitted to the official repository, it keeps serving. It is only dropped once an official file appears whose modification time is newer than the submission — that is the evidence the repository actually took it. Cutting over at the moment of submission would blank the person's photo for however many days the manual pipeline takes.

Uploading in SIS is not submitting

Action	Endpoint	Gate
Photo metadata	GET /people/persons/{personID}/photo	self-service
The image	GET /people/persons/{personID}/photo/file?size=	self-service
Upload	POST /people/persons/{personID}/photo	self-service
Delete	DELETE /people/persons/{personID}/photo	self-service

Self-service on purpose: the handler performs a stricter check than a node gate could — you may act on your own photo, or on someone else's only with PEOPLE.PHOTO_UPLOAD. A node gate would run first and refuse students uploading their own picture, which is the main thing the feature is for.

Deleting only ever removes a local upload. A request to delete an official photo is refused with an explanation rather than silently succeeding, because the share is read-only and the next sweep would simply restore the row.

The submission queue

PEOPLE.PHOTO_SUBMISSION_QUEUE — /people/photo-submissions

Approved uploads that still have to reach the official repository. Until the hand-off is automated this is deliberately manual: open the photos, put them through the official form, then Mark submitted. Marking does not hide anything from students; see the precedence table above.

The form URL is the photo_official_upload_url setting, not a hardcoded link.

The sweep

PEOPLE.PHOTO_SYNC — /people/photo-sync

Mirrors the share into the local cache. Runs nightly from cron on the prod host, which invokes the binary inside the backend container, and on demand from this screen. Both paths take the same Postgres advisory lock, so a scheduled run and an impatient user cannot collide -- the loser reports that a sync is already running.

Change detection is free: one directory listing yields every file's size and modification time, so a sweep that finds nothing new opens no files at all. A steady-state sweep of a large share is a single round trip plus a couple of queries.

Every run is recorded in core.photo_sync_runs and shown on the screen.

Aborted is not the same as failed

Two guards exist because a share that has gone wrong looks exactly like a share that has gone empty:

• Zero files listed aborts the run. A dropped mount, a revoked credential or a renamed share all present as an empty directory rather than an error, and treating that as "nobody has a photo any more" would wipe the whole mirror during a transient outage.
• Too many retirements aborts the run. If a sweep would retire more than photo_sync_churn_threshold_pct (default 20%) of the photos we hold, it refuses. This catches what the first guard cannot: a share that is intact and fully populated, but whose filenames have stopped matching our ID numbers. Then the file count looks perfectly healthy while every photo looks deleted.

An aborted run changes nothing — every guard runs before the first write. ABORTED shows as a warning rather than an error for that reason.

Both are bypassable with sync-photos -force, which should only be used once the reason for the drop is actually understood.

Running it by hand

go run ./cmd/sync-photos -dry-run                # report the plan, write nothing
go run ./cmd/sync-photos                         # normal run
go run ./cmd/sync-photos -limit 20               # smoke-test against the real share
go run ./cmd/sync-photos -dir /mnt/photos        # read a local directory instead of SMB
go run ./cmd/sync-photos -force                  # bypass the guards
go run ./cmd/sync-photos -memory-limit-mb 512    # tighten the heap ceiling

-dir is how the sweep is exercised on a machine with no share credentials -- a developer can point it at a mapped drive -- and how the empty-directory guard is verified without arranging for a real mount to fail. It is a convenience, not a deployment strategy: a mapped drive belongs to one interactive login session, so the scheduled sweep must always go through SMB.

What a sweep actually costs

Measured against the live repository (14,764 entries, 4.8 GB, median photo 40 KB but a tail of 7000x7000 / 30 MB scans):

Sweep	Files ingested	Wall clock	Peak RSS
First (empty cache)	14,652	33 min	~1.1 GB
Steady state (nothing changed)	0	1.8 s	64 MB
Typical incremental	25	1.7 s	78 MB

The gap between the first row and the others is the whole point of the design: change detection comes free with the directory listing, so a sweep that finds nothing new opens no files at all. Only the first sweep is expensive, and the resulting cache is ~610 MB (42.5 KB per person across the three sizes).

Memory, and why it is tunable

Peak memory is set by the largest single photo, not by how many run at once -- the pool is weighted by file size, so a 30 MB scan consumes the whole budget and effectively runs alone. Measured at concurrency 8, 4 and 2, peak and wall clock were identical.

What does move it is the soft heap ceiling (-memory-limit-mb, default 1024, or the standard GOMEMLIMIT environment variable). It is soft: the collector works harder as the heap approaches it rather than the process dying.

Ceiling	Peak RSS	Wall clock
1024 MB	1,075 MB	186 s
512 MB	795 MB	202 s
320 MB	491 MB	196 s

So the sweep fits whatever budget you give it, for single-digit percent more CPU. (Without the weighting and without a ceiling, the first full sweep peaked at 3.1 GB.)

Consequence for prod: give the first sweep an explicit ceiling (-memory-limit-mb 512), since it is the only bulk run. Every sweep after it is incremental at 64-78 MB, which is why the nightly job needs no special handling. Triggering a bulk re-ingest from the UI is the one thing that would put a gigabyte-scale spike inside a serving container, so wipe-and-rebuild belongs on the command line.

ID numbers are 7 digits

Files on the share are named for a person's ID number, zero-padded to 7 digits (0123456.jpg). Every core.people.person_id_number already satisfies this, so in practice the code validates rather than transforms. Anything that is not 1–7 digits is rejected outright rather than guessed at — a silent miss is indistinguishable from "this person has no photo", and that ambiguity is what lets such bugs survive. One records.students.student_number is 11 characters and correctly has no photo key.

Never derive a filename from an integer or from student_number; use photosync.PhotoKey.

Local development

PHOTO_STORAGE_PATH defaults to ./photos, so a sweep run from sis_backendgo/ fills sis_backendgo/photos/ — inside the working tree. Both that and uploads/ are gitignored, so they cannot be committed by accident.

Two things follow from living there. The tree holds real photographs of real people, so it should not be swept up into a backup or a copy of the repo. And git clean -xfd will delete it: harmless for official/, which the next sweep rebuilds, but uploads/ holds photos submitted through SIS that exist nowhere else yet and cannot be recovered. Move both out of the tree with PHOTO_STORAGE_PATH if that matters on a given machine.

Deployment notes

The sweep runs inside a backend container, not on the host. Prod's postgres publishes no ports, so nothing on the host can reach the database; the container already has both the connection and the SMB credentials. sync-photos is therefore built into the image next to server, and is invoked as:

DC="docker compose -f docker-compose.prod.yml --env-file .env.local"
$DC exec -T sdasis-backend ./sync-photos -dry-run
$DC exec -T sdasis-backend ./sync-photos -memory-limit-mb 512

docker compose exec inherits the container's environment, which is where DB_HOST=postgres and the PHOTO_SMB_* values come from. It is deliberately not in deploy.ps1's $GO_BINARIES -- that list is for host-run tools, and a host-run sweep cannot see the database.

Credentials live in .env.local on the server, and only there. Compose is invoked with --env-file .env.local and maps the SIS_/SIS2_-prefixed values onto the plain PHOTO_SMB_* names inside each container. Note that deploy.ps1 uploads .env.local only when it is missing, so editing a local copy does not propagate -- add the keys on the server. Keep it mode 600; it is gitignored and untracked, so a chmod there cannot block a git pull the way one on a tracked path would.

Every PHOTO_SMB_* reference in compose carries an explicit empty default (${VAR:-}), because having no photo repository configured is a legitimate state -- sis2 may never have one -- and a bare ${VAR} makes every compose invocation warn about it.

Editing .env.local is not enough on its own

A container's environment is fixed when the container is created, and docker compose exec runs inside the container that already exists. So an edit to .env.local has no effect until the container is recreated:

DC="docker compose -f docker-compose.prod.yml --env-file .env.local"
$DC up -d sdasis-backend

Prove the new value actually landed without printing it. Length and first character catch every mangling described below -- a stripped quote, an added backslash, a truncating escape:

fv=$(grep -m1 '^SIS_PHOTO_SMB_PASSWORD=' .env.local | cut -d= -f2-)
echo "file:      ${#fv} chars (includes any quotes)"
$DC exec -T sdasis-backend sh -c \
  'printf "container: %s chars, starts [%s]\n" ${#PHOTO_SMB_PASSWORD} "$(printf %.1s "$PHOTO_SMB_PASSWORD")"'

Quoting secrets in .env.local

Compose parses this file as dotenv and interpolates it, so several ordinary password characters change meaning. Wrap every secret in double quotes, and note what still bites:

In the file	The container receives
pw='abc	nothing -- hard parse error (unterminated quoted value), every compose command fails
pw=\'abc	\'abc -- the backslash is kept, so the password is wrong
pw="'abc"	'abc	correct
pw=ab$cd	ab -- $cd is interpolated as an unset variable and vanishes	escape as $$
pw="ab\ncd"	ab -- \n becomes a real newline inside double quotes	escape as \\n
pw="ab#cd", pw="ab\dcd"	unchanged	safe

A trailing $ is safe, which is why the share name IDPictures$ needs no escaping: there is nothing after it to interpolate.

None of these announce themselves. A mangled password comes back as "the attempted logon is invalid", which is the same thing the server says about a wrong account name.

The account may be given in any of three forms

PHOTO_SMB_USER accepts a user principal name (batch-0000001@puc.edu), a down-level name (PUC\batch-0000001), or a bare account with PHOTO_SMB_DOMAIN set separately. An explicitly configured domain always wins.

This is handled in SMBConfig.identity() because go-smb2 does no parsing of its own -- it copies User and Domain straight into the NTLM exchange -- so a UPN would otherwise go out as one opaque account name with an empty domain and be refused, while the identical value works in smbclient, which splits it.

sync-photos also needs IDENTIFIER_AES_KEY and IDENTIFIER_HMAC_KEY. It never touches encrypted identifiers, but it loads the shared config, which validates those keys unconditionally and on purpose. In a properly configured container they are already present; run the binary anywhere else and it fails with a message about encryption keys that has nothing to do with photos.

The photo trees must be bind-mounted. docker-compose.prod.yml mounts ./photos and ./uploads into each backend container. Without them both trees live on the container's writable layer and are destroyed on every container recreate -- which deploy.ps1 does routinely. The mirror is rebuildable; uploads/ is not.

Because the container runs as root and is the only writer, ownership is uniform and there is no uid collision to manage. (An earlier revision of this page described sharing the tree with a host-run binary and the group permissions that needed; that arrangement was never viable, for the reason in the first paragraph.)

The two instances get separate trees (photos/photos2). They have separate databases, so a shared cache would serve one instance's photo for the other's person.

Transport security. The client requires SMB message signing, so traffic is tamper-evident and cannot be silently downgraded to an unsigned dialect, and NTLMv2 keeps the password off the wire. There is no encryption in transit: this library implements none, so photo bytes and filenames cross the internal network in the clear. If that is unacceptable, it needs a different access route (a cifs mount with seal, or a tunnel) rather than a configuration change.

Limits

PHOTO_MAX_SIZE_MB (default 2 MB) caps an upload. Upload routes are exempt from the global 1 MB request-body limit so that cap can apply. Images must be JPEG or PNG — checked by magic bytes, not by the extension or the declared content type — and between 200×200 and 4000×4000 pixels, measured after orientation is corrected.

What to check when reviewing

The queue exists because a photograph is identity data that appears on rosters, ID cards and staff directories. Reject anything that is not a clear, current likeness of the person, and remember that a rejection is visible to the uploader — write the reason as though they will read it, because they will.