The sidecar is now a running service rather than a tested library. Verified against a live API: the real container claims a queued timing job, computes BJD_TDB, reports it, and the run settles `complete` with the software versions recorded for provenance. New in the sidecar - client.py: the protocol, with a retry policy that is deliberate rather than blanket. 5xx, 429 and connection errors back off and retry; a 4xx raises immediately, because a 4xx is our bug, our credential or a stale claim, and retrying any of those turns a clear failure into a silent loop. 409 gets its own ClaimLost type, since a lost claim means stop working rather than something is broken. - worker.py: the poll loop, plus a second slower loop for master building, which is per-group rather than per-frame and so does not fit the queue. A heartbeat thread runs for the duration of every stage, and a result computed after the claim was lost is DISCARDED rather than written over another worker's. - fits_io.py: reads the first HDU that actually has pixels, because a professional instrument often leaves the primary empty and guessing gives a zero-size array rather than an error. write_master stamps the combine method, clip bounds, input count and source frame ids into the file, so a master is self-describing once copied away from the database. - __main__.py: env-only config, SIGTERM sets a stop flag rather than killing mid-write. Server side - GET /api/reduction/worker/masters/current. I had written the sidecar calling this before it existed, which is exactly the dangling-reference problem I flagged in the docs earlier. It returns the newest matching non-superseded master, or 204, so a worker can bias-subtract a dark's inputs; only a bias-subtracted dark can be scaled. One real bug the end-to-end run caught - The work order's frame object omitted ra/dec_coord. The barycentric correction is direction-dependent, so the timing stage failed with "the frame has no sky position" on a frame that had one. The position now travels with every work order, not just the astrometry hints, with a regression test. Two things the container build caught that the venv did not - photutils is NOT pure Python, contrary to what I wrote in the sidecar README last commit. It has C extensions and ships no linux/aarch64 wheel, so pip compiled it from source and the single-stage build failed without build-essential. The Dockerfile is now multi-stage: wheels are built in one stage, installed with --no-index in a slim runtime with no compiler. Corrected in the README. - My earlier pip install loop reported OK for sep-pjw when it had actually failed, because $? captured the exit of a piped tail rather than pip. Also - Compose entries in both dev and prod, behind a `reduction` profile so nothing starts without a token. The prod entry is NOT in the deploy path: astrometry, extract and photometry are unimplemented, so it would claim those and fail every one. - No CI image-build job for the sidecar, deliberately. The shared runner already builds one image per pipeline for nine repos, and a scientific-Python build on every push is not worth it for a service that cannot yet do the stages that matter. - CAELUM_WORKER_TOKEN documented in both env examples, with why it must be reduction-scoped and not daemon. - The worker now logs completions with their duration. It logged claims and nothing else, which left an operator unable to tell a slow stage from a wedged one. Verified: 238 server + 22 client + 115 sidecar tests pass, lint clean, SPA builds, hadolint clean on the new Dockerfile, all three YAML files parse, and the built image runs as uid 10001 with no compiler and no leftover wheels. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
44 lines
1.9 KiB
Bash
44 lines
1.9 KiB
Bash
# Caelum docker-compose configuration.
|
|
#
|
|
# Copy to `.env` and edit the secrets before running `docker compose up`.
|
|
# These values are consumed by docker-compose.yml only. The app itself reads
|
|
# its config from environment variables that compose injects from here.
|
|
|
|
# --- Database ---
|
|
# Root password is only used during first-boot provisioning of the mysql:8
|
|
# container. The app never uses it.
|
|
MYSQL_ROOT_PASSWORD=change-me-root
|
|
|
|
# App database. The mysql:8 image creates this DB and grants the app user on
|
|
# first boot. The init script in docker/initdb/ provisions SpaceDB after.
|
|
MYSQL_DATABASE=caelum_app
|
|
MYSQL_USER=caelum
|
|
MYSQL_PASSWORD=change-me
|
|
|
|
# Optional NASA/JPL catalog database. Set to empty to disable /api/spacedb
|
|
# routes. If set, the init script creates it on first boot.
|
|
MYSQL_SPACEDB_DATABASE=caelum_spacedb
|
|
|
|
# --- Auth ---
|
|
# Used to sign JWTs. MUST be 32+ chars. The server refuses to start with the
|
|
# placeholder below — generate a real one with: openssl rand -hex 32
|
|
JWT_SECRET=REPLACE_ME_WITH_openssl_rand_hex_32_OUTPUT
|
|
|
|
# --- Networking ---
|
|
# Host port that the app listens on. Useful when running multiple Caelum
|
|
# instances on one machine.
|
|
CAELUM_PORT=3002
|
|
|
|
# --- Reduction sidecar (optional; docker compose --profile reduction) ---
|
|
# A cae_ API token whose scopes are exactly "reduction". Create it from the UI
|
|
# or POST /api/tokens, and give it nothing wider: the reduction scope is not
|
|
# account-wide write, so this token can claim and report reduction jobs and
|
|
# nothing else. A "daemon" or "write" token would also work and would hand an
|
|
# unattended container the owner's whole account.
|
|
CAELUM_WORKER_TOKEN=
|
|
# Identifies this worker in logs and on job claims. Defaults to the hostname.
|
|
CAELUM_WORKER_ID=
|
|
# Only the stages the sidecar build implements. Leaving this unset makes the
|
|
# worker claim astrometry and extract too and fail each as unimplemented.
|
|
CAELUM_WORKER_STAGES=timing,calibrate
|