Files
Paul HittandClaude Opus 5 aa62529ea7 feat(sidecar): the Python reduction sidecar, timing and calibration done
Phase 2 of item L. Two stages implemented with 53 tests, plus the protocol
document so someone can write a worker without reading the server.

sidecar/caelum_reduce/timing.py  (item O)
- UTC at the observatory to BJD_TDB at mid-exposure, via astropy.
- Refuses rather than guesses. A missing exposure time, a missing target
  position, a DATE-OBS carrying a UTC offset: all rejected, because a wrong
  epoch is undetectable downstream.
- The tests assert physics, not just that a float comes back: the correction
  is bounded by the light-crossing time of Earth's orbit, reverses sign
  between the solstices for a target in the ecliptic plane, nearly vanishes
  at quadrature, almost disappears toward the ecliptic poles, and two sites
  on opposite sides of Earth agree to within the light travel time across
  Earth's diameter.
- That thoroughness is the point. A hand-rolled approximation is wrong by
  ~0.1s and the error is PERIODIC with a one-year cycle, because it tracks
  Earth's orbit. A periodic error in a data set about periodic phenomena
  does not read as a bug, it reads as signal.

sidecar/caelum_reduce/calibrate.py  (items L and N)
- Master bias/dark/flat by sigma-clipped median, and the full reduction
  equation applied with uncertainties propagated: var = counts_e + rn^2
  through the gain, variances adding under subtraction, relative errors
  adding under division by the flat.
- Two refusals that matter more than the arithmetic. Scaling a dark not
  marked bias_subtracted is rejected, because dark current scales with
  exposure time and the bias does not, so scaling a bias-inclusive dark
  multiplies the bias too; this is the most common calibration mistake and
  the result looks entirely reasonable. And a master whose shape does not
  match the frame is rejected rather than allowed to broadcast.
- A dead column gets kept rather than clipped to a NaN hole, since a hole
  would propagate into every frame the master touches.

Three things learned while testing, all recorded in comments
- Single-pass sigma clipping does NOT catch an evenly split stack: 8 frames
  at one level plus 8 at another clips nothing, because the split inflates
  the stack's own standard deviation until both groups look like one-sigma
  deviations. Pinned as a test so the limitation stays visible; preventing
  it is the Node-side grouping rules' job.
- The barycentric correction extremes fall at the SOLSTICES for a target at
  RA 6h, not the equinoxes, where Earth is perpendicular to the line of
  sight. My first version of that test asserted the wrong dates.
- A flat normalizes by its median, not its peak, so the centre comes out
  above 1. Assert the centre-to-corner ratio, which is the physical
  quantity, not either absolute value.

SEP dropped in favour of photutils. Same capability, pure Python over numpy
so wheels exist everywhere, already an astropy-affiliated dependency. SEP is
a C extension with no wheel for Python 3.14.

docs/reduction-worker-protocol.md
- All 10 endpoints, the stage dependency table, the per-stage payloads, and
  the three things a worker author must get right: subtract bias before
  scaling a dark, never report a partial WCS, never report a magnitude
  without an uncertainty.

Not done in phase 2 yet: the worker poll loop (so this is a tested library,
not a running service), a Dockerfile, and a compose entry. astrometry,
extract and photometry stages remain unimplemented; astrometry needs a
solver binary in the image.

Verified: 233 server + 22 client + 53 sidecar tests pass, lint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 13:26:45 -04:00

70 lines
1.0 KiB
Plaintext

# Dependencies
node_modules/
# Build outputs
dist/
scripts/.release-notes-draft.md
# Local environment + secrets
.env
.env.local
.env.*.local
/env
# Production deploy secrets (templates committed under deploy/*.env.example)
deploy/caelum.env
deploy/caelum-spacedb.env
caelum.env
caelum-spacedb.env
server/.env
server/.env.local
client/.env.local
# Local database dumps. Too large for the public repo (spacedb.sql is 1.2 GB)
# and app.sql contains dev user records. Regenerate via the migrate script.
/sql/
# Local data
server/data/
*.sqlite
*.sqlite3
*.sqlite-journal
*.sqlite3-journal
*.sqlite-wal
*.sqlite3-wal
*.sqlite-shm
*.sqlite3-shm
# Vite / build cache
.vite/
*.tsbuildinfo
# Test coverage
coverage/
# Logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# Editor / OS junk
.vscode/
.idea/
.claude/
CLAUDE.md
RULES.md
*.swp
*~
.DS_Store
Thumbs.db
.gstack/
# Local secrets (GitLab PAT, VPS credential) — never commit
.pat
.vps
# Sidecar: local venv and pytest cache
sidecar/.venv/
sidecar/.pytest_cache/
sidecar/**/__pycache__/