Files
caelum/CHANGELOG.md
1andClaude Opus 5.5 fce190c03a
ci / server (mysql) (pull_request) Successful in 16s
ci / server (sqlite) (pull_request) Successful in 16s
ci / lint (pull_request) Successful in 26s
ci / nox (pull_request) Successful in 28s
ci / client (pull_request) Successful in 47s
ci / sidecar (pull_request) Successful in 37s
ci / image (pull_request) Successful in 1m19s
ci / sourcemaps (pull_request) Skipped
ci / deploy (pull_request) Skipped
Nox real mode requires rights on each telescope and dome it drives
Finding fmw149sl04. Real mode checked only sign-in and confirmed: true, then resolved any telescope or dome by name, so a signed-in account could drive another site's hardware.

The driver now takes the calling user and checks every op: canControlTelescope for a telescope, canEditSite for a dome. The route refuses with 403 before the first op when a string literal names a resource the caller has no rights on. Names repeat across sites, so a name resolves among the caller's rows, and two matches is an error.

canControl moved from routes/control.js to perms.js as canControlTelescope. It now passes an admin with no booking live, as the control.js header said it did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 18:01:41 -05:00

24 KiB
Raw Permalink Blame History

Changelog

All notable changes to Caelum are documented here. Format follows Keep a Changelog. Versioning follows SemVer.

[Unreleased]

Added

  • Object deep-dive dossier (/object/:name). Pick a frame from the Archive (or a research card) and open one page that pulls together everything Caelum knows about that object: every visible frame across telescopes, the observations that produced them (frozen provenance where linked), catalog identity (lazy SIMBAD), a local SpaceDB orbital match for NEO/comet objects, an observation-cadence timeline + per-filter coverage, prior frame analysis, and an embedded Lab that analyzes the object + its frames into a schedulable Nox program. Backend GET /api/research/object/:name is 100% local (works with no outbound network); catalog/SIMBAD is a separate optional call. Reuses the existing analysis engine via a new target_inline path so objects with no feed row can still be analyzed. Entry points from the Archive frame modal, research cards, and ⌘K. New server tests cover dossier visibility filtering, exact-vs-fuzzy name matching (no silent merging of M31/M310), and null-observation_id aggregation.
  • Add research by hand, offline. A prominent "+ Add research" button on the Research page opens a modal (category, designation, optional RA/Dec/magnitude/priority/note) that adds a target to the feed with POST /api/research/targets - no upstream fetch, no scheduling. Every prior add path needed the internet; this one works on the deploy that doesn't have it.
  • In-app "buy me a coffee" support checkout. The landing's coffee button now opens an in-app amount picker (presets + custom, server-validated bounds) and mounts Stripe's Embedded Checkout inside a modal - the supporter never leaves Caelum, and card fields stay in Stripe's iframe. Falls back to a hosted payment link when STRIPE_SECRET_KEY is unset. New /api/support route + 7 tests (amount validation, allowlisted return_url, offline fallback).

Changed

  • Settings toggles actually do something. Theme and density were saved to the account but never applied to the page, so toggling them changed nothing visible. They now reflect onto <html data-theme data-density> (applied at boot from a localStorage cache and synced from the account on login): a real light theme (the app was dark-only), and a compact density that tightens the spacing + type scale. Every setting also shows a "Saved ✓" confirmation. A handful of views still hardcode colors and read slightly off in light until tokenized.
  • Sidebar is text-only. Dropped the per-item emoji icons from the nav.
  • Messages has a page header ("Messages" + subtitle) so it matches every other page instead of jumping straight into the two-pane layout.
  • Pages use the full width. Dropped the 1200px centered content cap (which left ~480px of empty gutter on a 1920 screen and made data-dense pages like Messages look stranded in the middle) for a high 1800px safety cap. Content now fills the available width with comfortable padding.
  • Quieter build stamp. The sidebar footer's build widget lost its hover panel (client/server/db health) and the admin "Send Sentry test event" button. It's now just a small version + commit line. The hover panel had been popping up over the Logout button and blocking clicks, and the ops readout read as corporate SaaS rather than a personal project.
  • Copy de-slopped. Stripped em-dashes from the user-facing copy (landing, dashboard, research, Lab, dossier, inbox), replaced the "next generation of observatories" hero line with a plainer one, and removed the decorative emoji from the landing feature cards. The single-glyph sidebar icons stay. The point: it reads like an instrument, not an AI-generated marketing page.
  • Portal navigation follows the night's lifecycle. The sidebar is re-grouped from data-category buckets (Observe/Plan/Observatory/Network/Data) into the arc the landing tells: Tonight → Your Observatory → Plan & Book → Archive → Community. The Dashboard ("Tonight") reads down that same arc and shows a first-run getting-started path for empty accounts. Duplicate surfaces consolidated: Archive's "Global Feed" tab now links to Activity, and Inbox's "Upcoming Sessions" card defers to the Schedule. DESIGN.md's IA section rewritten to match.
  • Landing page rewritten. Full-bleed banded layout (topbar, hero + auth, "Live on the network" explainer, features, Nox showcase, footer) that no longer strands the auth card in whitespace and no longer presents seeded sample data as a live network. Seed trimmed of invented community prose (fabricated users, generic comment/testimonial filler) while keeping the real gear, categories, and links.
  • Nox Console works without an account. /nox is now publicly viewable, so "Try Nox without signing up" → Examples → "Open in Console" no longer bounces guests to the landing (mock runs were already unauthenticated server-side; the router just disagreed).
  • Clearer offline errors for research tools. Upstream fetch failures (SIMBAD, JPL, NASA, AAVSO, and now the Lab's Anthropic call) surface "No outbound network from this server - this tool needs internet" instead of a raw "fetch failed"/"aborted", and the two mismatched timeouts (6s/15s) are unified to 12s.

Fixed

  • Nox real mode checks rights on the hardware it drives. A signed-in account could run a program against any site's telescope or dome by name. /api/nox/run and /run-stream now refuse with 403 before the first op, and the driver re-checks on every op. Telescopes follow the /api/control rule and domes the /api/hardware rule. /api/control now also lets an admin in when no booking is live, as its header comment already said.

  • Slews refuse to point near the Sun. Every path that moves a mount (/api/control, /api/hardware, /api/astra, Nox real mode on each driver, what the dispatcher hands a bridge, and the target on what /api/jobs hands a daemon) now refuses a target inside 46 degrees of the Sun unless the telescope declares solar_filter on PUT /api/hardware/telescopes/:id/driver. Nox real mode also sent RA in degrees to drivers that take hours, so it pointed somewhere else; it now converts.

  • Vet endpoint 500 on MySQL. UPDATE sites SET successful_observations = MAX(0, …) is a syntax error on MySQL (2-arg MAX is aggregate-only) - every PUT /api/observations/:id/vet failed in production while passing SQLite dev tests. Replaced with a portable CASE expression, verified against MySQL 8.4.

  • Simulator hardware control. A freshly-seeded (idle) simulator now slews without a manual connect (the Nox real-mode path had no connect step), and disconnecting a simulator actually returns it to idle instead of leaving it stuck "connected". New HTTP-level control.js integration tests for the connect → status → slew → position → disconnect flow.

  • Astra api_key no longer leaks. GET /api/astra/instances returned SELECT ai.* including the credential that drives slew/expose/open-dome; now column-listed to exclude it.

  • User emails no longer exposed. The Leaderboard drill-down (/api/network/stats/users) selected email/role for any authenticated user; scoped to id/username/created_at.

  • Live feed no longer duplicates a target. /api/network/public/recent collapsed into the same object shot across filters; now one event per subject with the freed slots backfilled.

  • Health/Sentry version. The health endpoint reported a hardcoded 0.4.0 (prod never set npm_package_version); it now reads the real version from package.json.

  • Schedule edit no longer shifts a booking a day. openEdit mixed a UTC calendar date with a local wall time; a no-op save rewrote the booking ±24h. Both now use the local calendar date.

  • Command-palette remote results were unclickable (to vs link); GROUP_CONCAT mangled request-group targets on MySQL; a null dec crashed the Plans target table; the Proposal edit form offered an invalid "Expired" status; UserProfile rendered a stray leading separator for a never-returned role.

  • Local pickup + CRM/Books order sync for the shop. Checkout offers a "Local pickup" option (configurable store location via SHOP_PICKUP_* env) that skips shipping cost + address; orders carry a fulfillment_method and the admin Orders view adapts ("Mark picked up" vs "Mark shipped"). On a paid order, Caelum now pushes the buyer to crm.hitthosting.com (contact, tagged caelum-shop) and a paid invoice to books.hitthosting.com (customer + invoice with line items/shipping/tax + a card payment marking it paid). Best-effort and idempotent: failures record integration_status='error' with an admin Re-sync button and never block the order/webhook. Auth is a per-service API key (CRM_API_KEY/BOOKS_API_KEY, each bound to one org) via a new x-api-key path added to both apps. New shop_orders columns: fulfillment_method, crm_contact_id, books_invoice_id, integration_status. 4 new server tests (pickup checkout, invoice-lines-sum-to-total, sync idempotency, webhook-never-fails-on-integration-error).

  • Shop in the Setup Wizard + standalone storefront (Stripe-backed). Operators can sell telescopes, mounts, cameras, filter wheels, focusers, control computers, and accessories; buyers browse a /shop storefront (and matched gear in Step 4 of the wizard), add to a cart, and check out with a Stripe Payment Element. The catalog is curated by platform admins and seeded from real Agena Astro listings (npm run seed:shop, 22 products with current prices + source links); the DB is the source of truth for price and inventory, mirrored to Stripe Products/Prices on save. Checkout total is always computed server-side (client prices are ignored). A Stripe webhook (/api/shop/webhook, raw-body mounted before the JSON parser, exactly-once via an event ledger) marks orders paid and decrements inventory atomically. Admins get a Shop Catalog page (/shop/admin) with product CRUD, re-sync, and an Orders fulfillment view showing each paid order's line items + shipping address with a "Mark shipped" action. Each PaymentIntent carries a description + metadata (items, totals, customer) so the Stripe dashboard is legible. New tables: shop_products, shop_orders, shop_order_items, shop_webhook_events. Simple flat shipping/tax via env knobs (SHOP_SHIPPING_FLAT_CENTS, SHOP_TAX_RATE_BPS). Sidebar gains a customer Shop link (everyone) and an admin-only Shop Catalog link. 10 server tests cover admin gating, server-authoritative totals, stock/currency guards, webhook idempotency, and no-negative-inventory. Fail-closed: every shop route 503s until STRIPE_SECRET_KEY is set.

  • Nox REPL (nox repl): interactive shell on the mock driver. Bindings (use, let, var, fn) persist across inputs and each statement's ops stream live. Unbalanced braces/parens enter continuation mode (... prompt). Type errors don't commit to the buffer. Builtin commands: :help, :src, :ops, :history, :clear, :exit. 8 piped-stdin tests cover the banner, op streaming, scope persistence, fix-it hint surfacing, :clear reset, multi-line continuation, :help listing, and history-of-accepted-only.

  • /hardware fleet dashboard route. Lists every telescope and dome across every site the user can see, in a responsive card grid. Per-row: driver pill (Simulator / Stellarium · host:port / Alpaca · host:port #N), live status indicator with colored dot (ok/tracking/slewing/unreachable/simulator/probing), current RA/Dec, "last poll Xs ago", a "Test connection" button per real-driver row, quick-link to /telescope/:id or /site/:id. Polls /api/hardware/{telescopes|enclosures}/:id/{state|dome/state} every 4s while visible.

  • Nox CLI: nox run is verbose by default (prints each op then the summary). Added --quiet/-q to suppress per-op output. nox ops retained as a back-compat alias.

  • Nox type checker: diagnostics now carry optional suggest and seeAlso fields. Unknown identifiers/units/frames/resources, method-not-found, call-arg mismatches, and Bool-context errors include "did you mean ...?" hints and clickable doc anchors (CLI prints, IDE renders, gutter hover folds into the tooltip).

  • Nox IDE: "Report a problem" link in the page subtitle that opens a prefilled GitLab issue with the current source, diagnostics, mode, and browser context.

  • nox/vscode/: VSCode extension scaffold (TextMate grammar + snippets + language config) for .nox files. Build a vsix locally with cd nox/vscode && npx @vscode/vsce package.

  • Docs full-text search: new GET /api/nox/docs-search?q=... endpoint backed by Minisearch. Indexes every .md under nox/docs/ split per heading (214 passages today). The /nox/docs page gains a search box with keyboard navigation (/ to focus, arrows + Enter to jump, Esc to close). Clicking a result opens the doc and pulses the matched heading.

  • Self-host Docker stack: multi-stage Dockerfile, docker-compose.yml, docker/initdb/ provisioning script, and .env.example. docker compose up -d boots MySQL 8.4 + the app with a healthcheck and persistent volume.

  • Server test suite (server/test/nox-routes.test.js, 10 cases). Covers /api/nox/check (clean + with hints + missing-source 400), /api/nox/run (mock op log), /api/nox/docs-search (empty, ranked, limit-clamp), /api/nox/examples, /api/nox/docs (serve + path-escape guard). Uses node --test so no extra deps. Hooked into GitLab CI as the server-test job.

  • docker-validate CI job: validates docker-compose.yml config and builds the Dockerfile on every MR + push to catch infra drift.

  • Enriched /api/health endpoint. Now races a SELECT 1 against a 1.5s timeout for both the app DB and (if configured) SpaceDB, returns 503 when the app DB is unreachable, and includes version, startedAt, and per-pool status. Backed by a new services/db-health.js probe (5 unit tests).

  • Graceful shutdown for the API server. Handles SIGTERM/SIGINT, closes the Socket.IO and HTTP listeners, ends both mysql2 pools, and exits within ~1s under docker stop. Forced-exit safety net at 8s in case cleanup hangs.

  • DB-outage resilience (TODO K). The server no longer hard-exits when MySQL is unreachable at boot or drops mid-flight. initDb() runs in the background with capped exponential backoff (1s → 30s); HTTP comes up immediately so /api/health can report degraded. Background services (weather-sim, lifecycle, weather-stations) now run under a shared safeInterval helper that catches errors, suppresses repeat log lines within a 60s window, prevents tick overlap, and logs a recovered line when the underlying error clears. 5 unit tests cover the helper. Verified live: stopping the db container during an active session left the app process untouched (startedAt unchanged), /api/health flipped to 503 in 16ms, /api/nox/check kept serving, and recovery happened within 4s of bringing the DB back.

  • Real-hardware confirmation dialog in the Nox Console (TODO A). When Mode is "Real hardware" and the user clicks Run, the IDE first does a mock-mode dry-run, then surfaces a dialog with op count, resource list, and the first op's signature. "Don't ask again this session" suppresses for subsequent runs. Esc and Cancel back out. The Run button stays disabled if the dry-run itself errored. Pure client-side UX safety gate; works against any real driver.

  • ASCOM Alpaca driver translation for Nox (TODO H). Real-mode Nox programs now reach real hardware when a telescope row has driver_type='alpaca'. New services/alpaca-telescope.js covers slew (slewtocoordinatesasync + connected + tracking), park, home (findhome), and wait_until_settled (polls Slewing then reads final RA/Dec). New services/alpaca-camera.js covers cooldown (cooler + setccdtemperature with poll) and expose (startexposure + imageready). services/nox-driver.js dispatches per-telescope based on driver_type: simulator → DB-only mock, stellarium → existing TCP packets, alpaca → REST. 5 dispatcher tests with intercepted fetch lock in the wiring. The Alpaca camera/telescope modules share the existing alpacaGet/alpacaPut helpers in external-drivers.js. Scope honesty: INDI (Linux native binary protocol) and ASCOM-over-COM (Windows-only) are not implemented; Alpaca reaches Windows ASCOM drivers via the Alpaca Remote Server bridge. Verified live by running a Nox program against a local fake Alpaca server - the exact ASCOM call sequence (connect, tracking, slewtocoordinatesasync, slewing poll, position read, park) was recorded.

  • Hardware discovery wizard in the Nox Console (TODO B). When Mode = "Real hardware" the IDE shows a "Configure…" button that opens a wizard: enter host + port (defaults 127.0.0.1:11111), hit Discover, see the Alpaca server's name + manufacturer + version, get a table of every configured device (DeviceType + DeviceName + DeviceNumber), and pick which Caelum telescope or enclosure row each one attaches to. Save runs PUT /api/hardware/{telescopes|enclosures}/:id/driver per attachment. Server-side: new POST /api/hardware/discover/host calls /management/v1/configureddevices on a known host (UDP broadcast is unreachable across docker / network boundaries, hence the manual-host variant). Devices that don't yet have a wizard attach surface (cameras, focusers, filter wheels) are listed as "not yet supported in the wizard" instead of silently dropped.

  • SQLite as a first-class backend (TODO J). Set DB_KIND=sqlite (with optional SQLITE_PATH=...) and the server boots without MySQL. Highlights:

    • New server/src/config/database-sqlite.js exposes the same db.get/all/run/exec/tx surface as the MySQL backend.
    • server/src/config/database.js is now a thin dispatcher; default is still mysql so existing deploys are unchanged.
    • Single source of truth for the schema in server/src/config/schema-mysql.js; the SQLite path runs a regex-based translateDdl() over it at boot. Backtick-quoted identifiers are stashed and restored so column names that share a type keyword (e.g. `timestamp`) survive the type rewrite. 6 unit tests cover the translator + adapter.
    • Route audit complete: NOW() → CURRENT_TIMESTAMP across 14 files; DATE_SUB/DATE_ADD/INTERVAL rewritten to JS-computed cutoff timestamps in network.js (/feed, /stats/:metric, /leaderboard, /sites/:id/calendar) and services/{weather-sim,external-drivers}.js. Every route now uses portable SQL.
    • CI matrix: server-test runs under both DB_KIND=mysql and DB_KIND=sqlite on every push.
    • Verified live by booting the server cold with DB_KIND=sqlite: /api/auth/register, /api/auth/login, /api/sites (POST + GET, with the subquery-laden weather_readings.timestamp join intact), /api/nox/check, and /api/network/stats/users all work end-to-end.

Fixed

  • Schema completeness for fresh SQLite installs. Folded COLUMN_MIGRATIONS (driver_type, driver_host, driver_port, driver_device_number on telescopes/instruments/enclosures, plus visibility/about on sites, visibility/proposal_id/request_group_id on bookings, seeing_arcsec on weather_readings, proposal_id on observing_plans) into the canonical schema-mysql.js DDL. Existing MySQL deploys still get the migrations via the idempotent addColumnIfMissing path; new deploys (MySQL or SQLite) get the up-to-date shape from the first CREATE.
  • Missed dialect-specific SQL in middleware: session-lock.js (NOW() × 4) and auth.js (NOW() × 1). Caught when /api/sites/:id failed under SQLite during the H/B live smoke. Both now use CURRENT_TIMESTAMP.
  • Dockerfile resolves the file: workspace chain by installing nox/parser, nox/runtime, server, and client in dependency order. Without this, the runtime container couldn't import @caelum/nox-parser through @caelum/nox-runtime.
  • Dockerfile pre-creates server/data/uploads and chowns /app to the non-root caelum user. Previously the archive route crashed on first import with EACCES.
  • Dockerfile no longer depends on apk add tini. Removes a brittle network dependency at build time; Node 22 plus the new SIGTERM handlers handle PID 1 cleanly.
  • docker-compose.yml drops the --default-authentication-plugin=caching_sha2_password flag that MySQL 8.4 removed.

[0.5.1] - 2026-05-29

Security and bug-fix patch. No schema or API-shape changes; safe to deploy over 0.5.0.

Security

  • Enforce API token scopes. cae_ tokens were validated at creation but never checked at request time, so a read-scoped token had full write/delete/hardware-control power. authenticate now gates by HTTP method: read scopes permit GET/HEAD only; mutations require write or daemon. JWT sessions are unaffected.
  • Fix archive frame IDOR. GET /api/archive/:id and the comment/like/bookmark/sdss-crossmatch/analyze endpoints performed no visibility or ownership check, exposing and mutating other users' private frames. Added shared canViewFrame/canEditFrame helpers; reads require visibility, writes require uploader/site-member/admin.
  • Vetting requires owner/operator. PUT /api/observations/:id/vet moves a site's public trust counter but only required the weaker observer-level check, letting an invited observer inflate a site's score. Now gated by canEditSite.
  • Rate-limit auth endpoints. Added an in-process limiter (10 logins/15 min, 5 registrations/hr per IP) and switched bcrypt hash/compare to async so credential brute-force can't also block the event loop.
  • Sentry no longer leaks credentials. sendDefaultPii disabled; a beforeSend hook strips Authorization/Cookie headers and redacts cae_…/Bearer … strings from payloads.
  • Gate Astra hardware-read endpoints. Seven GET /api/astra/instances/:id/* routes (status, position, autoguide/autofocus status, plate-solve result, camera status, weather) lacked the site-membership gate the mutating routes had, leaking another operator's live telescope state. weather-stations/:id/poll now requires canEditSite.
  • Stop leaking email/role from GET /api/network/users/:username/profile.
  • SSRF guard on driver fetches. Block the link-local/cloud-metadata range (169.254.0.0/16, fe80::, metadata.google.internal) and non-HTTP(S) schemes in weather/Alpaca driver requests. LAN and loopback hosts remain allowed - real ASCOM Alpaca drivers run there.
  • Lock down CORS and add security headers. Replaced wildcard CORS / Socket.IO origin with an env-configurable allowlist (CORS_ORIGINS); added X-Content-Type-Options, X-Frame-Options: DENY, Referrer-Policy, and prod-only HSTS; set trust proxy for accurate client IPs.

Fixed

  • Dashboard "Recent Archive" widget rendered FWHM at full float precision (e.g. 1.664764056048128"); now rounds to two decimals like the archive table.
  • Interval hygiene: safeInterval gained guarded warm-up (immediateDelayMs) and unref options. The lifecycle and weather-station warm-up ticks now share the overlap guard, and the per-second telescope-position broadcast is guarded, unref'd, and cleared on shutdown.

[0.4.0] - 2026-05-14

Added

  • Public open-source release. MIT license. Contributing, code of conduct, and security policy.
  • Nox language v0.4: typed DSL for telescope and dome control. Parser, type checker, interpreter, CLI, HTTP API, in-browser IDE, public docs and examples pages.
  • Pluggable Nox catalog files via nox/runtime/catalogs/<name>.json.
  • Observatory setup wizard at /setup: four-step guided onboarding with capability tiers, six-category gear picker, generated equipment plan, driver integration steps, and starter Nox template. State is URL-encoded for shareable plans.

Changed

  • Deploy bundle (npm run package) now ships the Nox workspace alongside the server so file: deps resolve on the host.
  • All package versions synced to 0.4.0 (was a mix of 0.1.0 and "v0.4" references). Pre-1.0 SemVer commitment: minor bumps may break, patches are safe, breaking changes are listed here.

Fixed

  • Nox docs route layout: dashboard sidebar no longer fights the docs sidebar; back arrow uses browser history.
  • client/src/services/api.js is hardened against non-JSON error responses. Previously a 500 with an empty body crashed with Chrome's "Unexpected end of JSON input" or Safari's "The string did not match the expected pattern." Now surfaces HTTP status + body excerpt.

[0.1.0] - 2026-05-13

Initial private release. Web client, API server, MySQL migration from SQLite, telescope and dome integration. Not publicly released.