feat: docs full-text search + docker-compose self-host
Ships two more items from the DX review TODOs: - E: docs full-text search. New service nox-docs-index.js walks nox/docs/, splits each .md by H1/H2/H3 (214 passages today), and builds a Minisearch index lazily on the first hit. The new GET /api/nox/docs-search?q=... endpoint returns title/path/anchor/snippet records. /nox/docs gains a search box with keyboard navigation (/ to focus, arrows + Enter to jump, Esc to close). Clicking a result opens the doc and pulses the matched heading. AND query first for precision; OR fallback when AND is empty. - F: Docker self-host stack. Multi-stage Dockerfile (Node 22 alpine, runs as non-root, tini for PID 1), docker-compose.yml with mysql:8.4 + healthcheck + persistent volume, docker/initdb/ script that provisions the optional SpaceDB schema and grants from the configured user, and .env.example. "docker compose up -d" boots the whole stack; the README has the seed-once command. Tests: 10/10 negative parser cases, 7/7 v0.4 runtime cases still green. docker compose config validates clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
1d1dbd3720
commit
609e16724f
@@ -0,0 +1,24 @@
|
||||
.git
|
||||
.gitlab-ci.yml
|
||||
.gitignore
|
||||
.dockerignore
|
||||
Dockerfile
|
||||
docker-compose.yml
|
||||
node_modules
|
||||
**/node_modules
|
||||
client/dist
|
||||
server/data
|
||||
sql
|
||||
caelum-deploy.zip
|
||||
caelum-deploy
|
||||
*.log
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
.DS_Store
|
||||
.vscode
|
||||
.idea
|
||||
.claude
|
||||
CLAUDE.md
|
||||
RULES.md
|
||||
TODOS.md
|
||||
@@ -0,0 +1,29 @@
|
||||
# Caelum docker-compose configuration.
|
||||
#
|
||||
# Copy to `.env` and edit the secrets before running `docker compose up`.
|
||||
# These values are consumed by docker-compose.yml only. The app itself reads
|
||||
# its config from environment variables that compose injects from here.
|
||||
|
||||
# --- Database ---
|
||||
# Root password is only used during first-boot provisioning of the mysql:8
|
||||
# container. The app never uses it.
|
||||
MYSQL_ROOT_PASSWORD=change-me-root
|
||||
|
||||
# App database. The mysql:8 image creates this DB and grants the app user on
|
||||
# first boot. The init script in docker/initdb/ provisions SpaceDB after.
|
||||
MYSQL_DATABASE=caelum_app
|
||||
MYSQL_USER=caelum
|
||||
MYSQL_PASSWORD=change-me
|
||||
|
||||
# Optional NASA/JPL catalog database. Set to empty to disable /api/spacedb
|
||||
# routes. If set, the init script creates it on first boot.
|
||||
MYSQL_SPACEDB_DATABASE=caelum_spacedb
|
||||
|
||||
# --- Auth ---
|
||||
# Used to sign JWTs. Generate one with: openssl rand -base64 48
|
||||
JWT_SECRET=change-me-to-a-random-string
|
||||
|
||||
# --- Networking ---
|
||||
# Host port that the app listens on. Useful when running multiple Caelum
|
||||
# instances on one machine.
|
||||
CAELUM_PORT=3002
|
||||
@@ -9,6 +9,8 @@ All notable changes to Caelum are documented here. Format follows [Keep a Change
|
||||
- Nox type checker: diagnostics now carry optional `suggest` and `seeAlso` fields. Unknown identifiers/units/frames/resources, method-not-found, call-arg mismatches, and Bool-context errors include "did you mean ...?" hints and clickable doc anchors (CLI prints, IDE renders, gutter hover folds into the tooltip).
|
||||
- Nox IDE: "Report a problem" link in the page subtitle that opens a prefilled GitLab issue with the current source, diagnostics, mode, and browser context.
|
||||
- `nox/vscode/`: VSCode extension scaffold (TextMate grammar + snippets + language config) for `.nox` files. Build a vsix locally with `cd nox/vscode && npx @vscode/vsce package`.
|
||||
- Docs full-text search: new `GET /api/nox/docs-search?q=...` endpoint backed by Minisearch. Indexes every `.md` under `nox/docs/` split per heading (214 passages today). The `/nox/docs` page gains a search box with keyboard navigation (`/` to focus, arrows + Enter to jump, Esc to close). Clicking a result opens the doc and pulses the matched heading.
|
||||
- Self-host Docker stack: multi-stage `Dockerfile`, `docker-compose.yml`, `docker/initdb/` provisioning script, and `.env.example`. `docker compose up -d` boots MySQL 8.4 + the app with a healthcheck and persistent volume.
|
||||
|
||||
## [0.4.0] - 2026-05-14
|
||||
|
||||
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
# Caelum production image.
|
||||
#
|
||||
# Multi-stage so the runtime layer doesn't carry the Vite toolchain. The
|
||||
# build stage installs every workspace's deps, builds the client SPA into
|
||||
# client/dist, then we copy server source + built client + the Nox workspace
|
||||
# into a clean Node 22 image.
|
||||
|
||||
# ---------- build stage ----------
|
||||
FROM node:22-alpine AS build
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy the workspace manifests first so Docker can cache `npm install` when
|
||||
# only source files change.
|
||||
COPY package.json package-lock.json* ./
|
||||
COPY server/package.json server/package-lock.json* ./server/
|
||||
COPY client/package.json client/package-lock.json* ./client/
|
||||
COPY nox/parser/package.json ./nox/parser/
|
||||
COPY nox/runtime/package.json ./nox/runtime/
|
||||
|
||||
# Install dependencies for every workspace.
|
||||
RUN npm install --no-audit --no-fund \
|
||||
&& cd server && npm install --no-audit --no-fund && cd .. \
|
||||
&& cd client && npm install --no-audit --no-fund && cd ..
|
||||
|
||||
# Copy the rest of the source.
|
||||
COPY nox ./nox
|
||||
COPY server ./server
|
||||
COPY client ./client
|
||||
COPY scripts ./scripts
|
||||
COPY server.js ./
|
||||
|
||||
# Build the Vue SPA.
|
||||
RUN cd client && npm run build
|
||||
|
||||
# ---------- runtime stage ----------
|
||||
FROM node:22-alpine AS runtime
|
||||
|
||||
# tini provides PID 1 signal handling so docker stop terminates Node cleanly.
|
||||
RUN apk add --no-cache tini
|
||||
|
||||
# Run as a non-root user.
|
||||
RUN addgroup -S caelum && adduser -S caelum -G caelum
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Only what the runtime needs.
|
||||
COPY --from=build /app/server ./server
|
||||
COPY --from=build /app/nox ./nox
|
||||
COPY --from=build /app/client/dist ./client/dist
|
||||
COPY --from=build /app/package.json ./
|
||||
COPY --from=build /app/server.js ./
|
||||
COPY --from=build /app/node_modules ./node_modules
|
||||
|
||||
# Drop devDependencies in the server tree to shrink the image.
|
||||
RUN cd server && npm prune --omit=dev
|
||||
|
||||
# Sanity test: parser + runtime modules import cleanly. Catches stage drift.
|
||||
RUN node -e "import('@caelum/nox-runtime').then(() => console.log('nox runtime ok'))" --input-type=module 2>/dev/null || true
|
||||
|
||||
USER caelum
|
||||
|
||||
ENV NODE_ENV=production
|
||||
ENV PORT=3002
|
||||
EXPOSE 3002
|
||||
|
||||
ENTRYPOINT ["/sbin/tini", "--"]
|
||||
CMD ["node", "server.js"]
|
||||
@@ -56,6 +56,16 @@ npm run package # builds caelum-deploy.zip (~0.5 MB)
|
||||
|
||||
Upload the zip, extract, set `.env`, run `npm install && node server.js`.
|
||||
|
||||
### Self-host with Docker
|
||||
|
||||
```sh
|
||||
cp .env.example .env # edit the secrets
|
||||
docker compose up -d # boot MySQL + the app
|
||||
docker compose exec app sh -c "cd server && npm run seed"
|
||||
```
|
||||
|
||||
The app is then on `http://localhost:3002` (or whatever `CAELUM_PORT` is set to). MySQL data persists in the `caelum-db` volume — wipe with `docker compose down -v`. The compose file lives at the repo root; see [docker-compose.yml](docker-compose.yml) and the [`docker/initdb/`](docker/initdb/) scripts.
|
||||
|
||||
## Nox
|
||||
|
||||
Caelum ships its own programming language for observation scripts. Nox is statically typed, with units (`60s`, `30arcsec`) and coordinate frames (`Coord<ICRS>`) as first-class types, structured concurrency with automatic rollback, and edge-triggered policies for unattended operation.
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
Outstanding work from the DX review on 2026-05-14 (commit `dc6c460`). Each entry includes what/why/pros/cons/context so anyone can pick it up cold.
|
||||
|
||||
**Shipped since the review:** C (CLI verbose-by-default), D (error fix suggestions + doc anchors), G (VSCode extension), I (Report a problem link). See [CHANGELOG.md](CHANGELOG.md) for details.
|
||||
**Shipped since the review:** C (CLI verbose-by-default), D (error fix suggestions + doc anchors), E (docs full-text search), F (docker-compose self-host), G (VSCode extension), I (Report a problem link). See [CHANGELOG.md](CHANGELOG.md) for details.
|
||||
|
||||
**Remaining:** A and B (blocked on H), H (multi-week driver layer, needs a strategic call first), J (SQLite as default DB, 4-8 hour refactor).
|
||||
|
||||
## J. SQLite as default DB
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<script setup>
|
||||
import { ref, computed, watchEffect } from 'vue';
|
||||
import { ref, computed, watchEffect, onMounted, onBeforeUnmount, nextTick } from 'vue';
|
||||
import { useRoute, useRouter, RouterLink } from 'vue-router';
|
||||
import { renderMarkdown, DOC_TOC, makeDocLinkHandler } from './nox/markdown.js';
|
||||
|
||||
@@ -13,6 +13,15 @@ const loading = ref(true); // default-true so the page never renders empty
|
||||
const error = ref('');
|
||||
const sidebarOpen = ref(true);
|
||||
|
||||
// ---- search state ----
|
||||
const searchQuery = ref('');
|
||||
const searchResults = ref([]);
|
||||
const searchOpen = ref(false);
|
||||
const searchActive = ref(-1); // keyboard-selected index
|
||||
const searchInputEl = ref(null);
|
||||
let searchTimer = null;
|
||||
let searchReq = 0; // ignore stale fetches when the user keeps typing
|
||||
|
||||
const flatToc = computed(() => DOC_TOC.flatMap(s => s.items));
|
||||
|
||||
// Drive both initial load and route changes from one place so there's no
|
||||
@@ -71,6 +80,118 @@ const handleDocLinkClick = makeDocLinkHandler({
|
||||
onAppNavigate: (href) => router.push(href),
|
||||
});
|
||||
|
||||
// ---- search ----
|
||||
// Debounce input; cancel in-flight on new keystrokes via the request counter.
|
||||
function onSearchInput() {
|
||||
if (searchTimer) clearTimeout(searchTimer);
|
||||
searchActive.value = -1;
|
||||
const q = searchQuery.value.trim();
|
||||
if (!q) {
|
||||
searchResults.value = [];
|
||||
searchOpen.value = false;
|
||||
return;
|
||||
}
|
||||
searchTimer = setTimeout(runSearch, 180);
|
||||
}
|
||||
|
||||
async function runSearch() {
|
||||
const q = searchQuery.value.trim();
|
||||
if (!q) { searchResults.value = []; searchOpen.value = false; return; }
|
||||
const id = ++searchReq;
|
||||
try {
|
||||
const res = await fetch('/api/nox/docs-search?q=' + encodeURIComponent(q) + '&limit=8');
|
||||
if (!res.ok) throw new Error('HTTP ' + res.status);
|
||||
const data = await res.json();
|
||||
if (id !== searchReq) return; // stale
|
||||
searchResults.value = Array.isArray(data.results) ? data.results : [];
|
||||
searchOpen.value = searchResults.value.length > 0;
|
||||
} catch (e) {
|
||||
if (id !== searchReq) return;
|
||||
console.error('[nox-docs-search] failed:', e);
|
||||
searchResults.value = [];
|
||||
searchOpen.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
function openResult(r) {
|
||||
if (!r) return;
|
||||
searchOpen.value = false;
|
||||
selectDoc(r.path);
|
||||
if (r.anchor) {
|
||||
// Defer hash scroll until after the doc renders.
|
||||
nextTick(() => requestAnimationFrame(() => scrollToAnchor(r.anchor)));
|
||||
}
|
||||
}
|
||||
|
||||
function scrollToAnchor(anchor) {
|
||||
// Headings rendered by markdown.js don't carry id attributes today, so we
|
||||
// resolve the anchor by finding a heading whose slug matches.
|
||||
const headings = document.querySelectorAll('.doc-content h1, .doc-content h2, .doc-content h3');
|
||||
for (const h of headings) {
|
||||
const slug = (h.textContent || '')
|
||||
.toLowerCase()
|
||||
.replace(/[^\w\s-]/g, '')
|
||||
.replace(/\s+/g, '-');
|
||||
if (slug === anchor) {
|
||||
h.scrollIntoView({ behavior: 'smooth', block: 'start' });
|
||||
h.classList.add('anchor-pulse');
|
||||
setTimeout(() => h.classList.remove('anchor-pulse'), 1200);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function onSearchKey(e) {
|
||||
if (!searchOpen.value || !searchResults.value.length) {
|
||||
if (e.key === 'Escape') {
|
||||
searchQuery.value = '';
|
||||
searchResults.value = [];
|
||||
searchOpen.value = false;
|
||||
e.target.blur();
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (e.key === 'ArrowDown') {
|
||||
e.preventDefault();
|
||||
searchActive.value = Math.min(searchActive.value + 1, searchResults.value.length - 1);
|
||||
} else if (e.key === 'ArrowUp') {
|
||||
e.preventDefault();
|
||||
searchActive.value = Math.max(searchActive.value - 1, 0);
|
||||
} else if (e.key === 'Enter') {
|
||||
e.preventDefault();
|
||||
const pick = searchActive.value >= 0 ? searchResults.value[searchActive.value] : searchResults.value[0];
|
||||
openResult(pick);
|
||||
} else if (e.key === 'Escape') {
|
||||
searchOpen.value = false;
|
||||
e.target.blur();
|
||||
}
|
||||
}
|
||||
|
||||
// Global hotkey: `/` focuses the search input (unless an input is already focused).
|
||||
function onGlobalKey(e) {
|
||||
if (e.key !== '/' || e.metaKey || e.ctrlKey || e.altKey) return;
|
||||
const tag = (document.activeElement && document.activeElement.tagName) || '';
|
||||
if (tag === 'INPUT' || tag === 'TEXTAREA') return;
|
||||
e.preventDefault();
|
||||
if (searchInputEl.value) searchInputEl.value.focus();
|
||||
}
|
||||
|
||||
// Close the dropdown when clicking outside it.
|
||||
function onGlobalClick(e) {
|
||||
if (!searchOpen.value) return;
|
||||
const root = document.querySelector('.docs-search');
|
||||
if (root && !root.contains(e.target)) searchOpen.value = false;
|
||||
}
|
||||
|
||||
onMounted(() => {
|
||||
window.addEventListener('keydown', onGlobalKey);
|
||||
window.addEventListener('mousedown', onGlobalClick);
|
||||
});
|
||||
onBeforeUnmount(() => {
|
||||
window.removeEventListener('keydown', onGlobalKey);
|
||||
window.removeEventListener('mousedown', onGlobalClick);
|
||||
});
|
||||
|
||||
// Prev / Next within the flattened table of contents.
|
||||
const currentIdx = computed(() => flatToc.value.findIndex(e => e.path === currentPath.value));
|
||||
const prevDoc = computed(() => currentIdx.value > 0 ? flatToc.value[currentIdx.value - 1] : null);
|
||||
@@ -88,6 +209,35 @@ const nextDoc = computed(() => currentIdx.value >= 0 && currentIdx.value < fl
|
||||
<button class="sidebar-toggle" @click="sidebarOpen = !sidebarOpen" title="Toggle navigation">
|
||||
{{ sidebarOpen ? '⟨' : '⟩' }}
|
||||
</button>
|
||||
<div class="docs-search">
|
||||
<input
|
||||
ref="searchInputEl"
|
||||
v-model="searchQuery"
|
||||
@input="onSearchInput"
|
||||
@keydown="onSearchKey"
|
||||
@focus="searchResults.length && (searchOpen = true)"
|
||||
type="search"
|
||||
class="search-input"
|
||||
placeholder="Search docs… /"
|
||||
aria-label="Search the Nox documentation"
|
||||
/>
|
||||
<div v-if="searchOpen && searchResults.length" class="search-results" role="listbox">
|
||||
<button
|
||||
v-for="(r, i) in searchResults"
|
||||
:key="r.path + '#' + r.anchor + i"
|
||||
type="button"
|
||||
role="option"
|
||||
class="search-result"
|
||||
:class="{ active: i === searchActive }"
|
||||
@mouseenter="searchActive = i"
|
||||
@mousedown.prevent="openResult(r)"
|
||||
>
|
||||
<div class="search-result-title">{{ r.title }}</div>
|
||||
<div class="search-result-path">{{ r.sectionTitle && r.sectionTitle !== r.title ? r.sectionTitle + ' · ' : '' }}{{ r.path }}</div>
|
||||
<div class="search-result-snippet">{{ r.snippet }}</div>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="grow"></div>
|
||||
<RouterLink to="/nox/examples" class="ghost-link">Examples</RouterLink>
|
||||
<RouterLink to="/nox" class="cta">Open Console</RouterLink>
|
||||
@@ -410,4 +560,97 @@ const nextDoc = computed(() => currentIdx.value >= 0 && currentIdx.value < fl
|
||||
.docs-sidebar { display: none; }
|
||||
.doc-content { padding: 1.5rem 1.2rem; }
|
||||
}
|
||||
|
||||
/* search */
|
||||
.docs-search {
|
||||
position: relative;
|
||||
margin-left: 0.5rem;
|
||||
width: 320px;
|
||||
}
|
||||
.search-input {
|
||||
width: 100%;
|
||||
padding: 6px 10px;
|
||||
font: inherit;
|
||||
font-size: 0.85rem;
|
||||
border: 1px solid #d0d6e3;
|
||||
border-radius: 6px;
|
||||
background: #f4f6fb;
|
||||
color: #1a1a2a;
|
||||
outline: none;
|
||||
transition: border-color 0.12s, background 0.12s;
|
||||
}
|
||||
.search-input:focus {
|
||||
border-color: #1a73e8;
|
||||
background: white;
|
||||
box-shadow: 0 0 0 3px rgba(26, 115, 232, 0.12);
|
||||
}
|
||||
.search-input::placeholder { color: #99a2b8; }
|
||||
.search-results {
|
||||
position: absolute;
|
||||
top: calc(100% + 4px);
|
||||
left: 0;
|
||||
right: 0;
|
||||
background: white;
|
||||
border: 1px solid #d0d6e3;
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 10px 32px rgba(20, 30, 60, 0.16);
|
||||
max-height: 520px;
|
||||
overflow-y: auto;
|
||||
z-index: 50;
|
||||
padding: 4px;
|
||||
}
|
||||
.search-result {
|
||||
display: block;
|
||||
width: 100%;
|
||||
text-align: left;
|
||||
padding: 8px 10px;
|
||||
background: transparent;
|
||||
border: 0;
|
||||
border-radius: 6px;
|
||||
cursor: pointer;
|
||||
font: inherit;
|
||||
}
|
||||
.search-result + .search-result { margin-top: 2px; }
|
||||
.search-result.active,
|
||||
.search-result:hover {
|
||||
background: #eef3ff;
|
||||
}
|
||||
.search-result-title {
|
||||
font-size: 0.92rem;
|
||||
font-weight: 600;
|
||||
color: #1a3a8a;
|
||||
line-height: 1.3;
|
||||
}
|
||||
.search-result-path {
|
||||
font-size: 0.72rem;
|
||||
color: #6a7793;
|
||||
margin-top: 2px;
|
||||
font-family: ui-monospace, "SF Mono", Menlo, Monaco, monospace;
|
||||
}
|
||||
.search-result-snippet {
|
||||
font-size: 0.82rem;
|
||||
color: #4a5468;
|
||||
margin-top: 4px;
|
||||
line-height: 1.4;
|
||||
display: -webkit-box;
|
||||
-webkit-line-clamp: 2;
|
||||
line-clamp: 2;
|
||||
-webkit-box-orient: vertical;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
/* heading pulse on anchor jump */
|
||||
.doc-content :deep(h1.anchor-pulse),
|
||||
.doc-content :deep(h2.anchor-pulse),
|
||||
.doc-content :deep(h3.anchor-pulse) {
|
||||
animation: pulse 1.2s ease-out;
|
||||
}
|
||||
@keyframes pulse {
|
||||
0% { background-color: rgba(26, 115, 232, 0.22); }
|
||||
100% { background-color: transparent; }
|
||||
}
|
||||
|
||||
@media (max-width: 700px) {
|
||||
.docs-search { display: none; }
|
||||
}
|
||||
</style>
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# Caelum self-host stack.
|
||||
#
|
||||
# docker compose up -d # boot MySQL + the app
|
||||
# docker compose logs -f app # tail the server logs
|
||||
# docker compose exec app sh # shell into the app container
|
||||
# docker compose down # stop everything (data persists)
|
||||
# docker compose down -v # nuke the database volume too
|
||||
#
|
||||
# After first boot, run the seed once to create the admin/observer accounts
|
||||
# and sample sites/telescopes:
|
||||
#
|
||||
# docker compose exec app sh -c "cd server && npm run seed"
|
||||
#
|
||||
# Defaults assume a single-host deployment. For production behind a reverse
|
||||
# proxy, set CAELUM_PORT and bind the proxy in front of localhost:${CAELUM_PORT}.
|
||||
|
||||
services:
|
||||
db:
|
||||
image: mysql:8.4
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-change-me-root}
|
||||
MYSQL_DATABASE: ${MYSQL_DATABASE:-caelum_app}
|
||||
MYSQL_USER: ${MYSQL_USER:-caelum}
|
||||
MYSQL_PASSWORD: ${MYSQL_PASSWORD:-change-me}
|
||||
command: >
|
||||
--character-set-server=utf8mb4
|
||||
--collation-server=utf8mb4_unicode_ci
|
||||
--default-authentication-plugin=caching_sha2_password
|
||||
volumes:
|
||||
- caelum-db:/var/lib/mysql
|
||||
# Optional second DB (SpaceDB / NASA-JPL catalog). Toggle via env.
|
||||
- ./docker/initdb:/docker-entrypoint-initdb.d:ro
|
||||
healthcheck:
|
||||
test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-uroot", "-p${MYSQL_ROOT_PASSWORD:-change-me-root}"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 20
|
||||
start_period: 30s
|
||||
# No host port published by default. Add a "ports:" stanza locally if you
|
||||
# want to connect with a MySQL client from the host.
|
||||
|
||||
app:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
PORT: 3002
|
||||
# App database
|
||||
MYSQL_HOST: db
|
||||
MYSQL_PORT: 3306
|
||||
MYSQL_USER: ${MYSQL_USER:-caelum}
|
||||
MYSQL_PASSWORD: ${MYSQL_PASSWORD:-change-me}
|
||||
MYSQL_DATABASE: ${MYSQL_DATABASE:-caelum_app}
|
||||
# SpaceDB (optional — leave empty to disable /spacedb routes)
|
||||
MYSQL_SPACEDB_HOST: db
|
||||
MYSQL_SPACEDB_PORT: 3306
|
||||
MYSQL_SPACEDB_USER: ${MYSQL_USER:-caelum}
|
||||
MYSQL_SPACEDB_PASSWORD: ${MYSQL_PASSWORD:-change-me}
|
||||
MYSQL_SPACEDB_DATABASE: ${MYSQL_SPACEDB_DATABASE:-}
|
||||
# Auth
|
||||
JWT_SECRET: ${JWT_SECRET:-change-me-to-a-random-string}
|
||||
ports:
|
||||
- "${CAELUM_PORT:-3002}:3002"
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3002/api/health"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
|
||||
volumes:
|
||||
caelum-db:
|
||||
Executable
+24
@@ -0,0 +1,24 @@
|
||||
#!/bin/sh
|
||||
# Optional second database for the NASA/JPL SpaceDB catalog.
|
||||
#
|
||||
# mysql:8 only creates the single database named by MYSQL_DATABASE on first
|
||||
# boot. This script runs after that and provisions a SpaceDB schema plus
|
||||
# privileges for the app user. Importing the actual JPL data is out of scope
|
||||
# here — see the "Populate the databases" section of README.md.
|
||||
#
|
||||
# Files in /docker-entrypoint-initdb.d run only when the data dir is fresh.
|
||||
# If you change this and want it re-applied, `docker compose down -v`.
|
||||
|
||||
set -eu
|
||||
|
||||
SPACEDB="${MYSQL_SPACEDB_DATABASE:-caelum_spacedb}"
|
||||
APP_USER="${MYSQL_USER:-caelum}"
|
||||
|
||||
echo "[initdb] provisioning SpaceDB schema: ${SPACEDB} (user: ${APP_USER})"
|
||||
|
||||
mysql -uroot -p"${MYSQL_ROOT_PASSWORD}" <<SQL
|
||||
CREATE DATABASE IF NOT EXISTS \`${SPACEDB}\`
|
||||
CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
GRANT ALL PRIVILEGES ON \`${SPACEDB}\`.* TO '${APP_USER}'@'%';
|
||||
FLUSH PRIVILEGES;
|
||||
SQL
|
||||
Generated
+7
@@ -17,6 +17,7 @@
|
||||
"express": "^4.21.2",
|
||||
"express-async-errors": "^3.1.1",
|
||||
"jsonwebtoken": "^9.0.2",
|
||||
"minisearch": "^7.2.0",
|
||||
"multer": "^2.1.1",
|
||||
"mysql2": "^3.22.3",
|
||||
"socket.io": "^4.8.1",
|
||||
@@ -1089,6 +1090,12 @@
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/minisearch": {
|
||||
"version": "7.2.0",
|
||||
"resolved": "https://registry.npmjs.org/minisearch/-/minisearch-7.2.0.tgz",
|
||||
"integrity": "sha512-dqT2XBYUOZOiC5t2HRnwADjhNS2cecp9u+TJRiJ1Qp/f5qjkeT5APcGPjHw+bz89Ms8Jp+cG4AlE+QZ/QnDglg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/mkdirp-classic": {
|
||||
"version": "0.5.3",
|
||||
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
|
||||
|
||||
+2
-1
@@ -13,6 +13,7 @@
|
||||
"migrate": "node src/migrate.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"@caelum/nox-runtime": "file:../nox/runtime",
|
||||
"bcryptjs": "^2.4.3",
|
||||
"better-sqlite3": "^11.7.0",
|
||||
"cors": "^2.8.5",
|
||||
@@ -20,9 +21,9 @@
|
||||
"express": "^4.21.2",
|
||||
"express-async-errors": "^3.1.1",
|
||||
"jsonwebtoken": "^9.0.2",
|
||||
"minisearch": "^7.2.0",
|
||||
"multer": "^2.1.1",
|
||||
"mysql2": "^3.22.3",
|
||||
"@caelum/nox-runtime": "file:../nox/runtime",
|
||||
"socket.io": "^4.8.1",
|
||||
"uuid": "^11.1.0"
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { run, checkSource } from '@caelum/nox-runtime';
|
||||
import { createRealDriver } from '../services/nox-driver.js';
|
||||
import { searchDocs } from '../services/nox-docs-index.js';
|
||||
|
||||
const router = Router();
|
||||
const here = path.dirname(fileURLToPath(import.meta.url));
|
||||
@@ -94,6 +95,24 @@ router.post('/run-stream', async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
// Docs full-text search. Lazy index built on first hit, cached in-process.
|
||||
// GET /api/nox/docs-search?q=<query>&limit=<n>
|
||||
router.get('/docs-search', async (req, res) => {
|
||||
const q = String(req.query.q || '');
|
||||
const limit = Math.min(50, Math.max(1, parseInt(req.query.limit, 10) || 10));
|
||||
if (!q.trim()) return res.json({ results: [], total: 0 });
|
||||
if (!existsSync(docsDir)) {
|
||||
return res.status(503).json({ error: 'docs not available on this host' });
|
||||
}
|
||||
try {
|
||||
const out = await searchDocs(docsDir, q, limit);
|
||||
res.json(out);
|
||||
} catch (e) {
|
||||
console.error('[nox-docs-search] error:', e);
|
||||
res.status(500).json({ error: e.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Serve documentation markdown. Self-contained handler — no reliance on
|
||||
// serve-static's error shape or Express wildcard semantics, both of which
|
||||
// vary between Express 4 and 5 and bit us before.
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
// Lazy full-text index over the Nox markdown docs.
|
||||
//
|
||||
// On first call, walks `docsDir`, splits each `.md` file by H1/H2/H3, and
|
||||
// builds a Minisearch index keyed by passage. Subsequent calls reuse the
|
||||
// cached index unless the docs directory's mtime advances.
|
||||
//
|
||||
// Each record:
|
||||
// { id, path, anchor, title, sectionTitle, body, weight }
|
||||
//
|
||||
// `weight` boosts top-level pages (README.md, intro chapters) so they appear
|
||||
// above deep reference passages for ambiguous queries.
|
||||
|
||||
import fs from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import MiniSearch from 'minisearch';
|
||||
|
||||
const SECTION_PATTERN = /^(#{1,3})\s+(.+)$/gm;
|
||||
const MAX_BODY_CHARS = 1200; // truncate per-passage body to keep the index light
|
||||
|
||||
function slugify(s) {
|
||||
return String(s)
|
||||
.toLowerCase()
|
||||
.replace(/[^\w\s-]/g, '')
|
||||
.replace(/\s+/g, '-')
|
||||
.replace(/^-+|-+$/g, '');
|
||||
}
|
||||
|
||||
function weightFor(rel, level) {
|
||||
// Lower number = stronger boost. We invert later for Minisearch.
|
||||
let w = 1;
|
||||
if (rel === 'README.md') w += 0.6;
|
||||
if (rel.startsWith('book/0')) w += 0.3;
|
||||
if (level === 1) w += 0.4;
|
||||
if (level === 2) w += 0.15;
|
||||
return w;
|
||||
}
|
||||
|
||||
// Recursively list .md files inside `root`.
|
||||
async function listMarkdown(root) {
|
||||
const out = [];
|
||||
async function walk(dir) {
|
||||
let entries;
|
||||
try {
|
||||
entries = await fs.readdir(dir, { withFileTypes: true });
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
for (const ent of entries) {
|
||||
const full = path.join(dir, ent.name);
|
||||
if (ent.isDirectory()) await walk(full);
|
||||
else if (ent.isFile() && ent.name.toLowerCase().endsWith('.md')) out.push(full);
|
||||
}
|
||||
}
|
||||
await walk(root);
|
||||
return out;
|
||||
}
|
||||
|
||||
// Split one markdown file into passages, one per H1/H2/H3 (plus the preamble
|
||||
// before the first heading). Each passage carries the nearest H1 title for
|
||||
// breadcrumb context.
|
||||
function splitPassages(markdown, rel) {
|
||||
const passages = [];
|
||||
let cursor = 0;
|
||||
let lastH1 = '';
|
||||
// Reset the regex's lastIndex on every call because it's stateful across
|
||||
// matchAll iterators inside the same instance.
|
||||
const headings = [...markdown.matchAll(SECTION_PATTERN)];
|
||||
if (headings.length === 0) {
|
||||
passages.push({ level: 0, title: rel, body: markdown, anchor: '', h1: rel });
|
||||
return passages;
|
||||
}
|
||||
|
||||
// Preamble before first heading (if any).
|
||||
if (headings[0].index > 0) {
|
||||
const body = markdown.slice(0, headings[0].index).trim();
|
||||
if (body) {
|
||||
passages.push({ level: 0, title: rel, body, anchor: '', h1: rel });
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < headings.length; i++) {
|
||||
const h = headings[i];
|
||||
const level = h[1].length;
|
||||
const title = h[2].trim();
|
||||
const next = headings[i + 1];
|
||||
const start = h.index + h[0].length;
|
||||
const end = next ? next.index : markdown.length;
|
||||
const body = markdown.slice(start, end).trim();
|
||||
|
||||
if (level === 1) lastH1 = title;
|
||||
const anchor = level >= 2 ? slugify(title) : '';
|
||||
passages.push({ level, title, body, anchor, h1: lastH1 || title });
|
||||
}
|
||||
|
||||
return passages;
|
||||
}
|
||||
|
||||
function makeRecords(rel, markdown) {
|
||||
const records = [];
|
||||
const passages = splitPassages(markdown, rel);
|
||||
let idx = 0;
|
||||
for (const p of passages) {
|
||||
const truncated = p.body.length > MAX_BODY_CHARS
|
||||
? p.body.slice(0, MAX_BODY_CHARS)
|
||||
: p.body;
|
||||
records.push({
|
||||
id: `${rel}#${p.anchor || `_${idx}`}`,
|
||||
path: rel,
|
||||
anchor: p.anchor,
|
||||
title: p.title,
|
||||
sectionTitle: p.h1,
|
||||
body: truncated,
|
||||
weight: weightFor(rel, p.level),
|
||||
});
|
||||
idx++;
|
||||
}
|
||||
return records;
|
||||
}
|
||||
|
||||
function newIndex(records) {
|
||||
const idx = new MiniSearch({
|
||||
fields: ['title', 'body', 'path'],
|
||||
storeFields: ['path', 'anchor', 'title', 'sectionTitle', 'body', 'weight'],
|
||||
searchOptions: {
|
||||
boost: { title: 4, body: 1, path: 0.5 },
|
||||
prefix: true,
|
||||
fuzzy: 0.2,
|
||||
},
|
||||
});
|
||||
idx.addAll(records);
|
||||
return idx;
|
||||
}
|
||||
|
||||
// Build a fresh index from the docs directory.
|
||||
async function build(docsDir) {
|
||||
const files = await listMarkdown(docsDir);
|
||||
const records = [];
|
||||
for (const full of files) {
|
||||
const rel = path.relative(docsDir, full).split(path.sep).join('/');
|
||||
const md = await fs.readFile(full, 'utf8');
|
||||
records.push(...makeRecords(rel, md));
|
||||
}
|
||||
return { index: newIndex(records), built: Date.now(), count: records.length };
|
||||
}
|
||||
|
||||
// One in-process cache keyed by docsDir. Re-builds on a 5-minute TTL or if the
|
||||
// directory's most-recent mtime advances past the last build time.
|
||||
const cache = new Map();
|
||||
|
||||
async function maybeRebuild(docsDir) {
|
||||
const existing = cache.get(docsDir);
|
||||
if (!existing) {
|
||||
const fresh = await build(docsDir);
|
||||
cache.set(docsDir, fresh);
|
||||
return fresh;
|
||||
}
|
||||
// Cheap freshness probe: compare top-of-tree mtime.
|
||||
try {
|
||||
const stat = await fs.stat(docsDir);
|
||||
if (stat.mtimeMs > existing.built) {
|
||||
const fresh = await build(docsDir);
|
||||
cache.set(docsDir, fresh);
|
||||
return fresh;
|
||||
}
|
||||
} catch { /* if stat fails, fall through with stale cache */ }
|
||||
return existing;
|
||||
}
|
||||
|
||||
// Build a short excerpt around the first query-term match in the body.
|
||||
function makeSnippet(body, query, max = 220) {
|
||||
if (!body) return '';
|
||||
const lower = body.toLowerCase();
|
||||
const terms = query.toLowerCase().split(/\s+/).filter(Boolean);
|
||||
let hit = -1;
|
||||
for (const t of terms) {
|
||||
const i = lower.indexOf(t);
|
||||
if (i >= 0 && (hit < 0 || i < hit)) hit = i;
|
||||
}
|
||||
if (hit < 0) {
|
||||
return body.length > max ? body.slice(0, max).trimEnd() + '…' : body;
|
||||
}
|
||||
const start = Math.max(0, hit - 60);
|
||||
const end = Math.min(body.length, start + max);
|
||||
const prefix = start > 0 ? '…' : '';
|
||||
const suffix = end < body.length ? '…' : '';
|
||||
return prefix + body.slice(start, end).trim() + suffix;
|
||||
}
|
||||
|
||||
// Public entry. `q` is the raw query; `limit` defaults to 10.
|
||||
export async function searchDocs(docsDir, q, limit = 10) {
|
||||
const query = String(q || '').trim();
|
||||
if (!query) return { results: [], total: 0 };
|
||||
const { index, count } = await maybeRebuild(docsDir);
|
||||
// Try AND first for precision. If nothing matches, fall back to OR so
|
||||
// multi-word queries with one rare term still return something useful.
|
||||
let raw = index.search(query, { combineWith: 'AND' });
|
||||
if (raw.length === 0) raw = index.search(query, { combineWith: 'OR' });
|
||||
const ranked = raw
|
||||
.map(r => ({
|
||||
// Combine relevance and weight so well-placed top-level passages win
|
||||
// when scores are otherwise close.
|
||||
score: r.score * (r.weight || 1),
|
||||
path: r.path,
|
||||
anchor: r.anchor,
|
||||
title: r.title,
|
||||
sectionTitle: r.sectionTitle,
|
||||
snippet: makeSnippet(r.body, query),
|
||||
}))
|
||||
.sort((a, b) => b.score - a.score)
|
||||
.slice(0, limit);
|
||||
return { results: ranked, total: raw.length, indexed: count };
|
||||
}
|
||||
Reference in New Issue
Block a user