First public release

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-28 14:49:02 -04:00
co-authored by Claude Opus 5.5
commit d140ea7669
14 changed files with 2129 additions and 0 deletions
+12
View File
@@ -0,0 +1,12 @@
root = true
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
indent_style = space
indent_size = 2
[*.md]
trim_trailing_whitespace = false
+9
View File
@@ -0,0 +1,9 @@
node_modules/
dist/
*.log
.DS_Store
.env
.env.*
!.env.example
.npmrc
dist-cjs/
+6
View File
@@ -0,0 +1,6 @@
stages: [test]
test:
stage: test
image: node:22-alpine
script: [npm ci, npm run build, npm test]
+15
View File
@@ -0,0 +1,15 @@
# Changelog
## 0.1.0 - 2026-09-28
First public release under MIT.
- `verifyRollout()` and `formatRollout()` verify a deployment target by
target, by the artifact each one is serving.
- A successful pipeline whose deploy job is missing from `jobsRun` is a
critical failure.
- `http-status` and `pipeline-status` observations are NOT ASSESSED, never a
pass.
- A target serving the previous artifact is a critical failure.
- `started: false` reports NOT ASSESSED.
- `safeToProceed()` blocks on unverified targets as well as failed ones.
+21
View File
@@ -0,0 +1,21 @@
# Contributing
Issues and merge requests are welcome at
https://gitlab.com/extant2000/rollout-rail.
## Ground rules
- A test that cannot fail proves nothing. If you fix a bug, add a test and
check that it fails against the unfixed code before you submit.
- Measure, don't assume. Two modules with the same line count can still be
different programs.
- Keep dependencies minimal. Every new runtime dependency needs a reason.
- Explain why in comments, not what. The what is already in the code.
- A breaking change needs a major version bump and a note in CHANGELOG.md.
## Before you open a merge request
Run both of these and make sure they pass:
npm run build
npm test
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Extant 2000 LLC
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+99
View File
@@ -0,0 +1,99 @@
# RolloutRail
Verify a deployment target by target, by the artifact each one is actually
serving.
RolloutRail takes what you observed about each target in a rollout and
decides whether the deployment can be evidenced. It deploys nothing. Its only
runtime dependency is `@extant2000/evidence-record`.
## What it checks and why
Three common traps share one shape: a signal that is easy to collect stands
in for the one that matters.
1. A green pipeline is not a deployed artifact. Suppose an unpinned linter
starts failing, the deploy job's dependency is no longer satisfied, and
the deploy job is skipped. A skipped job does not fail a pipeline, so the
pipeline stays green while nothing ships. The job list is the only place
that shows it, so `jobsRun` is a first-class field. A successful pipeline
whose deploy job is missing from `jobsRun` is a critical failure.
2. A 200 is not your build. The old container answering healthily returns
the same 200 as the new one, and a loose smoke test can even pass on
HTTP 500. Only the served artifact (a version, a commit SHA, a build id)
tells them apart.
3. An untouched target reports its previous success. In a staged rollout,
pipeline status for a repo that has not been merged yet returns the
pre-rollout run, which is green. A monitor that polls status reports "all
green" while half the targets have not started. `started: false` reports
NOT ASSESSED.
The only pass is an `artifact-probe` whose `observed` value matches
`expected`. Everything else is NOT ASSESSED and names the weaker signal it
settled for (`pipeline-status`, `http-status` or `none`). A target that is up,
healthy and serving the previous build is a critical failure.
`safeToProceed()` gates the next wave of a staged rollout, and it is strict:
NOT ASSESSED blocks exactly as a failure does. The next wave depends on the
previous one having worked, and "we did not check" is not evidence that it
did.
## Usage
```ts
import { verifyRollout, safeToProceed, formatRollout, type Target } from '@extant2000/rollout-rail'
const targets: Target[] = [
{ name: 'api', expected: 'a1b2c3d', observed: 'a1b2c3d', via: 'artifact-probe', where: 'https://api.example.com/__version' },
{ name: 'billing', expected: 'a1b2c3d', observed: '9f8e7d6', via: 'artifact-probe', where: 'https://billing.example.com/__version' },
{ name: 'web', expected: 'a1b2c3d', observed: 'a1b2c3d', via: 'artifact-probe', where: 'https://web.example.com/__version',
pipelineStatus: 'success', jobsRun: ['build', 'test', 'image-lint'] },
{ name: 'worker', expected: 'a1b2c3d', observed: 'up', via: 'http-status', httpStatus: 200, where: 'https://worker.example.com/healthz' },
{ name: 'search', expected: 'a1b2c3d', observed: 'success', via: 'pipeline-status', pipelineStatus: 'success', where: 'pipeline 41', started: false },
]
const { ok, reason } = safeToProceed(targets)
console.log(formatRollout(targets))
```
`verifyRollout()` returns a `CapabilityReport` from
`@extant2000/evidence-record`. If your deploy job has another name, pass
`{ deployJob: 'ship' }`.
## Sample output
Produced by running the example above. `safeToProceed()` returns `ok: false`
with the reason `2 target(s) failed: web/skipped-deploy, billing`. The report,
without its title line:
```
NOT MET · 5 items examined · 6 findings
2 of 5 target(s) confirmed serving the intended artifact by independent probe
✗ CRITICAL billing is serving 9f8e7d6, expected a1b2c3d [NOT MET]
The target is up, healthy, and running the previous build. This is the failure a status check cannot see.
evidence: https://billing.example.com/__version
✗ CRITICAL web: pipeline is green and "deploy" never ran [NOT MET]
A skipped job does not fail a pipeline. Jobs that ran: build, test, image-lint. Nothing shipped, and the only place that is visible is the job list.
evidence: $ pipeline jobs for web → exit 0
✗ HIGH worker: answered HTTP 200, artifact not checked [NOT ASSESSED]
The previous build returns the same status as the new one. A response code identifies neither the version nor, with a loose smoke test, even the health: a check that accepts any response passes on HTTP 500.
evidence: https://worker.example.com/healthz
· MEDIUM search has not started [NOT ASSESSED]
Any status read from this target right now describes the previous rollout, not this one. It is green because nothing has happened yet.
evidence: pipeline 41
◦ info api is serving a1b2c3d [MET]
evidence: https://api.example.com/__version
◦ info web is serving a1b2c3d [MET]
evidence: https://web.example.com/__version
```
## Install
```
npm install @extant2000/rollout-rail
```
## License
MIT. Copyright (c) 2026 Extant 2000 LLC. See [LICENSE](LICENSE).
+11
View File
@@ -0,0 +1,11 @@
# Security policy
## Reporting a vulnerability
Please report vulnerabilities privately by email to security@extant2000.com.
Do not open a public issue or merge request for a security problem.
Include the affected version, a description of the issue, and steps or a test
that reproduce it if you have them.
We aim to acknowledge every report within 5 business days.
+1504
View File
File diff suppressed because it is too large Load Diff
+46
View File
@@ -0,0 +1,46 @@
{
"name": "@extant2000/rollout-rail",
"version": "0.1.0",
"private": false,
"description": "Pushes one capability across an estate with per-target verification.",
"license": "MIT",
"type": "module",
"main": "./dist-cjs/index.js",
"files": [
"dist",
"dist-cjs",
"src",
"README.md",
"LICENSE"
],
"repository": {
"type": "git",
"url": "https://gitlab.com/extant2000/rollout-rail.git"
},
"module": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.js",
"require": "./dist-cjs/index.js"
}
},
"scripts": {
"build": "tsc && tsc -p tsconfig.cjs.json && node -e \"require('fs').writeFileSync('dist-cjs/package.json', JSON.stringify({type:'commonjs'}))\"",
"test": "vitest run",
"prepublishOnly": "npm run build"
},
"devDependencies": {
"typescript": "^5.6.0",
"vitest": "^2.1.0"
},
"dependencies": {
"@extant2000/evidence-record": "^0.1.2"
},
"author": "Extant 2000 LLC",
"homepage": "https://gitlab.com/extant2000/rollout-rail",
"bugs": {
"url": "https://gitlab.com/extant2000/rollout-rail/-/issues"
}
}
+233
View File
@@ -0,0 +1,233 @@
/**
* RolloutRail — verify a deployment per target, by what is actually serving.
*
* All three traps below are common in practice, and they share one
* shape: a signal that is easy to collect standing in for the one that
* matters.
*
* 1. **A green pipeline is not a deployed artifact.** An unpinned linter
* started failing, the deploy job's dependency was unsatisfied, and the
* job was SKIPPED. A skipped job does not fail a pipeline, so the
* pipeline stayed green and nothing shipped.
*
* 2. **A 200 is not your build.** The old container answering healthily
* returns exactly the same 200 as the new one. Worse, a loose smoke test
* can pass on HTTP 500, so even the status check may not be checking
* what it appears to. Only the served ARTIFACT — a version, a
* commit SHA, a build id — distinguishes them.
*
* 3. **An untouched target reports its previous success.** During a staged
* rollout, querying pipeline status for a repo that has not been merged
* yet returns the PRE-rollout run, which is green. A monitor that polls
* status therefore reports "all green" while half the fleet has not
* started.
*
* So the only pass here is an observed artifact matching the intended one,
* obtained independently of the deploy. Everything else is NOT ASSESSED, and
* says which weaker signal it settled for.
*
* Pure: takes observations, returns findings. It deploys nothing.
*/
import {
type CapabilityReport,
type Finding,
type FormatOptions,
evidence,
formatReport as renderReport,
} from '@extant2000/evidence-record'
/** How a target's served artifact was established. */
export type Probe =
/** The target was asked what it is serving, and answered. The only pass. */
| 'artifact-probe'
/** CI said the pipeline was green. Says nothing about what is serving. */
| 'pipeline-status'
/** Something answered a request. Says nothing about which build. */
| 'http-status'
/** Nothing was checked. */
| 'none'
export type PipelineStatus = 'success' | 'failed' | 'skipped' | 'running' | 'created' | 'canceled'
export interface Target {
name: string
/** The artifact intended for this rollout — a version, SHA or build id. */
expected: string
/** What the target reports serving. `null` when unknown. */
observed: string | null
via: Probe
/** Where the observation came from — a URL, a command, a dashboard. */
where: string
httpStatus?: number
pipelineStatus?: PipelineStatus
/**
* Jobs that actually RAN. A pipeline can be green with the deploy job
* skipped, and this is the only field that shows it.
*/
jobsRun?: string[]
/** Whether this target has been started at all in a staged rollout. */
started?: boolean
}
export interface RolloutOptions {
/** Name of the job that performs the deploy. Defaults to "deploy". */
deployJob?: string
}
function targetEvidence(t: Target) {
return evidence.file(t.where, undefined,
t.observed === null ? 'no artifact observed' : `serving ${t.observed}`,
`probe: ${t.via}${t.pipelineStatus ? `, pipeline ${t.pipelineStatus}` : ''}`)
}
/**
* A green pipeline whose deploy job never ran.
*
* The highest-value check in the library, because the job list is the only
* place this is visible. A skipped job does not fail a pipeline.
*/
function skippedDeploy(t: Target, deployJob: string): Finding | null {
if (t.pipelineStatus !== 'success' || !t.jobsRun) return null
if (t.jobsRun.includes(deployJob)) return null
return {
id: `${t.name}/skipped-deploy`,
summary: `${t.name}: pipeline is green and "${deployJob}" never ran`,
determination: 'fail',
severity: 'critical',
detail: `A skipped job does not fail a pipeline. Jobs that ran: ${t.jobsRun.join(', ') || 'none'}. Nothing shipped, and the only place that is visible is the job list.`,
evidence: [evidence.command(`pipeline jobs for ${t.name}`, 0, t.jobsRun.join(', ') || '(none)', t.where)],
}
}
function findingFor(t: Target): Finding {
const id = t.name
if (t.started === false) {
return {
id,
summary: `${t.name} has not started`,
determination: 'not-assessed',
severity: 'medium',
// Trap 3. A status query here returns the PRE-rollout run, which is
// green, and reads as success for work that has not begun.
detail: 'Any status read from this target right now describes the previous rollout, not this one. It is green because nothing has happened yet.',
evidence: [targetEvidence(t)],
}
}
if (t.via === 'none' || t.observed === null) {
return {
id,
summary: `${t.name}: nothing was observed serving`,
determination: 'not-assessed',
severity: 'high',
evidence: [targetEvidence(t)],
}
}
if (t.via === 'pipeline-status') {
return {
id,
summary: `${t.name}: pipeline reported ${t.pipelineStatus ?? 'success'}, artifact not checked`,
determination: 'not-assessed',
severity: 'high',
detail: 'A green pipeline is not a deployed artifact. Ask the target what it is serving.',
evidence: [targetEvidence(t)],
}
}
if (t.via === 'http-status') {
return {
id,
summary: `${t.name}: answered HTTP ${t.httpStatus ?? '2xx'}, artifact not checked`,
determination: 'not-assessed',
severity: 'high',
detail: 'The previous build returns the same status as the new one. A response code identifies neither the version nor, with a loose smoke test, even the health: a check that accepts any response passes on HTTP 500.',
evidence: [targetEvidence(t)],
}
}
if (t.observed !== t.expected) {
return {
id,
summary: `${t.name} is serving ${t.observed}, expected ${t.expected}`,
determination: 'fail',
severity: 'critical',
detail: 'The target is up, healthy, and running the previous build. This is the failure a status check cannot see.',
evidence: [targetEvidence(t)],
}
}
return {
id,
summary: `${t.name} is serving ${t.observed}`,
determination: 'pass',
severity: 'info',
evidence: [targetEvidence(t)],
}
}
/** Verify a rollout target by target. */
export function verifyRollout(targets: Target[], opts: RolloutOptions = {}): CapabilityReport {
const list = targets ?? []
const deployJob = opts.deployJob ?? 'deploy'
if (list.length === 0) {
return {
capability: 'RolloutRail',
scope: 'no targets',
examined: 0,
findings: [],
notes: ['No targets were supplied, so no rollout was verified.'],
}
}
const skipped = list.map(t => skippedDeploy(t, deployJob)).filter((f): f is Finding => f !== null)
const confirmed = list.filter(t => t.via === 'artifact-probe' && t.observed === t.expected && t.started !== false)
return {
capability: 'RolloutRail',
scope: `${list.length} target(s)`,
examined: list.length,
findings: [...skipped, ...list.map(findingFor)],
notes: [
`${confirmed.length} of ${list.length} target(s) confirmed serving the intended artifact by independent probe`,
],
}
}
/**
* Whether it is safe to continue a staged rollout.
*
* Deliberately strict: NOT ASSESSED blocks exactly as a failure does. The
* point of a staged rollout is that the next wave depends on the previous
* one having worked, and "we did not check" is not evidence that it did.
*/
export function safeToProceed(
targets: Target[],
opts: RolloutOptions = {},
): { ok: boolean, reason: string } {
const report = verifyRollout(targets, opts)
if (report.examined === 0) return { ok: false, reason: 'No targets were verified.' }
const bad = report.findings.filter(f => f.determination === 'fail')
const unknown = report.findings.filter(f => f.determination === 'not-assessed')
if (bad.length > 0) {
return { ok: false, reason: `${bad.length} target(s) failed: ${bad.map(f => f.id).join(', ')}` }
}
if (unknown.length > 0) {
return {
ok: false,
reason: `${unknown.length} target(s) unverified: ${unknown.map(f => f.id).join(', ')}. The next wave depends on this one having worked, and "not checked" is not evidence that it did.`,
}
}
return { ok: true, reason: `All ${report.examined} target(s) confirmed serving the intended artifact.` }
}
/** Render a rollout for a terminal, CI log or evidence package. */
export function formatRollout(targets: Target[], opts: RolloutOptions & FormatOptions = {}): string {
return renderReport(verifyRollout(targets, opts), opts)
}
+135
View File
@@ -0,0 +1,135 @@
import { describe, expect, it } from 'vitest'
import { overall, validateReport } from '@extant2000/evidence-record'
import { formatRollout, safeToProceed, verifyRollout, type Target } from '../src/index.js'
const shipped = (name: string, sha = 'a1b2c3d'): Target =>
({ name, expected: sha, observed: sha, via: 'artifact-probe', where: `https://${name}/__version` })
describe('a green pipeline is not a deployed artifact', () => {
it('CATCHES THE SKIPPED DEPLOY: deploy skipped, pipeline still green', () => {
// An unpinned linter failed, the deploy job's dependency was
// unsatisfied, and it was SKIPPED. A skipped job does not fail a
// pipeline, so nothing ships behind a green tick.
const t: Target = {
name: 'api',
expected: 'a1b2c3d',
observed: 'a1b2c3d',
via: 'artifact-probe',
where: 'gitlab pipelines/123',
pipelineStatus: 'success',
jobsRun: ['build', 'test', 'image-lint'],
}
const r = verifyRollout([t])
const f = r.findings.find(x => x.id === 'api/skipped-deploy')!
expect(f.determination).toBe('fail')
expect(f.severity).toBe('critical')
expect(f.detail).toContain('build, test, image-lint')
expect(overall(r)).toBe('fail')
})
it('does not fire when the deploy job actually ran', () => {
const t: Target = { ...shipped('api'), pipelineStatus: 'success', jobsRun: ['build', 'deploy'] }
expect(verifyRollout([t]).findings.some(f => f.id.endsWith('/skipped-deploy'))).toBe(false)
expect(overall(verifyRollout([t]))).toBe('pass')
})
it('honours a custom deploy job name', () => {
const t: Target = { ...shipped('api'), pipelineStatus: 'success', jobsRun: ['build', 'ship'] }
expect(verifyRollout([t], { deployJob: 'ship' }).findings.some(f => f.id.endsWith('/skipped-deploy'))).toBe(false)
expect(verifyRollout([t], { deployJob: 'deploy' }).findings.some(f => f.id.endsWith('/skipped-deploy'))).toBe(true)
})
})
describe('a 200 is not your build', () => {
it('refuses to accept an HTTP status as a deployment check', () => {
const t: Target = {
name: 'web', expected: 'a1b2c3d', observed: 'up', via: 'http-status',
httpStatus: 200, where: 'https://web.example.com/healthz',
}
const r = verifyRollout([t])
expect(r.findings[0]!.determination).toBe('not-assessed')
expect(r.findings[0]!.detail).toContain('passes on HTTP 500')
expect(overall(r)).toBe('not-assessed')
})
it('refuses to accept a pipeline status as a deployment check', () => {
const t: Target = {
name: 'worker', expected: 'a1b2c3d', observed: 'success', via: 'pipeline-status',
pipelineStatus: 'success', where: 'gitlab pipelines/9',
}
expect(verifyRollout([t]).findings[0]!.determination).toBe('not-assessed')
expect(formatRollout([t])).toContain('A green pipeline is not a deployed artifact')
})
it('CATCHES THE HEALTHY OLD BUILD — up, 200, and the previous version', () => {
const t: Target = {
name: 'billing', expected: 'a1b2c3d', observed: '9f8e7d6', via: 'artifact-probe',
where: 'https://billing.example.com/__version',
}
const f = verifyRollout([t]).findings[0]!
expect(f.determination).toBe('fail')
expect(f.summary).toContain('serving 9f8e7d6, expected a1b2c3d')
expect(f.detail).toContain('up, healthy, and running the previous build')
})
it('passes only on an artifact probe that matches', () => {
expect(overall(verifyRollout([shipped('api')]))).toBe('pass')
})
})
describe('an untouched target reports its previous success', () => {
it('reports a not-yet-started target as unassessed, not green', () => {
// Querying status for a repo not yet merged returns the PRE-rollout run.
const t: Target = {
name: 'search', expected: 'a1b2c3d', observed: 'success', via: 'pipeline-status',
pipelineStatus: 'success', where: 'gitlab pipelines/1', started: false,
}
const f = verifyRollout([t]).findings[0]!
expect(f.determination).toBe('not-assessed')
expect(f.detail).toContain('green because nothing has happened yet')
})
it('does not count an unstarted target as confirmed', () => {
const t: Target = { ...shipped('search'), started: false }
expect(formatRollout([t])).toContain('0 of 1 target(s) confirmed')
})
})
describe('safeToProceed blocks on unverified, not only on failed', () => {
it('proceeds when every target is confirmed by probe', () => {
const r = safeToProceed([shipped('a'), shipped('b')])
expect(r.ok).toBe(true)
expect(r.reason).toContain('All 2 target(s) confirmed')
})
it('blocks on a failure', () => {
expect(safeToProceed([shipped('a'), { name: 'b', expected: 'x', observed: 'y', via: 'artifact-probe', where: 'w' }]).ok).toBe(false)
})
it('blocks on unverified — "not checked" is not evidence it worked', () => {
const r = safeToProceed([
shipped('a'),
{ name: 'b', expected: 'x', observed: 'up', via: 'http-status', where: 'w' },
])
expect(r.ok).toBe(false)
expect(r.reason).toContain('is not evidence that it did')
})
it('blocks on an empty target list', () => {
expect(safeToProceed([]).ok).toBe(false)
})
})
describe('conformance with the evidence-record standard', () => {
it('an empty rollout is not-assessed, never clean', () => {
expect(overall(verifyRollout([]))).toBe('not-assessed')
expect(formatRollout([])).not.toContain('✓')
})
it('every conclusion carries a citation', () => {
expect(validateReport(verifyRollout([
shipped('a'),
{ name: 'b', expected: 'x', observed: 'y', via: 'artifact-probe', where: 'w', pipelineStatus: 'success', jobsRun: ['build'] },
]))).toEqual([])
})
})
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"module": "CommonJS",
"moduleResolution": "Node",
"outDir": "dist-cjs",
"declaration": false
}
}
+8
View File
@@ -0,0 +1,8 @@
{
"compilerOptions": {
"target": "ES2022", "module": "ES2022", "moduleResolution": "bundler",
"declaration": true, "outDir": "dist", "rootDir": "src",
"strict": true, "skipLibCheck": true
},
"include": ["src/**/*.ts"]
}