First public release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,12 @@
|
||||
root = true
|
||||
|
||||
[*]
|
||||
charset = utf-8
|
||||
end_of_line = lf
|
||||
insert_final_newline = true
|
||||
trim_trailing_whitespace = true
|
||||
indent_style = space
|
||||
indent_size = 2
|
||||
|
||||
[*.md]
|
||||
trim_trailing_whitespace = false
|
||||
@@ -0,0 +1,9 @@
|
||||
node_modules/
|
||||
dist/
|
||||
*.log
|
||||
.DS_Store
|
||||
.env*
|
||||
!.env.example
|
||||
.npmrc
|
||||
dist/
|
||||
dist-cjs/
|
||||
@@ -0,0 +1,6 @@
|
||||
stages: [test]
|
||||
|
||||
test:
|
||||
stage: test
|
||||
image: node:22-alpine
|
||||
script: [npm ci, npm run build, npm test]
|
||||
@@ -0,0 +1,15 @@
|
||||
# Changelog
|
||||
|
||||
## 0.2.0 - 2026-09-28
|
||||
|
||||
First public release under MIT.
|
||||
|
||||
- `computeCoverage()` computes traceability coverage from loaded rows. A
|
||||
requirement with no verifications is never counted as verified.
|
||||
- `detectGaps()` finds orphaned requirements, missing tests and risks,
|
||||
suspect links and failed verifications.
|
||||
- `qualityScore()` gives a weighted composite score and grade, with
|
||||
injectable weights and thresholds.
|
||||
- `toReport()` and `formatTrace()` produce an `@extant2000/evidence-record`
|
||||
report that cites the row behind every gap and splits the score into its
|
||||
inputs. An empty requirement set is not assessed.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Contributing
|
||||
|
||||
Issues and merge requests are welcome at
|
||||
https://gitlab.com/extant2000/trace-proof.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **A test that cannot fail proves nothing.** If you fix a bug, add a test
|
||||
that fails without your fix, and check that it does fail before you submit.
|
||||
- Measure, do not assume. Two modules with the same line count can be
|
||||
different programs.
|
||||
- Keep dependencies minimal. Every new dependency needs a reason.
|
||||
- Explain why in comments, not what.
|
||||
|
||||
## Before you open a merge request
|
||||
|
||||
Run both of these and make sure they pass:
|
||||
|
||||
```
|
||||
npm run build
|
||||
npm test
|
||||
```
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Extant 2000 LLC
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,98 @@
|
||||
# TraceProof
|
||||
|
||||
Builds a requirements-to-evidence traceability matrix from rows you load, and
|
||||
reports the gaps and a quality score that leads back to those rows. It is
|
||||
used in Hitt Hosting products.
|
||||
|
||||
## What it does and why
|
||||
|
||||
You load requirements, their links to verifications, risks, parents and
|
||||
stakeholder needs, and the verification items themselves, from whatever
|
||||
database you use. TraceProof takes the rows as plain objects and computes:
|
||||
|
||||
- coverage: how many requirements are traced to a test, verified, linked to a
|
||||
risk, linked to a parent and linked to a stakeholder need;
|
||||
- gaps: orphaned requirements, requirements with no test or no risk, suspect
|
||||
links, and failed verifications;
|
||||
- a composite quality score from 0 to 100 with a letter grade.
|
||||
|
||||
A few rules keep the numbers from flattering the programme.
|
||||
|
||||
**A requirement with no verifications is not verified.** `[].every(...)` is
|
||||
`true`, so a naive check reports a requirement with no evidence at all as
|
||||
fully verified. A requirement counts as verified only when it has at least one
|
||||
verification and every one of them is `Passed` or `Waived`.
|
||||
|
||||
A requirement linked to a test both through a junction table and through a
|
||||
foreign key on the verification item is counted once.
|
||||
|
||||
Absent evidence and evidence of failure are different claims. A missing test,
|
||||
risk or parent is reported as not assessed. A failed verification is a
|
||||
failure, and it is the only critical gap.
|
||||
|
||||
An empty requirement set is not assessed, not 0% with grade F. Zero next to an
|
||||
F reads as "this programme traced nothing" when the truth is that nothing was
|
||||
loaded. It is the same error as reporting 100% uptime from zero samples, in
|
||||
the other direction.
|
||||
|
||||
A composite score is easy to assert and hard to check. The report splits it
|
||||
into its four inputs, each with the count of requirements it was computed
|
||||
over, so a reader can redo the arithmetic. Every gap cites the row that
|
||||
produced it, for example `requirements#<id>`, with the table name set by
|
||||
`requirementTable` and `verificationTable`.
|
||||
|
||||
## Scoring
|
||||
|
||||
The default weights are traced to a test 30%, verified 30%, linked to a risk
|
||||
20% and linked to a parent 20%. Grades are A from 90, B from 75, C from 60 and
|
||||
D from 40. `qualityScore()` accepts other weights and thresholds for a
|
||||
different accreditation regime. Weights are not normalised: if they do not
|
||||
sum to 1, the score will not span 0 to 100, which is a caller error worth
|
||||
seeing. `toReport()` uses the default weights.
|
||||
|
||||
## Usage
|
||||
|
||||
```ts
|
||||
import { computeCoverage, detectGaps, qualityScore, formatTrace, type TraceInput } from '@extant2000/trace-proof'
|
||||
|
||||
const input: TraceInput = {
|
||||
requirements: [
|
||||
{ id: 'r1', req_number: 'SYS-001', title: 'Deorbit within 25 years', parent_id: 'p1' },
|
||||
{ id: 'r2', req_number: 'SYS-002', title: 'Survive launch loads', parent_id: 'p1' },
|
||||
],
|
||||
requirementVerifications: [{ requirement_id: 'r1' }],
|
||||
requirementRisks: [{ requirement_id: 'r1' }, { requirement_id: 'r2' }],
|
||||
verificationItems: [{ id: 'v1', requirement_id: 'r1', status: 'Passed', method: 'Test' }],
|
||||
}
|
||||
|
||||
const coverage = computeCoverage(input) // pct_test 50, pct_verified 50, ...
|
||||
const { gaps } = detectGaps(input) // SYS-002 has no test
|
||||
const score = qualityScore(coverage) // { score: 70, grade: 'C', ... }
|
||||
|
||||
console.log(formatTrace(input, { requirementTable: 'requirements' }))
|
||||
```
|
||||
|
||||
The score lines from that output:
|
||||
|
||||
```
|
||||
· MEDIUM traced to a test: 1 of 2 (50%) [NOT MET]
|
||||
· MEDIUM verified: 1 of 2 (50%) [NOT MET]
|
||||
◦ info linked to a parent: 2 of 2 (100%) [MET]
|
||||
◦ info linked to a risk: 2 of 2 (100%) [MET]
|
||||
```
|
||||
|
||||
The report also lists SYS-002 as not assessed for having no linked test, and
|
||||
notes the composite score of 70/100, grade C.
|
||||
|
||||
Gap ids (`gap-0`, `gap-1`, ...) are positional and only stable within one
|
||||
call. Do not store them as entity keys.
|
||||
|
||||
## Install
|
||||
|
||||
```
|
||||
npm install @extant2000/trace-proof
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT. See [LICENSE](LICENSE).
|
||||
@@ -0,0 +1,6 @@
|
||||
# Security
|
||||
|
||||
Please report vulnerabilities privately by email to security@extant2000.com.
|
||||
Do not open a public issue for a security problem.
|
||||
|
||||
We aim to acknowledge every report within 5 business days.
|
||||
Generated
+1504
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"name": "@extant2000/trace-proof",
|
||||
"version": "0.2.0",
|
||||
"private": false,
|
||||
"description": "Links requirements to the evidence that they are met.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
"files": [
|
||||
"dist",
|
||||
"src",
|
||||
"README.md",
|
||||
"LICENSE"
|
||||
],
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://gitlab.com/extant2000/trace-proof.git"
|
||||
},
|
||||
"types": "dist/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./dist/index.d.ts",
|
||||
"import": "./dist/index.js"
|
||||
}
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"test": "vitest run",
|
||||
"prepublishOnly": "npm run build"
|
||||
},
|
||||
"devDependencies": {
|
||||
"typescript": "^5.6.0",
|
||||
"vitest": "^2.1.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@extant2000/evidence-record": "^0.1.2"
|
||||
},
|
||||
"author": "Extant 2000 LLC",
|
||||
"homepage": "https://gitlab.com/extant2000/trace-proof",
|
||||
"bugs": {
|
||||
"url": "https://gitlab.com/extant2000/trace-proof/-/issues"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
import type { Coverage, TraceInput } from './types.js'
|
||||
|
||||
/** Requirement ids that have at least one verification linked, by either route. */
|
||||
function testedRequirementIds(input: TraceInput): Set<string> {
|
||||
const ids = new Set<string>()
|
||||
// Two routes exist to the same relationship: an explicit junction table and
|
||||
// an FK on the verification item. Both count as "traced to test", and a
|
||||
// requirement linked by both must not be double-counted — hence a Set.
|
||||
for (const rv of input.requirementVerifications ?? []) ids.add(rv.requirement_id)
|
||||
for (const vi of input.verificationItems ?? []) {
|
||||
if (vi.requirement_id) ids.add(vi.requirement_id)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
function idsFrom(rows: { requirement_id: string }[] | undefined): Set<string> {
|
||||
const ids = new Set<string>()
|
||||
for (const r of rows ?? []) ids.add(r.requirement_id)
|
||||
return ids
|
||||
}
|
||||
|
||||
/**
|
||||
* A requirement counts as verified only when it has at least one verification
|
||||
* AND every one of them is Passed or Waived.
|
||||
*
|
||||
* The "at least one" clause matters: without it, a requirement with zero
|
||||
* verifications vacuously satisfies `every()` and would be reported as
|
||||
* verified — the most flattering possible reading of no evidence at all.
|
||||
*/
|
||||
function verifiedRequirementCount(input: TraceInput): number {
|
||||
const byRequirement = new Map<string, string[]>()
|
||||
for (const vi of input.verificationItems ?? []) {
|
||||
if (!vi.requirement_id) continue
|
||||
const list = byRequirement.get(vi.requirement_id)
|
||||
if (list) list.push(vi.status)
|
||||
else byRequirement.set(vi.requirement_id, [vi.status])
|
||||
}
|
||||
|
||||
let verified = 0
|
||||
for (const statuses of byRequirement.values()) {
|
||||
if (statuses.length > 0 && statuses.every(s => s === 'Passed' || s === 'Waived')) {
|
||||
verified++
|
||||
}
|
||||
}
|
||||
return verified
|
||||
}
|
||||
|
||||
const EMPTY_COVERAGE: Coverage = {
|
||||
total_requirements: 0,
|
||||
traced_to_parent: 0,
|
||||
traced_to_test: 0,
|
||||
traced_to_risk: 0,
|
||||
verified: 0,
|
||||
with_stakeholder_need: 0,
|
||||
pct_parent: 0,
|
||||
pct_test: 0,
|
||||
pct_risk: 0,
|
||||
pct_verified: 0,
|
||||
pct_stakeholder: 0,
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute traceability coverage over an already-loaded set of rows.
|
||||
*
|
||||
* Pure: no I/O, no clock, no database. Given the same rows it returns the same
|
||||
* numbers, which is what makes a coverage figure auditable.
|
||||
*/
|
||||
export function computeCoverage(input: TraceInput): Coverage {
|
||||
const requirements = input.requirements ?? []
|
||||
const total = requirements.length
|
||||
if (total === 0) return { ...EMPTY_COVERAGE }
|
||||
|
||||
const withParent = requirements.filter(r => r.parent_id).length
|
||||
const tested = testedRequirementIds(input)
|
||||
const risked = idsFrom(input.requirementRisks)
|
||||
const stakeholder = idsFrom(input.stakeholderNeedRequirements)
|
||||
const verified = verifiedRequirementCount(input)
|
||||
|
||||
const pct = (n: number) => Math.round((n / total) * 100)
|
||||
|
||||
return {
|
||||
total_requirements: total,
|
||||
traced_to_parent: withParent,
|
||||
traced_to_test: tested.size,
|
||||
traced_to_risk: risked.size,
|
||||
verified,
|
||||
with_stakeholder_need: stakeholder.size,
|
||||
pct_parent: pct(withParent),
|
||||
pct_test: pct(tested.size),
|
||||
pct_risk: pct(risked.size),
|
||||
pct_verified: pct(verified),
|
||||
pct_stakeholder: pct(stakeholder.size),
|
||||
}
|
||||
}
|
||||
+101
@@ -0,0 +1,101 @@
|
||||
import type { Gap, RequirementRow, TraceInput } from './types.js'
|
||||
|
||||
function displayId(req: RequirementRow): string {
|
||||
return req.req_number || req.id.slice(0, 8)
|
||||
}
|
||||
|
||||
function idsFrom(rows: { requirement_id: string }[] | undefined): Set<string> {
|
||||
const ids = new Set<string>()
|
||||
for (const r of rows ?? []) ids.add(r.requirement_id)
|
||||
return ids
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect traceability gaps: orphaned requirements, missing coverage, suspect
|
||||
* links, and failed verifications.
|
||||
*
|
||||
* Pure. Gap ids are positional (`gap-0`, `gap-1`, …) and therefore stable only
|
||||
* within a single call — they identify a row in THIS report, not a durable
|
||||
* record. Do not persist them as if they were entity keys.
|
||||
*/
|
||||
export function detectGaps(input: TraceInput): { gaps: Gap[], total: number } {
|
||||
const gaps: Gap[] = []
|
||||
const requirements = input.requirements ?? []
|
||||
if (requirements.length === 0) return { gaps, total: 0 }
|
||||
|
||||
const tested = new Set<string>()
|
||||
for (const rv of input.requirementVerifications ?? []) tested.add(rv.requirement_id)
|
||||
for (const vi of input.verificationItems ?? []) {
|
||||
if (vi.requirement_id) tested.add(vi.requirement_id)
|
||||
}
|
||||
|
||||
const risked = idsFrom(input.requirementRisks)
|
||||
const suspect = idsFrom(input.suspectLinks)
|
||||
|
||||
let idx = 0
|
||||
const push = (g: Omit<Gap, 'id'>) => { gaps.push({ id: `gap-${idx++}`, ...g }) }
|
||||
|
||||
for (const req of requirements) {
|
||||
const base = {
|
||||
entity_type: 'requirement' as const,
|
||||
entity_id: req.id,
|
||||
display_id: displayId(req),
|
||||
title: req.title || 'Untitled',
|
||||
}
|
||||
|
||||
// A top-level requirement legitimately has no parent, so this is reported
|
||||
// as minor: it flags "check the hierarchy", not "this is wrong".
|
||||
if (!req.parent_id) {
|
||||
push({
|
||||
...base,
|
||||
gap_type: 'orphaned',
|
||||
severity: 'minor',
|
||||
description: 'No parent requirement — may be an orphan or missing hierarchy link',
|
||||
})
|
||||
}
|
||||
|
||||
if (!tested.has(req.id)) {
|
||||
push({
|
||||
...base,
|
||||
gap_type: 'no_test',
|
||||
severity: 'major',
|
||||
description: 'No verification item or test linked to this requirement',
|
||||
})
|
||||
}
|
||||
|
||||
if (!risked.has(req.id)) {
|
||||
push({
|
||||
...base,
|
||||
gap_type: 'no_risk',
|
||||
severity: 'minor',
|
||||
description: 'No risk assessment linked to this requirement',
|
||||
})
|
||||
}
|
||||
|
||||
if (suspect.has(req.id)) {
|
||||
push({
|
||||
...base,
|
||||
gap_type: 'suspect',
|
||||
severity: 'major',
|
||||
description: 'Has suspect verification links that need review',
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Failed verifications are the only critical gap: everything above is
|
||||
// absent evidence, this is evidence that the thing does not work.
|
||||
for (const vi of input.verificationItems ?? []) {
|
||||
if (vi.status !== 'Failed') continue
|
||||
push({
|
||||
entity_type: 'verification',
|
||||
entity_id: vi.id,
|
||||
display_id: vi.id.slice(0, 8),
|
||||
title: `${vi.method || 'Verification'} — Failed`,
|
||||
gap_type: 'failed_verification',
|
||||
severity: 'critical',
|
||||
description: 'Verification item has failed status',
|
||||
})
|
||||
}
|
||||
|
||||
return { gaps, total: gaps.length }
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
export * from './types.js'
|
||||
export { computeCoverage } from './coverage.js'
|
||||
export { detectGaps } from './gaps.js'
|
||||
export { qualityScore, DEFAULT_WEIGHTS, DEFAULT_THRESHOLDS } from './score.js'
|
||||
export { toReport, formatTrace, displayIdOf } from './report.js'
|
||||
export type { ReportOptions } from './report.js'
|
||||
+129
@@ -0,0 +1,129 @@
|
||||
import {
|
||||
type CapabilityReport,
|
||||
type Finding,
|
||||
type FormatOptions,
|
||||
type Severity,
|
||||
evidence,
|
||||
formatReport as renderReport,
|
||||
} from '@extant2000/evidence-record'
|
||||
import type { Coverage, Gap, QualityScore, RequirementRow, TraceInput } from './types.js'
|
||||
import { computeCoverage } from './coverage.js'
|
||||
import { detectGaps } from './gaps.js'
|
||||
import { qualityScore } from './score.js'
|
||||
|
||||
export interface ReportOptions {
|
||||
/**
|
||||
* Table name used in citations, so a finding reads
|
||||
* `mission_requirements#a1b2c3` and can be looked up. Defaults to the generic
|
||||
* name because this library is deliberately not tied to one schema.
|
||||
*/
|
||||
requirementTable?: string
|
||||
verificationTable?: string
|
||||
}
|
||||
|
||||
const SEVERITY: Record<Gap['severity'], Severity> = {
|
||||
critical: 'critical',
|
||||
major: 'high',
|
||||
minor: 'info',
|
||||
}
|
||||
|
||||
/**
|
||||
* Cite the row that produced a gap.
|
||||
*
|
||||
* This is the gap the first version left open: gaps knew their `entity_id`
|
||||
* but nothing rendered it, so a reader saw "quality 50/100, grade D" with no
|
||||
* route back to the requirements that produced the number.
|
||||
*/
|
||||
function gapFinding(g: Gap, opts: ReportOptions): Finding {
|
||||
const table = g.entity_type === 'requirement'
|
||||
? (opts.requirementTable ?? 'requirements')
|
||||
: (opts.verificationTable ?? 'verification_items')
|
||||
|
||||
return {
|
||||
id: g.id,
|
||||
summary: `${g.display_id} — ${g.title}: ${g.description}`,
|
||||
// A failed verification is evidence the thing does not work. Everything
|
||||
// else here is absent evidence, which is a different claim and gets the
|
||||
// determination that says so.
|
||||
determination: g.gap_type === 'failed_verification' ? 'fail' : 'not-assessed',
|
||||
severity: SEVERITY[g.severity],
|
||||
evidence: [evidence.record(table, g.entity_id, undefined, g.display_id, g.gap_type)],
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Turn each component of the composite score into a citable measurement.
|
||||
*
|
||||
* A composite number is the easiest thing in a report to assert and the
|
||||
* hardest to check. Splitting it back into its four inputs — each carrying
|
||||
* the requirement count it was computed over — makes it arithmetic a reader
|
||||
* can redo rather than a figure they have to accept.
|
||||
*/
|
||||
function scoreFindings(c: Coverage, q: QualityScore): Finding[] {
|
||||
const n = c.total_requirements
|
||||
const parts: [string, number, number, string][] = [
|
||||
['traced to a test', c.pct_test, c.traced_to_test, 'traced'],
|
||||
['verified', c.pct_verified, c.verified, 'verified'],
|
||||
['linked to a risk', c.pct_risk, c.traced_to_risk, 'risk'],
|
||||
['linked to a parent', c.pct_parent, c.traced_to_parent, 'parent'],
|
||||
]
|
||||
|
||||
return parts.map(([label, pct, count, key]) => ({
|
||||
id: `score.${key}`,
|
||||
summary: `${label}: ${count} of ${n} (${pct}%)`,
|
||||
determination: pct === 100 ? 'pass' : 'fail',
|
||||
severity: pct >= 75 ? 'info' : pct >= 40 ? 'medium' : 'high',
|
||||
detail: `Contributes to the composite score of ${q.score}/100, grade ${q.grade}.`,
|
||||
evidence: [evidence.measurement(label, pct, n, { unit: '%' })],
|
||||
}))
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the portfolio-standard report for one traceability scope.
|
||||
*
|
||||
* An empty requirement set is `not-assessed`, not 0%. `computeCoverage`
|
||||
* correctly returns zeros for no rows, but a zero percentage rendered next to
|
||||
* a grade F reads as "this programme traced nothing", when the truth is that
|
||||
* nothing was loaded. Same class of error as reporting 100% uptime from zero
|
||||
* samples, in the opposite direction.
|
||||
*/
|
||||
export function toReport(input: TraceInput, opts: ReportOptions = {}): CapabilityReport {
|
||||
const coverage = computeCoverage(input)
|
||||
const { gaps } = detectGaps(input)
|
||||
const q = qualityScore(coverage)
|
||||
const n = coverage.total_requirements
|
||||
|
||||
if (n === 0) {
|
||||
return {
|
||||
capability: 'TraceProof',
|
||||
scope: 'no requirements loaded',
|
||||
examined: 0,
|
||||
findings: [],
|
||||
notes: ['No requirements were loaded. This is not a score of zero — nothing was traced because nothing was read.'],
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
capability: 'TraceProof',
|
||||
scope: `${n} requirement${n === 1 ? '' : 's'}`,
|
||||
examined: n,
|
||||
findings: [...scoreFindings(coverage, q), ...gaps.map(g => gapFinding(g, opts))],
|
||||
notes: [
|
||||
`composite quality ${q.score}/100, grade ${q.grade}`,
|
||||
`weights: traced 30%, verified 30%, risk 20%, parent 20% — reweight per accreditation regime`,
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
/** Render a traceability scope for a terminal, CI log or evidence package. */
|
||||
export function formatTrace(
|
||||
input: TraceInput,
|
||||
opts: ReportOptions & FormatOptions = {},
|
||||
): string {
|
||||
return renderReport(toReport(input, opts), opts)
|
||||
}
|
||||
|
||||
/** Convenience: the display id used in citations, exported for callers. */
|
||||
export function displayIdOf(req: RequirementRow): string {
|
||||
return req.req_number || req.id.slice(0, 8)
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
import type {
|
||||
Coverage,
|
||||
GradeThresholds,
|
||||
QualityGrade,
|
||||
QualityScore,
|
||||
QualityWeights,
|
||||
} from './types.js'
|
||||
|
||||
/** Default weights. Sum to 1. */
|
||||
export const DEFAULT_WEIGHTS: QualityWeights = {
|
||||
traced: 0.3,
|
||||
verified: 0.3,
|
||||
risk: 0.2,
|
||||
parent: 0.2,
|
||||
}
|
||||
|
||||
export const DEFAULT_THRESHOLDS: GradeThresholds = { A: 90, B: 75, C: 60, D: 40 }
|
||||
|
||||
function grade(score: number, t: GradeThresholds): QualityGrade {
|
||||
if (score >= t.A) return 'A'
|
||||
if (score >= t.B) return 'B'
|
||||
if (score >= t.C) return 'C'
|
||||
if (score >= t.D) return 'D'
|
||||
return 'F'
|
||||
}
|
||||
|
||||
/**
|
||||
* Composite 0–100 traceability quality score with a letter grade.
|
||||
*
|
||||
* Takes a `Coverage` object rather than fetching one, so reusing this
|
||||
* arithmetic never needs a network round-trip, auth replay and JSON
|
||||
* encode/decode just to multiply four numbers.
|
||||
*
|
||||
* Weights and thresholds are injectable so a different accreditation regime can
|
||||
* reweight without forking. Weights are NOT normalised: if they do not sum to
|
||||
* 1 the score simply will not span 0–100, which is a caller error worth
|
||||
* surfacing loudly rather than silently correcting.
|
||||
*/
|
||||
export function qualityScore(
|
||||
coverage: Pick<Coverage, 'pct_test' | 'pct_verified' | 'pct_risk' | 'pct_parent'>,
|
||||
weights: QualityWeights = DEFAULT_WEIGHTS,
|
||||
thresholds: GradeThresholds = DEFAULT_THRESHOLDS,
|
||||
): QualityScore {
|
||||
const traced = coverage.pct_test
|
||||
const verified = coverage.pct_verified
|
||||
const riskCov = coverage.pct_risk
|
||||
const parentCov = coverage.pct_parent
|
||||
|
||||
const score = Math.round(
|
||||
traced * weights.traced
|
||||
+ verified * weights.verified
|
||||
+ riskCov * weights.risk
|
||||
+ parentCov * weights.parent,
|
||||
)
|
||||
|
||||
return {
|
||||
score,
|
||||
breakdown: {
|
||||
traced_pct: traced,
|
||||
verified_pct: verified,
|
||||
risk_coverage_pct: riskCov,
|
||||
parent_coverage_pct: parentCov,
|
||||
},
|
||||
grade: grade(score, thresholds),
|
||||
}
|
||||
}
|
||||
+119
@@ -0,0 +1,119 @@
|
||||
/**
|
||||
* Row shapes TraceProof operates on.
|
||||
*
|
||||
* Deliberately structural, not tied to any ORM or database client: the caller
|
||||
* loads rows however it likes and hands them over. That keeps the scoring and
|
||||
* gap-detection logic testable without standing up a database, and lets a
|
||||
* non-Supabase system use the same matrix.
|
||||
*/
|
||||
|
||||
/** A requirement in the traceability tree. */
|
||||
export interface RequirementRow {
|
||||
id: string
|
||||
/** Null means no parent — reported as an orphan gap. */
|
||||
parent_id?: string | null
|
||||
/** Human-facing identifier (e.g. "SYS-014"). Falls back to a short id. */
|
||||
req_number?: string | null
|
||||
title?: string | null
|
||||
}
|
||||
|
||||
/** A verification item (test, analysis, inspection, demonstration). */
|
||||
export interface VerificationItemRow {
|
||||
id: string
|
||||
requirement_id?: string | null
|
||||
/** 'Passed' | 'Failed' | 'Waived' | anything else the caller uses. */
|
||||
status: string
|
||||
method?: string | null
|
||||
}
|
||||
|
||||
/** A junction row linking a requirement to something else. */
|
||||
export interface RequirementLinkRow {
|
||||
requirement_id: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Everything needed to compute coverage and gaps for one traceability scope
|
||||
* (a mission, program, product line — whatever the caller scopes by).
|
||||
*/
|
||||
export interface TraceInput {
|
||||
requirements: RequirementRow[]
|
||||
/** requirement → verification junction rows. */
|
||||
requirementVerifications?: RequirementLinkRow[]
|
||||
/** requirement → risk junction rows. */
|
||||
requirementRisks?: RequirementLinkRow[]
|
||||
/** stakeholder need → requirement junction rows. */
|
||||
stakeholderNeedRequirements?: RequirementLinkRow[]
|
||||
/** Verification items carrying an FK back to a requirement. */
|
||||
verificationItems?: VerificationItemRow[]
|
||||
/** Links flagged as suspect and needing review. */
|
||||
suspectLinks?: RequirementLinkRow[]
|
||||
}
|
||||
|
||||
export interface Coverage {
|
||||
total_requirements: number
|
||||
traced_to_parent: number
|
||||
traced_to_test: number
|
||||
traced_to_risk: number
|
||||
verified: number
|
||||
with_stakeholder_need: number
|
||||
pct_parent: number
|
||||
pct_test: number
|
||||
pct_risk: number
|
||||
pct_verified: number
|
||||
pct_stakeholder: number
|
||||
}
|
||||
|
||||
export type GapType =
|
||||
| 'orphaned'
|
||||
| 'no_test'
|
||||
| 'no_risk'
|
||||
| 'suspect'
|
||||
| 'failed_verification'
|
||||
|
||||
export type GapSeverity = 'minor' | 'major' | 'critical'
|
||||
|
||||
export interface Gap {
|
||||
id: string
|
||||
entity_type: 'requirement' | 'verification'
|
||||
entity_id: string
|
||||
display_id: string
|
||||
title: string
|
||||
gap_type: GapType
|
||||
severity: GapSeverity
|
||||
description: string
|
||||
}
|
||||
|
||||
export type QualityGrade = 'A' | 'B' | 'C' | 'D' | 'F'
|
||||
|
||||
export interface QualityScore {
|
||||
score: number
|
||||
breakdown: {
|
||||
traced_pct: number
|
||||
verified_pct: number
|
||||
risk_coverage_pct: number
|
||||
parent_coverage_pct: number
|
||||
}
|
||||
grade: QualityGrade
|
||||
}
|
||||
|
||||
/**
|
||||
* Weights for the composite quality score. Must sum to 1.
|
||||
*
|
||||
* Extracted as a parameter rather than baked in: an accreditation regime that
|
||||
* cares more about verification than hierarchy should be able to say so
|
||||
* without forking the scorer.
|
||||
*/
|
||||
export interface QualityWeights {
|
||||
traced: number
|
||||
verified: number
|
||||
risk: number
|
||||
parent: number
|
||||
}
|
||||
|
||||
/** Grade cutoffs, inclusive lower bounds, highest first. */
|
||||
export interface GradeThresholds {
|
||||
A: number
|
||||
B: number
|
||||
C: number
|
||||
D: number
|
||||
}
|
||||
@@ -0,0 +1,191 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { computeCoverage, detectGaps, qualityScore, toReport, formatTrace } from '../src/index.js'
|
||||
import { overall, validateReport } from '@extant2000/evidence-record'
|
||||
import type { TraceInput } from '../src/index.js'
|
||||
|
||||
const req = (id: string, parent?: string | null) => ({
|
||||
id, parent_id: parent ?? null, req_number: `R-${id}`, title: `Requirement ${id}`,
|
||||
})
|
||||
|
||||
describe('computeCoverage', () => {
|
||||
it('returns an all-zero shape for no requirements', () => {
|
||||
const c = computeCoverage({ requirements: [] })
|
||||
expect(c.total_requirements).toBe(0)
|
||||
expect(c.pct_verified).toBe(0)
|
||||
})
|
||||
|
||||
it('counts a requirement traced by junction OR by verification FK, without double counting', () => {
|
||||
const input: TraceInput = {
|
||||
requirements: [req('a'), req('b')],
|
||||
requirementVerifications: [{ requirement_id: 'a' }],
|
||||
verificationItems: [{ id: 'v1', requirement_id: 'a', status: 'Passed' }],
|
||||
}
|
||||
// 'a' is linked by both routes but must count once.
|
||||
expect(computeCoverage(input).traced_to_test).toBe(1)
|
||||
})
|
||||
|
||||
it('treats a requirement with zero verifications as NOT verified', () => {
|
||||
// Guards the vacuous-every() trap: [].every(...) is true, which would
|
||||
// report a requirement with no evidence at all as fully verified.
|
||||
const input: TraceInput = { requirements: [req('a')], verificationItems: [] }
|
||||
expect(computeCoverage(input).verified).toBe(0)
|
||||
})
|
||||
|
||||
it('requires EVERY linked verification to pass or be waived', () => {
|
||||
const mixed: TraceInput = {
|
||||
requirements: [req('a')],
|
||||
verificationItems: [
|
||||
{ id: 'v1', requirement_id: 'a', status: 'Passed' },
|
||||
{ id: 'v2', requirement_id: 'a', status: 'Failed' },
|
||||
],
|
||||
}
|
||||
expect(computeCoverage(mixed).verified).toBe(0)
|
||||
|
||||
const allGood: TraceInput = {
|
||||
requirements: [req('a')],
|
||||
verificationItems: [
|
||||
{ id: 'v1', requirement_id: 'a', status: 'Passed' },
|
||||
{ id: 'v2', requirement_id: 'a', status: 'Waived' },
|
||||
],
|
||||
}
|
||||
expect(computeCoverage(allGood).verified).toBe(1)
|
||||
})
|
||||
|
||||
it('computes percentages against the requirement total', () => {
|
||||
const input: TraceInput = {
|
||||
requirements: [req('a', 'root'), req('b'), req('c'), req('d')],
|
||||
}
|
||||
expect(computeCoverage(input).pct_parent).toBe(25)
|
||||
})
|
||||
})
|
||||
|
||||
describe('detectGaps', () => {
|
||||
it('reports nothing for an empty requirement set', () => {
|
||||
expect(detectGaps({ requirements: [] })).toEqual({ gaps: [], total: 0 })
|
||||
})
|
||||
|
||||
it('flags an untested, unrisked, parentless requirement three ways', () => {
|
||||
const { gaps } = detectGaps({ requirements: [req('a')] })
|
||||
expect(gaps.map(g => g.gap_type).sort()).toEqual(['no_risk', 'no_test', 'orphaned'])
|
||||
})
|
||||
|
||||
it('rates a failed verification critical and everything else lower', () => {
|
||||
const { gaps } = detectGaps({
|
||||
requirements: [req('a', 'root')],
|
||||
requirementVerifications: [{ requirement_id: 'a' }],
|
||||
requirementRisks: [{ requirement_id: 'a' }],
|
||||
verificationItems: [{ id: 'v1', requirement_id: 'a', status: 'Failed', method: 'Test' }],
|
||||
})
|
||||
expect(gaps).toHaveLength(1)
|
||||
expect(gaps[0]!.gap_type).toBe('failed_verification')
|
||||
expect(gaps[0]!.severity).toBe('critical')
|
||||
})
|
||||
|
||||
it('flags suspect links', () => {
|
||||
const { gaps } = detectGaps({
|
||||
requirements: [req('a', 'root')],
|
||||
requirementVerifications: [{ requirement_id: 'a' }],
|
||||
requirementRisks: [{ requirement_id: 'a' }],
|
||||
suspectLinks: [{ requirement_id: 'a' }],
|
||||
})
|
||||
expect(gaps.map(g => g.gap_type)).toEqual(['suspect'])
|
||||
})
|
||||
|
||||
it('falls back to a short id when req_number is absent', () => {
|
||||
const { gaps } = detectGaps({ requirements: [{ id: 'abcdef1234567890' }] })
|
||||
expect(gaps[0]!.display_id).toBe('abcdef12')
|
||||
})
|
||||
})
|
||||
|
||||
describe('qualityScore', () => {
|
||||
it('applies the documented 0.3/0.3/0.2/0.2 weighting', () => {
|
||||
const s = qualityScore({ pct_test: 100, pct_verified: 100, pct_risk: 0, pct_parent: 0 })
|
||||
expect(s.score).toBe(60)
|
||||
expect(s.grade).toBe('C')
|
||||
})
|
||||
|
||||
it('grades the boundaries inclusively', () => {
|
||||
const at = (n: number) => qualityScore({
|
||||
pct_test: n, pct_verified: n, pct_risk: n, pct_parent: n,
|
||||
}).grade
|
||||
expect(at(90)).toBe('A')
|
||||
expect(at(75)).toBe('B')
|
||||
expect(at(60)).toBe('C')
|
||||
expect(at(40)).toBe('D')
|
||||
expect(at(39)).toBe('F')
|
||||
})
|
||||
|
||||
it('accepts custom weights for a different accreditation regime', () => {
|
||||
const s = qualityScore(
|
||||
{ pct_test: 0, pct_verified: 100, pct_risk: 0, pct_parent: 0 },
|
||||
{ traced: 0, verified: 1, risk: 0, parent: 0 },
|
||||
)
|
||||
expect(s.score).toBe(100)
|
||||
expect(s.grade).toBe('A')
|
||||
})
|
||||
|
||||
it('round-trips from computeCoverage output', () => {
|
||||
const coverage = computeCoverage({
|
||||
requirements: [req('a', 'root')],
|
||||
requirementVerifications: [{ requirement_id: 'a' }],
|
||||
requirementRisks: [{ requirement_id: 'a' }],
|
||||
verificationItems: [{ id: 'v1', requirement_id: 'a', status: 'Passed' }],
|
||||
})
|
||||
expect(qualityScore(coverage).score).toBe(100)
|
||||
})
|
||||
})
|
||||
|
||||
describe('report — the score has to lead back to the rows', () => {
|
||||
const input = {
|
||||
requirements: [
|
||||
{ id: 'r1-uuid-aaaa', req_number: 'SYS-001', title: 'Deorbit within 25 years', parent_id: 'p1' },
|
||||
{ id: 'r2-uuid-bbbb', req_number: 'SYS-002', title: 'Survive launch loads' },
|
||||
],
|
||||
verificationItems: [
|
||||
{ id: 'v1-uuid-cccc', requirement_id: 'r1-uuid-aaaa', status: 'Failed', method: 'Test' },
|
||||
],
|
||||
}
|
||||
|
||||
it('cites the requirement row behind every gap', () => {
|
||||
// The gap this closes: "quality 50/100, grade D" with no route back to
|
||||
// the requirements that produced it.
|
||||
const text = formatTrace(input, { requirementTable: 'mission_requirements', evidence: 'full' })
|
||||
expect(text).toContain('mission_requirements#r2-uuid-bbbb')
|
||||
expect(text).toContain('SYS-002')
|
||||
})
|
||||
|
||||
it('cites the verification row behind a failed verification', () => {
|
||||
const text = formatTrace(input, { verificationTable: 'mission_verification_items', evidence: 'full' })
|
||||
expect(text).toContain('mission_verification_items#v1-uuid-cccc')
|
||||
})
|
||||
|
||||
it('breaks the composite score into checkable measurements', () => {
|
||||
const r = toReport(input)
|
||||
const traced = r.findings.find(f => f.id === 'score.traced')!
|
||||
expect(traced.evidence[0]!.source).toMatchObject({ kind: 'measurement', samples: 2 })
|
||||
expect(formatTrace(input)).toContain('composite quality')
|
||||
})
|
||||
|
||||
it('treats an empty requirement set as not-assessed, not as a score of zero', () => {
|
||||
// computeCoverage correctly returns zeros, but 0% next to grade F reads
|
||||
// as "traced nothing" when the truth is "loaded nothing".
|
||||
const r = toReport({ requirements: [] })
|
||||
expect(overall(r)).toBe('not-assessed')
|
||||
const text = formatTrace({ requirements: [] })
|
||||
expect(text).toContain('NOT ASSESSED')
|
||||
expect(text).toContain('nothing was traced because nothing was read')
|
||||
expect(text).not.toContain('grade F')
|
||||
})
|
||||
|
||||
it('separates absent evidence from evidence of failure', () => {
|
||||
const r = toReport(input)
|
||||
const failed = r.findings.find(f => f.summary.includes('Failed'))!
|
||||
expect(failed.determination).toBe('fail')
|
||||
const noTest = r.findings.find(f => f.summary.includes('No verification item'))!
|
||||
expect(noTest.determination).toBe('not-assessed')
|
||||
})
|
||||
|
||||
it('every conclusion carries a citation', () => {
|
||||
expect(validateReport(toReport(input))).toEqual([])
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ES2022",
|
||||
"moduleResolution": "bundler",
|
||||
"declaration": true,
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"strict": true,
|
||||
"skipLibCheck": true
|
||||
},
|
||||
"include": ["src/**/*.ts"]
|
||||
}
|
||||
Reference in New Issue
Block a user