First public release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -0,0 +1,12 @@
|
||||
root = true
|
||||
|
||||
[*]
|
||||
charset = utf-8
|
||||
end_of_line = lf
|
||||
insert_final_newline = true
|
||||
trim_trailing_whitespace = true
|
||||
indent_style = space
|
||||
indent_size = 2
|
||||
|
||||
[*.md]
|
||||
trim_trailing_whitespace = false
|
||||
@@ -0,0 +1,9 @@
|
||||
node_modules/
|
||||
dist/
|
||||
*.log
|
||||
.DS_Store
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
.npmrc
|
||||
dist-cjs/
|
||||
@@ -0,0 +1,6 @@
|
||||
stages: [test]
|
||||
|
||||
test:
|
||||
stage: test
|
||||
image: node:22-alpine
|
||||
script: [npm ci, npm run build, npm test]
|
||||
@@ -0,0 +1,19 @@
|
||||
# Changelog
|
||||
|
||||
## 0.2.0 - 2026-09-28
|
||||
|
||||
First public release under MIT.
|
||||
|
||||
- `scoreDimension()`, `scoreProduct()` and `scoreSuite()` apply a
|
||||
five-dimension readiness rubric with hard caps and six bands.
|
||||
- The lowest open cap on a dimension wins; an unknown cap id throws.
|
||||
- A score above 70 with no evidence is held at 70 and reported as NOT
|
||||
ASSESSED.
|
||||
- The overall score is the weakest dimension; the average is reported
|
||||
alongside and labelled as such.
|
||||
- Suite-wide caps apply before the tier-weighted average.
|
||||
- A cap can cite the evidence that establishes it; an uncited cap is still
|
||||
applied and the report says so.
|
||||
- `productReport()`, `suiteReport()`, `formatProduct()` and `formatSuite()`
|
||||
produce an evidence-record report.
|
||||
- Worked example in `examples/cap-by-worst-defect.ts`.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Contributing
|
||||
|
||||
Issues and merge requests are welcome at
|
||||
https://gitlab.com/extant2000/sustain-score.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- A test that cannot fail proves nothing. If you fix a bug, add a test and
|
||||
check that it fails against the unfixed code before you submit.
|
||||
- Measure, don't assume. Two modules with the same line count can still be
|
||||
different programs.
|
||||
- Keep dependencies minimal. Every new runtime dependency needs a reason.
|
||||
- Explain why in comments, not what. The what is already in the code.
|
||||
- A breaking change needs a major version bump and a note in CHANGELOG.md.
|
||||
|
||||
## Before you open a merge request
|
||||
|
||||
Run both of these and make sure they pass:
|
||||
|
||||
npm run build
|
||||
npm test
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Extant 2000 LLC
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,104 @@
|
||||
# SustainScore
|
||||
|
||||
A production-readiness score that separates demo-ware from software that is
|
||||
ready to hand over, with the rubric's caps enforced in code.
|
||||
|
||||
SustainScore encodes a five-dimension readiness rubric (security, bugs,
|
||||
compliance, consistency, usability) with six bands from Prototype to Mature
|
||||
and a table of hard caps. You supply the scorer's raw judgement per dimension,
|
||||
the defects that are currently open, and the evidence. It returns scores that
|
||||
apply the rubric's anti-inflation rules mechanically.
|
||||
|
||||
## Why
|
||||
|
||||
A rubric kept as prose gets applied charitably. A scorer can write down, in
|
||||
the same row, that an open defect caps Security at 45 and still record 70.
|
||||
Nothing in a document stops that. Encoding the rubric does.
|
||||
|
||||
The worked example in `examples/cap-by-worst-defect.ts` scores a fictional
|
||||
product whose security review notes an exposed API key that was never
|
||||
rotated. Run it with `npx tsx examples/cap-by-worst-defect.ts`:
|
||||
|
||||
```
|
||||
Hand-scored vs rubric-enforced
|
||||
|
||||
security raw 70 -> 45 Beta <- CAPPED by sec.leaked-secret (45)
|
||||
bugs raw 68 -> 68 Launch-capable, with debt
|
||||
compliance raw 73 -> 73 Launch-capable, with debt
|
||||
consistency raw 68 -> 68 Launch-capable, with debt
|
||||
usability raw 68 -> 68 Launch-capable, with debt
|
||||
|
||||
hand-scored overall : ~69
|
||||
enforced overall : 45 (Beta)
|
||||
weakest : security
|
||||
average : 64.4 (reported, never the headline)
|
||||
delta : -24
|
||||
```
|
||||
|
||||
A 24-point overstatement and a band change, from one cap a human reader
|
||||
skipped.
|
||||
|
||||
## What it enforces
|
||||
|
||||
| Rule | Enforcement |
|
||||
|---|---|
|
||||
| Cap by the worst defect | The lowest open cap on a dimension wins. One serious hole is not averaged away by ten strengths. |
|
||||
| Evidence required above 70 | A score above 70 with no cited evidence is held at 70 and reported as NOT ASSESSED, not as a pass. Unverified is treated as absent. |
|
||||
| Overall is the lowest dimension | `overall` is the minimum. The average is reported alongside it and labelled so it cannot be read as the headline. |
|
||||
| Suite caps apply before averaging | A shared platform flaw hits every product. Averaging first would let healthy products dilute it. |
|
||||
|
||||
An unknown cap id throws. Dropping a mistyped cap without a word would
|
||||
inflate the score, which is the exact failure this library prevents. A cap
|
||||
assigned to the wrong dimension also throws, and so does a product with a
|
||||
missing dimension.
|
||||
|
||||
Caps can cite what establishes them: `openCaps` takes a bare id or
|
||||
`{ id, evidence }`. A cap that cites nothing is still applied, and the report
|
||||
says it cited nothing. An unproven cap is a smaller error than an unapplied
|
||||
one.
|
||||
|
||||
The full cap table is exported as `CAP_RULES`, the bands as `BANDS`.
|
||||
|
||||
## Usage
|
||||
|
||||
```ts
|
||||
import { scoreProduct, scoreSuite, formatProduct } from '@extant2000/sustain-score'
|
||||
|
||||
const result = scoreProduct({
|
||||
product: 'example-app',
|
||||
tier: 'A',
|
||||
dimensions: {
|
||||
security: { raw: 70, openCaps: ['sec.leaked-secret'], evidence: [{ ref: 'docs/security-review.md:14' }] },
|
||||
bugs: { raw: 68, evidence: [{ ref: 'test/api.test.ts' }] },
|
||||
compliance: { raw: 73, evidence: [{ ref: 'docs/compliance-checklist.md' }] },
|
||||
consistency: { raw: 68, evidence: [{ ref: 'shared/ui/' }] },
|
||||
usability: { raw: 68, evidence: [{ ref: 'reports/accessibility-scan.json' }] },
|
||||
},
|
||||
})
|
||||
|
||||
result.overall // 45, the lowest dimension
|
||||
result.weakest // 'security'
|
||||
result.band.name // 'Beta'
|
||||
|
||||
console.log(formatProduct(result))
|
||||
```
|
||||
|
||||
`scoreSuite(products, { suiteCaps })` rolls several products up with tier
|
||||
weighting (A = 2, B = 1.5, C = 1). `productReport()` and `suiteReport()`
|
||||
return a `CapabilityReport` from `@extant2000/evidence-record`, and
|
||||
`formatProduct()` and `formatSuite()` render one. Evidence can be the
|
||||
evidence-record shape or a plain `{ ref, note }`, where a `path:line` ref
|
||||
becomes a file citation.
|
||||
|
||||
To rank several scored products by what to fix next, see
|
||||
`@extant2000/portfolio-advisor`.
|
||||
|
||||
## Install
|
||||
|
||||
```
|
||||
npm install @extant2000/sustain-score
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT. Copyright (c) 2026 Extant 2000 LLC. See [LICENSE](LICENSE).
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
# Security policy
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Please report vulnerabilities privately by email to security@extant2000.com.
|
||||
Do not open a public issue or merge request for a security problem.
|
||||
|
||||
Include the affected version, a description of the issue, and steps or a test
|
||||
that reproduce it if you have them.
|
||||
|
||||
We aim to acknowledge every report within 5 business days.
|
||||
@@ -0,0 +1,66 @@
|
||||
/**
|
||||
* Worked example: cap by the worst defect.
|
||||
*
|
||||
* A hand-scored assessment of a fictional product. The scorer rated Security
|
||||
* at 70 and noted, in the same row, an open defect that the rubric caps at 45
|
||||
* (a secret that was exposed and never rotated). A human reading prose can be
|
||||
* charitable and skip the cap. The library cannot: the lowest open cap wins,
|
||||
* and the product's overall score is its weakest dimension, not the average.
|
||||
*
|
||||
* Run: npx tsx examples/cap-by-worst-defect.ts
|
||||
*/
|
||||
import { scoreProduct, type DimensionAssessment, type DimensionKey } from '../src/index.js'
|
||||
|
||||
/** The overall score a hand-scored summary might report: roughly the average. */
|
||||
const HAND_SCORED_OVERALL = 69
|
||||
|
||||
const dimensions: Record<DimensionKey, DimensionAssessment> = {
|
||||
security: {
|
||||
raw: 70,
|
||||
// The scorer named this cap and still wrote 70.
|
||||
openCaps: [{
|
||||
id: 'sec.leaked-secret',
|
||||
evidence: [{ ref: 'docs/security-review.md:14', note: 'exposed API key not yet rotated' }],
|
||||
}],
|
||||
evidence: [{ ref: 'docs/security-review.md' }],
|
||||
bindingConstraint: 'exposed API key not yet rotated',
|
||||
nextAction: 'rotate the key and redeploy',
|
||||
},
|
||||
bugs: {
|
||||
raw: 68,
|
||||
evidence: [{ ref: 'test/api.test.ts' }],
|
||||
bindingConstraint: 'API handlers lack behaviour tests',
|
||||
},
|
||||
compliance: {
|
||||
raw: 73,
|
||||
evidence: [{ ref: 'docs/compliance-checklist.md' }],
|
||||
},
|
||||
consistency: {
|
||||
raw: 68,
|
||||
evidence: [{ ref: 'shared/ui/' }],
|
||||
},
|
||||
usability: {
|
||||
raw: 68,
|
||||
evidence: [{ ref: 'reports/accessibility-scan.json' }],
|
||||
bindingConstraint: 'accessibility findings not yet triaged',
|
||||
},
|
||||
}
|
||||
|
||||
const result = scoreProduct({ product: 'example-app', tier: 'A', dimensions })
|
||||
|
||||
console.log('Hand-scored vs rubric-enforced\n')
|
||||
for (const d of result.dimensions) {
|
||||
const flag = d.boundBy ? ` <- CAPPED by ${d.boundBy.id} (${d.boundBy.max})` : ''
|
||||
console.log(
|
||||
` ${d.dimension.padEnd(12)} raw ${String(d.raw).padStart(3)} -> ${String(d.score).padStart(3)} ${d.band.name}${flag}`,
|
||||
)
|
||||
}
|
||||
console.log(`\n hand-scored overall : ~${HAND_SCORED_OVERALL}`)
|
||||
console.log(` enforced overall : ${result.overall} (${result.band.name})`)
|
||||
console.log(` weakest : ${result.weakest}`)
|
||||
console.log(` average : ${result.average} (reported, never the headline)`)
|
||||
console.log(` delta : ${result.overall - HAND_SCORED_OVERALL}`)
|
||||
|
||||
if (result.overall < HAND_SCORED_OVERALL) {
|
||||
console.log('\n The hand-scored number is higher than the rubric permits.')
|
||||
}
|
||||
Generated
+1504
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"name": "@extant2000/sustain-score",
|
||||
"version": "0.2.0",
|
||||
"private": false,
|
||||
"description": "Maturity score separating demo-ware from transition-ready.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
"files": [
|
||||
"dist",
|
||||
"src",
|
||||
"examples",
|
||||
"README.md",
|
||||
"LICENSE"
|
||||
],
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://gitlab.com/extant2000/sustain-score.git"
|
||||
},
|
||||
"types": "dist/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./dist/index.d.ts",
|
||||
"import": "./dist/index.js"
|
||||
}
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"test": "vitest run",
|
||||
"prepublishOnly": "npm run build"
|
||||
},
|
||||
"devDependencies": {
|
||||
"typescript": "^5.6.0",
|
||||
"vitest": "^2.1.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@extant2000/evidence-record": "^0.1.2"
|
||||
},
|
||||
"author": "Extant 2000 LLC",
|
||||
"homepage": "https://gitlab.com/extant2000/sustain-score",
|
||||
"bugs": {
|
||||
"url": "https://gitlab.com/extant2000/sustain-score/-/issues"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
export * from './rubric.js'
|
||||
export * from './score.js'
|
||||
export * from './report.js'
|
||||
@@ -0,0 +1,97 @@
|
||||
import {
|
||||
type CapabilityReport,
|
||||
type Finding,
|
||||
type FormatOptions,
|
||||
evidence as ev,
|
||||
formatReport as renderReport,
|
||||
} from '@extant2000/evidence-record'
|
||||
import type { DimensionScore, ProductScore, SuiteScore } from './score.js'
|
||||
import { EVIDENCE_THRESHOLD } from './rubric.js'
|
||||
|
||||
/**
|
||||
* One finding per dimension, carrying what set the number.
|
||||
*
|
||||
* The previous output said `CAPPED sec.leaked-secret` and stopped there. A
|
||||
* reader could not tell whether the cap was still warranted, which is how a
|
||||
* cap ends up recorded but not applied. A cap now cites what establishes it,
|
||||
* and a cap that cites nothing says so.
|
||||
*/
|
||||
function findingFor(product: string, d: DimensionScore): Finding {
|
||||
const id = `${product}.${d.dimension}`
|
||||
|
||||
if (d.boundBy) {
|
||||
return {
|
||||
id,
|
||||
summary: `${d.dimension} capped at ${d.score} (assessed ${d.raw}) — ${d.boundBy.description}`,
|
||||
determination: 'fail',
|
||||
severity: d.score < 50 ? 'critical' : 'high',
|
||||
detail: d.nextAction ?? d.bindingConstraint,
|
||||
evidence: d.boundBy.evidence.length > 0
|
||||
? d.boundBy.evidence
|
||||
// Loud rather than silent: an uncited cap is still applied, because
|
||||
// the cap being wrong is a better failure than the cap being skipped.
|
||||
: [ev.document(`Cap rule ${d.boundBy.id}`, undefined, undefined,
|
||||
'No evidence was recorded for this cap. It is applied anyway — an unproven cap is a smaller error than an unapplied one.')],
|
||||
}
|
||||
}
|
||||
|
||||
if (d.evidenceWithheld) {
|
||||
return {
|
||||
id,
|
||||
summary: `${d.dimension} held at ${EVIDENCE_THRESHOLD} (assessed ${d.raw}) — no evidence was cited`,
|
||||
determination: 'not-assessed',
|
||||
severity: 'high',
|
||||
detail: `A score above ${EVIDENCE_THRESHOLD} requires evidence. Unverified is treated as absent, so the assessed value was not granted.`,
|
||||
evidence: [],
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
id,
|
||||
summary: `${d.dimension} scored ${d.score} (${d.band.name})`,
|
||||
determination: 'pass',
|
||||
severity: 'info',
|
||||
detail: d.bindingConstraint,
|
||||
evidence: d.evidence.length > 0
|
||||
? d.evidence
|
||||
: [ev.document(`${d.dimension} assessment`, undefined, undefined,
|
||||
`At or below ${EVIDENCE_THRESHOLD}, evidence is not required by the rubric.`)],
|
||||
}
|
||||
}
|
||||
|
||||
/** Convert one product's score into the portfolio-standard report shape. */
|
||||
export function productReport(p: ProductScore): CapabilityReport {
|
||||
return {
|
||||
capability: 'SustainScore',
|
||||
scope: `${p.product} (${p.tier}) — ${p.dimensions.length} dimensions`,
|
||||
examined: p.dimensions.length,
|
||||
findings: p.dimensions.map(d => findingFor(p.product, d)),
|
||||
notes: [
|
||||
`overall ${p.overall} (${p.band.name}), set by the weakest dimension: ${p.weakest}`,
|
||||
// Printed second and labelled, so it cannot be mistaken for the headline.
|
||||
`average ${p.average} — reported because the rubric asks for it, never as the headline`,
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
/** Convert a suite roll-up into the portfolio-standard report shape. */
|
||||
export function suiteReport(s: SuiteScore): CapabilityReport {
|
||||
return {
|
||||
capability: 'SustainScore',
|
||||
scope: `${s.products.length} products, weighted by tier`,
|
||||
examined: s.products.reduce((n, p) => n + p.dimensions.length, 0),
|
||||
findings: s.products.flatMap(p => p.dimensions.map(d => findingFor(p.product, d))),
|
||||
notes: [
|
||||
`suite overall ${s.overall} (${s.band.name}), set by the weakest dimension: ${s.weakest}`,
|
||||
`average ${s.average} — suite-wide caps are applied BEFORE averaging, so healthy products cannot dilute a shared flaw`,
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
export function formatProduct(p: ProductScore, opts: FormatOptions = {}): string {
|
||||
return renderReport(productReport(p), opts)
|
||||
}
|
||||
|
||||
export function formatSuite(s: SuiteScore, opts: FormatOptions = {}): string {
|
||||
return renderReport(suiteReport(s), opts)
|
||||
}
|
||||
+118
@@ -0,0 +1,118 @@
|
||||
/**
|
||||
* Production-Readiness Rubric v1.0 — encoded.
|
||||
*
|
||||
* A rubric kept as prose and applied by hand gets applied charitably. A scorer
|
||||
* can write down a defect that caps a dimension at 45 and still record 70 in
|
||||
* the same row. Encoding the rubric is not a formatting exercise: a charitable
|
||||
* scorer is the failure mode this module removes.
|
||||
*/
|
||||
|
||||
export type BandName =
|
||||
| 'Prototype' | 'Alpha' | 'Beta'
|
||||
| 'Launch-capable, with debt' | 'Production' | 'Mature'
|
||||
|
||||
export interface Band {
|
||||
name: BandName
|
||||
min: number
|
||||
max: number
|
||||
meaning: string
|
||||
}
|
||||
|
||||
/** Identical for every dimension. Ordered low → high. */
|
||||
export const BANDS: readonly Band[] = Object.freeze([
|
||||
{ name: 'Prototype', min: 0, max: 19,
|
||||
meaning: 'Unsafe or broken in ordinary use. Would harm a user or the business if strangers used it today.' },
|
||||
{ name: 'Alpha', min: 20, max: 39,
|
||||
meaning: 'Works on the happy path. Known defects that actively harm users or expose the company. Not sellable.' },
|
||||
{ name: 'Beta', min: 40, max: 59,
|
||||
meaning: 'Genuinely usable, but ≥1 gap that blocks charging money or inviting scrutiny.' },
|
||||
{ name: 'Launch-capable, with debt', min: 60, max: 74,
|
||||
meaning: 'No blocking defect. Real, catalogued debt a competitor or auditor would notice.' },
|
||||
{ name: 'Production', min: 75, max: 89,
|
||||
meaning: 'No known blockers. Controls exist AND are verified working. Failures are monitored and recoverable.' },
|
||||
{ name: 'Mature', min: 90, max: 100,
|
||||
meaning: 'Independently verified (external audit, pen test, or a real incident survived). Evidence, not assertion.' },
|
||||
])
|
||||
|
||||
export function bandFor(score: number): Band {
|
||||
const s = Math.max(0, Math.min(100, Math.round(score)))
|
||||
// Walk high → low so the first match is the highest band the score reaches.
|
||||
for (let i = BANDS.length - 1; i >= 0; i--) {
|
||||
const b = BANDS[i]!
|
||||
if (s >= b.min) return b
|
||||
}
|
||||
return BANDS[0]!
|
||||
}
|
||||
|
||||
export type DimensionKey =
|
||||
| 'security' | 'bugs' | 'compliance' | 'consistency' | 'usability'
|
||||
|
||||
export const DIMENSIONS: readonly DimensionKey[] = Object.freeze([
|
||||
'security', 'bugs', 'compliance', 'consistency', 'usability',
|
||||
])
|
||||
|
||||
export interface CapRule {
|
||||
/** Stable id so a scorecard can reference which cap bound a score. */
|
||||
id: string
|
||||
dimension: DimensionKey
|
||||
/** Ceiling this defect imposes on its dimension. */
|
||||
max: number
|
||||
description: string
|
||||
}
|
||||
|
||||
/** Every hard cap in rubric v1.0, verbatim. */
|
||||
export const CAP_RULES: readonly CapRule[] = Object.freeze([
|
||||
// Dimension 1 — SECURITY
|
||||
{ id: 'sec.cross-tenant', dimension: 'security', max: 30,
|
||||
description: 'Any live cross-tenant data exposure' },
|
||||
{ id: 'sec.leaked-secret', dimension: 'security', max: 45,
|
||||
description: 'Any secret in the repo, in logs, or known-leaked and unrotated' },
|
||||
{ id: 'sec.no-restore', dimension: 'security', max: 70,
|
||||
description: 'No verified backup restore' },
|
||||
{ id: 'sec.no-dep-monitoring', dimension: 'security', max: 75,
|
||||
description: 'No dependency-vulnerability monitoring' },
|
||||
// Dimension 2 — BUGS
|
||||
{ id: 'bugs.data-loss', dimension: 'bugs', max: 35,
|
||||
description: 'Any known defect causing silent data loss or corruption' },
|
||||
{ id: 'bugs.takes-money-no-deliver', dimension: 'bugs', max: 40,
|
||||
description: "Any user-facing flow that takes money and doesn't deliver" },
|
||||
{ id: 'bugs.no-error-monitoring', dimension: 'bugs', max: 60,
|
||||
description: 'No error monitoring in production' },
|
||||
{ id: 'bugs.smoke-only', dimension: 'bugs', max: 70,
|
||||
description: 'Smoke-tests-only (tests assert files exist, not behavior)' },
|
||||
// Dimension 3 — COMPLIANCE
|
||||
{ id: 'comp.false-claim', dimension: 'compliance', max: 50,
|
||||
description: 'Any live false or unsubstantiated public claim' },
|
||||
{ id: 'comp.missing-registration', dimension: 'compliance', max: 60,
|
||||
description: 'Any missing registration that is legally required NOW' },
|
||||
{ id: 'comp.legal-contradicts-code', dimension: 'compliance', max: 65,
|
||||
description: 'Legal pages that contradict code behavior' },
|
||||
// Dimension 4 — CONSISTENCY
|
||||
{ id: 'cons.per-repo-fix', dimension: 'consistency', max: 65,
|
||||
description: 'Same critical bug class needing a separate fix per repo' },
|
||||
{ id: 'cons.docs-contradict', dimension: 'consistency', max: 70,
|
||||
description: 'Docs/CLAUDE.md materially contradicting the code' },
|
||||
// Dimension 5 — USABILITY
|
||||
{ id: 'use.keyboard-inoperable', dimension: 'usability', max: 55,
|
||||
description: 'Keyboard-inoperable or unlabeled primary flow' },
|
||||
{ id: 'use.contrast', dimension: 'usability', max: 65,
|
||||
description: 'Body-copy contrast below AA' },
|
||||
{ id: 'use.no-empty-error-states', dimension: 'usability', max: 70,
|
||||
description: 'No empty/error states on primary screens' },
|
||||
])
|
||||
|
||||
const CAPS_BY_ID = new Map(CAP_RULES.map(c => [c.id, c]))
|
||||
|
||||
export function capRule(id: string): CapRule | undefined {
|
||||
return CAPS_BY_ID.get(id)
|
||||
}
|
||||
|
||||
/**
|
||||
* Score at or above which evidence is mandatory (anti-inflation rule 5:
|
||||
* "Evidence required above 70").
|
||||
*/
|
||||
export const EVIDENCE_THRESHOLD = 70
|
||||
|
||||
/** Revenue/exposure weights for the suite average. */
|
||||
export const TIER_WEIGHTS = Object.freeze({ A: 2, B: 1.5, C: 1 })
|
||||
export type Tier = keyof typeof TIER_WEIGHTS
|
||||
+267
@@ -0,0 +1,267 @@
|
||||
import {
|
||||
DIMENSIONS,
|
||||
EVIDENCE_THRESHOLD,
|
||||
TIER_WEIGHTS,
|
||||
bandFor,
|
||||
capRule,
|
||||
type Band,
|
||||
type DimensionKey,
|
||||
type Tier,
|
||||
} from './rubric.js'
|
||||
|
||||
import { type Evidence as SourcedEvidence, evidence as ev } from '@extant2000/evidence-record'
|
||||
|
||||
/**
|
||||
* The original shape: a free-text reference.
|
||||
*
|
||||
* @deprecated Pass `@extant2000/evidence-record` evidence instead — `ref` is a
|
||||
* string a reader has to trust, and this library's own rubric penalises
|
||||
* unverified claims under "Unverified = absent". Still accepted, and
|
||||
* converted: `path:42` becomes a file citation, anything else a document one.
|
||||
*/
|
||||
export interface Evidence {
|
||||
/** file:line, a passing test name, a live check, or a production query. */
|
||||
ref: string
|
||||
note?: string
|
||||
}
|
||||
|
||||
export type EvidenceLike = Evidence | SourcedEvidence
|
||||
|
||||
/** Normalise either shape into a citation the shared renderer can print. */
|
||||
export function toSourcedEvidence(e: EvidenceLike): SourcedEvidence {
|
||||
if ('source' in e) return e
|
||||
const m = /^(.+):(\d+)$/.exec(e.ref)
|
||||
return m
|
||||
? ev.file(m[1]!, Number(m[2]), undefined, e.note)
|
||||
: ev.document(e.ref, undefined, undefined, e.note)
|
||||
}
|
||||
|
||||
/**
|
||||
* A cap that is currently open, and what establishes that.
|
||||
*
|
||||
* The bare-string form is what the first version accepted, and it is exactly
|
||||
* the gap this release closes: the output said `CAPPED sec.leaked-secret`
|
||||
* without citing anything, so a reader had no way to check whether the cap
|
||||
* was still warranted, which is how a cap goes unapplied in the first place.
|
||||
*/
|
||||
export type CapClaim = string | { id: string, evidence?: EvidenceLike[] }
|
||||
|
||||
export function capId(c: CapClaim): string {
|
||||
return typeof c === 'string' ? c : c.id
|
||||
}
|
||||
|
||||
function capEvidence(c: CapClaim): SourcedEvidence[] {
|
||||
return typeof c === 'string' ? [] : (c.evidence ?? []).map(toSourcedEvidence)
|
||||
}
|
||||
|
||||
export interface DimensionAssessment {
|
||||
/** The scorer's judgement BEFORE caps are applied. */
|
||||
raw: number
|
||||
/** Cap rules whose defect is currently open, ideally with what proves it. */
|
||||
openCaps?: CapClaim[]
|
||||
/** Required to hold a final score above EVIDENCE_THRESHOLD. */
|
||||
evidence?: EvidenceLike[]
|
||||
/** What is holding the score down. Free text, carried through. */
|
||||
bindingConstraint?: string
|
||||
/** Single highest-leverage action to raise it. */
|
||||
nextAction?: string
|
||||
}
|
||||
|
||||
export interface DimensionScore {
|
||||
dimension: DimensionKey
|
||||
raw: number
|
||||
score: number
|
||||
band: Band
|
||||
/**
|
||||
* The cap that actually bound the score, if any — with whatever established
|
||||
* it, so "CAPPED sec.leaked-secret" is checkable rather than asserted.
|
||||
*/
|
||||
boundBy?: { id: string, max: number, description: string, evidence: SourcedEvidence[] }
|
||||
/** Citations supporting the raw judgement, normalised. */
|
||||
evidence: SourcedEvidence[]
|
||||
/** True when the score was reduced to EVIDENCE_THRESHOLD for lack of evidence. */
|
||||
evidenceWithheld: boolean
|
||||
bindingConstraint?: string
|
||||
nextAction?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Score one dimension, applying the anti-inflation rules mechanically.
|
||||
*
|
||||
* Rule 3 — cap by the WORST defect: the lowest open cap wins. "One serious hole
|
||||
* is not averaged away by ten strengths."
|
||||
*
|
||||
* Rule 5 — evidence required above 70: a score that cannot cite evidence is
|
||||
* reduced to the threshold rather than rejected, so an un-evidenced assessment
|
||||
* still produces a usable (and honest) number.
|
||||
*
|
||||
* These two are the whole point. Applied by hand, rule 3 is the one that gets
|
||||
* skipped: a scorer notes an open defect that caps Security at 45 and still
|
||||
* records 70.
|
||||
*/
|
||||
export function scoreDimension(
|
||||
dimension: DimensionKey,
|
||||
input: DimensionAssessment,
|
||||
): DimensionScore {
|
||||
const raw = clamp(input.raw)
|
||||
let score = raw
|
||||
let boundBy: DimensionScore['boundBy']
|
||||
|
||||
for (const claim of input.openCaps ?? []) {
|
||||
const id = capId(claim)
|
||||
const rule = capRule(id)
|
||||
if (!rule) throw new Error(`Unknown cap rule: ${id}`)
|
||||
if (rule.dimension !== dimension) {
|
||||
throw new Error(`Cap ${id} belongs to ${rule.dimension}, not ${dimension}`)
|
||||
}
|
||||
if (rule.max < score) {
|
||||
score = rule.max
|
||||
boundBy = {
|
||||
id: rule.id,
|
||||
max: rule.max,
|
||||
description: rule.description,
|
||||
evidence: capEvidence(claim),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const evidence = (input.evidence ?? []).map(toSourcedEvidence)
|
||||
const hasEvidence = evidence.length > 0
|
||||
let evidenceWithheld = false
|
||||
if (score > EVIDENCE_THRESHOLD && !hasEvidence) {
|
||||
score = EVIDENCE_THRESHOLD
|
||||
evidenceWithheld = true
|
||||
}
|
||||
|
||||
return {
|
||||
dimension,
|
||||
raw,
|
||||
score,
|
||||
band: bandFor(score),
|
||||
boundBy,
|
||||
evidence,
|
||||
evidenceWithheld,
|
||||
bindingConstraint: input.bindingConstraint,
|
||||
nextAction: input.nextAction,
|
||||
}
|
||||
}
|
||||
|
||||
export interface ProductAssessment {
|
||||
product: string
|
||||
tier: Tier
|
||||
dimensions: Record<DimensionKey, DimensionAssessment>
|
||||
}
|
||||
|
||||
export interface ProductScore {
|
||||
product: string
|
||||
tier: Tier
|
||||
weight: number
|
||||
dimensions: DimensionScore[]
|
||||
/** The headline number: the LOWEST dimension, not the average. */
|
||||
overall: number
|
||||
/** Reported alongside, never instead of, `overall`. */
|
||||
average: number
|
||||
band: Band
|
||||
/** Which dimension set the overall score. */
|
||||
weakest: DimensionKey
|
||||
}
|
||||
|
||||
/**
|
||||
* Score one product.
|
||||
*
|
||||
* The overall score is the MINIMUM across dimensions — "a product is only as
|
||||
* shippable as its weakest dimension." The average is reported too, because the
|
||||
* rubric says to, but it is deliberately not the headline: averaging is how a
|
||||
* single disqualifying hole disappears behind four healthy numbers.
|
||||
*/
|
||||
export function scoreProduct(input: ProductAssessment): ProductScore {
|
||||
const dimensions = DIMENSIONS.map(d => {
|
||||
const a = input.dimensions[d]
|
||||
if (!a) throw new Error(`Missing assessment for dimension: ${d}`)
|
||||
return scoreDimension(d, a)
|
||||
})
|
||||
|
||||
let weakest = dimensions[0]!
|
||||
for (const d of dimensions) if (d.score < weakest.score) weakest = d
|
||||
|
||||
const overall = weakest.score
|
||||
const average = round(dimensions.reduce((s, d) => s + d.score, 0) / dimensions.length)
|
||||
|
||||
return {
|
||||
product: input.product,
|
||||
tier: input.tier,
|
||||
weight: TIER_WEIGHTS[input.tier],
|
||||
dimensions,
|
||||
overall,
|
||||
average,
|
||||
band: bandFor(overall),
|
||||
weakest: weakest.dimension,
|
||||
}
|
||||
}
|
||||
|
||||
export interface SuiteOptions {
|
||||
/**
|
||||
* Cap ids that apply suite-wide (a shared platform flaw hits everything).
|
||||
* Applied BEFORE averaging, per the rubric's scoring procedure — averaging
|
||||
* first would let healthy products dilute a flaw that affects them all.
|
||||
*/
|
||||
suiteCaps?: CapClaim[]
|
||||
}
|
||||
|
||||
export interface SuiteScore {
|
||||
products: ProductScore[]
|
||||
/** Weighted average per dimension, suite caps applied first. */
|
||||
dimensions: Record<DimensionKey, number>
|
||||
/** Lowest suite dimension score. The headline. */
|
||||
overall: number
|
||||
average: number
|
||||
band: Band
|
||||
weakest: DimensionKey
|
||||
}
|
||||
|
||||
export function scoreSuite(
|
||||
inputs: ProductAssessment[],
|
||||
options: SuiteOptions = {},
|
||||
): SuiteScore {
|
||||
if (inputs.length === 0) throw new Error('scoreSuite requires at least one product')
|
||||
const products = inputs.map(scoreProduct)
|
||||
|
||||
const dimensions = {} as Record<DimensionKey, number>
|
||||
for (const d of DIMENSIONS) {
|
||||
let totalWeight = 0
|
||||
let acc = 0
|
||||
for (const p of products) {
|
||||
const ds = p.dimensions.find(x => x.dimension === d)!
|
||||
// Suite-wide cap applied to each product's score BEFORE it enters the
|
||||
// weighted mean.
|
||||
let s = ds.score
|
||||
for (const claim of options.suiteCaps ?? []) {
|
||||
const id = capId(claim)
|
||||
const rule = capRule(id)
|
||||
if (rule && rule.dimension === d && rule.max < s) s = rule.max
|
||||
}
|
||||
acc += s * p.weight
|
||||
totalWeight += p.weight
|
||||
}
|
||||
dimensions[d] = round(acc / totalWeight)
|
||||
}
|
||||
|
||||
let weakest: DimensionKey = DIMENSIONS[0]!
|
||||
for (const d of DIMENSIONS) if (dimensions[d] < dimensions[weakest]) weakest = d
|
||||
|
||||
const overall = dimensions[weakest]
|
||||
const average = round(
|
||||
DIMENSIONS.reduce((s, d) => s + dimensions[d], 0) / DIMENSIONS.length,
|
||||
)
|
||||
|
||||
return { products, dimensions, overall, average, band: bandFor(overall), weakest }
|
||||
}
|
||||
|
||||
function clamp(n: number): number {
|
||||
if (!Number.isFinite(n)) throw new Error(`Score must be a finite number, got ${n}`)
|
||||
return Math.max(0, Math.min(100, Math.round(n)))
|
||||
}
|
||||
|
||||
function round(n: number): number {
|
||||
return Math.round(n * 10) / 10
|
||||
}
|
||||
@@ -0,0 +1,289 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import {
|
||||
BANDS,
|
||||
CAP_RULES,
|
||||
DIMENSIONS,
|
||||
bandFor,
|
||||
scoreDimension,
|
||||
scoreProduct,
|
||||
scoreSuite,
|
||||
type DimensionAssessment,
|
||||
type DimensionKey,
|
||||
type ProductAssessment,
|
||||
formatProduct,
|
||||
productReport,
|
||||
} from '../src/index.js'
|
||||
import { validateReport } from '@extant2000/evidence-record'
|
||||
|
||||
const ev = [{ ref: 'test/foo.test.ts:12' }]
|
||||
|
||||
/** A product scoring `n` on every dimension, with evidence. */
|
||||
function flat(product: string, n: number, tier: 'A' | 'B' | 'C' = 'B'): ProductAssessment {
|
||||
const dimensions = {} as Record<DimensionKey, DimensionAssessment>
|
||||
for (const d of DIMENSIONS) dimensions[d] = { raw: n, evidence: ev }
|
||||
return { product, tier, dimensions }
|
||||
}
|
||||
|
||||
/** A healthy product with one dimension replaced, for isolating its output. */
|
||||
function assessmentWith(dim: DimensionKey, a: DimensionAssessment): ProductAssessment {
|
||||
const p = flat('app', 60)
|
||||
p.dimensions[dim] = a
|
||||
return p
|
||||
}
|
||||
|
||||
describe('bandFor', () => {
|
||||
it('maps the documented boundaries', () => {
|
||||
expect(bandFor(0).name).toBe('Prototype')
|
||||
expect(bandFor(19).name).toBe('Prototype')
|
||||
expect(bandFor(20).name).toBe('Alpha')
|
||||
expect(bandFor(40).name).toBe('Beta')
|
||||
expect(bandFor(60).name).toBe('Launch-capable, with debt')
|
||||
expect(bandFor(75).name).toBe('Production')
|
||||
expect(bandFor(90).name).toBe('Mature')
|
||||
expect(bandFor(100).name).toBe('Mature')
|
||||
})
|
||||
|
||||
it('covers 0-100 with no gaps', () => {
|
||||
for (let s = 0; s <= 100; s++) expect(bandFor(s)).toBeDefined()
|
||||
})
|
||||
|
||||
it('has contiguous non-overlapping bands', () => {
|
||||
for (let i = 1; i < BANDS.length; i++) {
|
||||
expect(BANDS[i]!.min).toBe(BANDS[i - 1]!.max + 1)
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('scoreDimension — anti-inflation rule 3 (cap by worst defect)', () => {
|
||||
it('CATCHES THE CHARITABLE SCORE: an unrotated leaked secret caps Security at 45', () => {
|
||||
// A hand scorer can note that a known-leaked, unrotated secret caps the
|
||||
// dimension at 45 and still record ~70. Charity is not available here.
|
||||
const r = scoreDimension('security', {
|
||||
raw: 70,
|
||||
openCaps: ['sec.leaked-secret'],
|
||||
evidence: ev,
|
||||
})
|
||||
expect(r.raw).toBe(70)
|
||||
expect(r.score).toBe(45)
|
||||
expect(r.band.name).toBe('Beta')
|
||||
expect(r.boundBy?.id).toBe('sec.leaked-secret')
|
||||
})
|
||||
|
||||
it('applies the LOWEST open cap, not the first or last', () => {
|
||||
const r = scoreDimension('security', {
|
||||
raw: 90,
|
||||
openCaps: ['sec.no-dep-monitoring', 'sec.cross-tenant', 'sec.no-restore'],
|
||||
evidence: ev,
|
||||
})
|
||||
expect(r.score).toBe(30)
|
||||
expect(r.boundBy?.id).toBe('sec.cross-tenant')
|
||||
})
|
||||
|
||||
it('leaves a score below the cap untouched', () => {
|
||||
const r = scoreDimension('security', {
|
||||
raw: 25, openCaps: ['sec.leaked-secret'], evidence: ev,
|
||||
})
|
||||
expect(r.score).toBe(25)
|
||||
expect(r.boundBy).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects a cap belonging to another dimension', () => {
|
||||
expect(() => scoreDimension('bugs', { raw: 80, openCaps: ['sec.leaked-secret'] }))
|
||||
.toThrow(/belongs to security/)
|
||||
})
|
||||
|
||||
it('rejects an unknown cap id rather than ignoring it', () => {
|
||||
// Silently ignoring a typo'd cap would inflate the score — the exact
|
||||
// failure mode this library exists to prevent.
|
||||
expect(() => scoreDimension('security', { raw: 80, openCaps: ['sec.tpyo'] }))
|
||||
.toThrow(/Unknown cap rule/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('scoreDimension — anti-inflation rule 5 (evidence above 70)', () => {
|
||||
it('withholds an un-evidenced score above the threshold', () => {
|
||||
const r = scoreDimension('bugs', { raw: 85 })
|
||||
expect(r.score).toBe(70)
|
||||
expect(r.evidenceWithheld).toBe(true)
|
||||
})
|
||||
|
||||
it('allows an evidenced score above the threshold', () => {
|
||||
const r = scoreDimension('bugs', { raw: 85, evidence: ev })
|
||||
expect(r.score).toBe(85)
|
||||
expect(r.evidenceWithheld).toBe(false)
|
||||
})
|
||||
|
||||
it('does not touch an un-evidenced score at or below the threshold', () => {
|
||||
const r = scoreDimension('bugs', { raw: 70 })
|
||||
expect(r.score).toBe(70)
|
||||
expect(r.evidenceWithheld).toBe(false)
|
||||
})
|
||||
|
||||
it('an empty evidence array is not evidence', () => {
|
||||
expect(scoreDimension('bugs', { raw: 85, evidence: [] }).score).toBe(70)
|
||||
})
|
||||
})
|
||||
|
||||
describe('scoreProduct', () => {
|
||||
it('reports the LOWEST dimension as overall, not the average', () => {
|
||||
const a = flat('app', 80)
|
||||
a.dimensions.security = { raw: 40, evidence: ev }
|
||||
const r = scoreProduct(a)
|
||||
expect(r.overall).toBe(40)
|
||||
expect(r.weakest).toBe('security')
|
||||
expect(r.average).toBe(72) // (40+80*4)/5
|
||||
expect(r.band.name).toBe('Beta')
|
||||
})
|
||||
|
||||
it('a single capped dimension pins the whole product', () => {
|
||||
// Four strong dimensions must not average away one disqualifying hole.
|
||||
const a = flat('api', 88)
|
||||
a.dimensions.security = { raw: 88, openCaps: ['sec.cross-tenant'], evidence: ev }
|
||||
const r = scoreProduct(a)
|
||||
expect(r.overall).toBe(30)
|
||||
expect(r.band.name).toBe('Alpha')
|
||||
})
|
||||
|
||||
it('assigns the tier weight', () => {
|
||||
expect(scoreProduct(flat('x', 80, 'A')).weight).toBe(2)
|
||||
expect(scoreProduct(flat('x', 80, 'B')).weight).toBe(1.5)
|
||||
expect(scoreProduct(flat('x', 80, 'C')).weight).toBe(1)
|
||||
})
|
||||
|
||||
it('throws on a missing dimension rather than scoring a partial product', () => {
|
||||
const a = flat('x', 80)
|
||||
delete (a.dimensions as Partial<Record<DimensionKey, DimensionAssessment>>).usability
|
||||
expect(() => scoreProduct(a)).toThrow(/Missing assessment/)
|
||||
})
|
||||
|
||||
it('carries the binding constraint and next action through', () => {
|
||||
const a = flat('x', 80)
|
||||
a.dimensions.bugs = {
|
||||
raw: 68, evidence: ev,
|
||||
bindingConstraint: 'server/api/** handlers untested',
|
||||
nextAction: 'behaviour-asserting coverage',
|
||||
}
|
||||
const bugs = scoreProduct(a).dimensions.find(d => d.dimension === 'bugs')!
|
||||
expect(bugs.bindingConstraint).toBe('server/api/** handlers untested')
|
||||
expect(bugs.nextAction).toBe('behaviour-asserting coverage')
|
||||
})
|
||||
})
|
||||
|
||||
describe('scoreSuite', () => {
|
||||
it('weights by tier', () => {
|
||||
// A-tier (2) at 90, C-tier (1) at 60 → (90*2 + 60*1) / 3 = 80
|
||||
const r = scoreSuite([flat('big', 90, 'A'), flat('small', 60, 'C')])
|
||||
expect(r.dimensions.security).toBe(80)
|
||||
})
|
||||
|
||||
it('applies a suite-wide cap BEFORE averaging', () => {
|
||||
// Averaging first would let healthy products dilute a flaw affecting all
|
||||
// of them: (90+90)/2 = 90, capped after → 90. Capping first → 45.
|
||||
const r = scoreSuite(
|
||||
[flat('a', 90, 'A'), flat('b', 90, 'A')],
|
||||
{ suiteCaps: ['sec.leaked-secret'] },
|
||||
)
|
||||
expect(r.dimensions.security).toBe(45)
|
||||
})
|
||||
|
||||
it('leads with the lowest suite dimension', () => {
|
||||
const a = flat('a', 85)
|
||||
a.dimensions.consistency = { raw: 63, evidence: ev }
|
||||
const r = scoreSuite([a])
|
||||
expect(r.overall).toBe(63)
|
||||
expect(r.weakest).toBe('consistency')
|
||||
})
|
||||
|
||||
it('throws on an empty suite instead of returning a flattering zero', () => {
|
||||
expect(() => scoreSuite([])).toThrow(/at least one product/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('rubric integrity', () => {
|
||||
it('every cap rule has a unique id', () => {
|
||||
const ids = CAP_RULES.map(c => c.id)
|
||||
expect(new Set(ids).size).toBe(ids.length)
|
||||
})
|
||||
|
||||
it('every cap belongs to a known dimension and sits in 0-100', () => {
|
||||
for (const c of CAP_RULES) {
|
||||
expect(DIMENSIONS).toContain(c.dimension)
|
||||
expect(c.max).toBeGreaterThanOrEqual(0)
|
||||
expect(c.max).toBeLessThanOrEqual(100)
|
||||
}
|
||||
})
|
||||
|
||||
it('every dimension carries at least one cap', () => {
|
||||
for (const d of DIMENSIONS) {
|
||||
expect(CAP_RULES.some(c => c.dimension === d)).toBe(true)
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('conformance with the evidence-record standard', () => {
|
||||
const withCapEvidence = scoreDimension('security', {
|
||||
raw: 70,
|
||||
openCaps: [{
|
||||
id: CAP_RULES.find(c => c.dimension === 'security')!.id,
|
||||
evidence: [{ ref: 'docs/secrets-review.md:12', note: 'exposed key, never rotated' }],
|
||||
}],
|
||||
})
|
||||
|
||||
it('a cap cites what establishes it', () => {
|
||||
// The gap this closes: "CAPPED sec.leaked-secret" with nothing behind it.
|
||||
expect(withCapEvidence.boundBy!.evidence).toHaveLength(1)
|
||||
const text = formatProduct(scoreProduct(assessmentWith('security', {
|
||||
raw: 70,
|
||||
openCaps: [{
|
||||
id: CAP_RULES.find(c => c.dimension === 'security')!.id,
|
||||
evidence: [{ ref: 'docs/secrets-review.md:12', note: 'never rotated' }],
|
||||
}],
|
||||
})), { evidence: 'full' })
|
||||
expect(text).toContain('docs/secrets-review.md:12')
|
||||
expect(text).toContain('never rotated')
|
||||
})
|
||||
|
||||
it('still applies a cap that cites nothing, and says it cited nothing', () => {
|
||||
// An unproven cap is a smaller error than an unapplied one.
|
||||
const bare = scoreDimension('security', {
|
||||
raw: 90,
|
||||
openCaps: [CAP_RULES.find(c => c.dimension === 'security')!.id],
|
||||
})
|
||||
expect(bare.score).toBeLessThan(90)
|
||||
const text = formatProduct(scoreProduct(assessmentWith('security', {
|
||||
raw: 90,
|
||||
openCaps: [CAP_RULES.find(c => c.dimension === 'security')!.id],
|
||||
})), { evidence: 'full' })
|
||||
expect(text).toContain('No evidence was recorded for this cap')
|
||||
})
|
||||
|
||||
it('converts a legacy path:line ref into a real citation', () => {
|
||||
const d = scoreDimension('security', { raw: 50, evidence: [{ ref: 'server/api/x.ts:9' }] })
|
||||
expect(d.evidence[0]!.source).toMatchObject({ kind: 'file', path: 'server/api/x.ts', line: 9 })
|
||||
})
|
||||
|
||||
it('a score withheld for lack of evidence is not-assessed, not a pass', () => {
|
||||
const p = scoreProduct(assessmentWith('security', { raw: 95 }))
|
||||
const f = productReport(p).findings.find(x => x.id.endsWith('.security'))!
|
||||
expect(f.determination).toBe('not-assessed')
|
||||
expect(formatProduct(p)).toContain('no evidence was cited')
|
||||
})
|
||||
|
||||
it('every conclusion carries a citation', () => {
|
||||
expect(validateReport(productReport(scoreProduct(assessmentWith('security', { raw: 60 }))))).toEqual([])
|
||||
})
|
||||
|
||||
it('labels the average so it cannot be read as the headline', () => {
|
||||
const text = formatProduct(scoreProduct(assessmentWith('security', { raw: 60 })))
|
||||
expect(text).toContain('never as the headline')
|
||||
expect(text).toContain('set by the weakest dimension')
|
||||
})
|
||||
})
|
||||
|
||||
describe('band rendering', () => {
|
||||
it('prints the band name, not a stringified object', () => {
|
||||
const text = formatProduct(scoreProduct(flat('app', 80)))
|
||||
expect(text).not.toContain('[object Object]')
|
||||
expect(text).toContain('Production')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022", "module": "ES2022", "moduleResolution": "bundler",
|
||||
"declaration": true, "outDir": "dist", "rootDir": "src",
|
||||
"strict": true, "skipLibCheck": true
|
||||
},
|
||||
"include": ["src/**/*.ts"]
|
||||
}
|
||||
Reference in New Issue
Block a user