First public release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,12 @@
|
||||
root = true
|
||||
|
||||
[*]
|
||||
charset = utf-8
|
||||
end_of_line = lf
|
||||
insert_final_newline = true
|
||||
trim_trailing_whitespace = true
|
||||
indent_style = space
|
||||
indent_size = 2
|
||||
|
||||
[*.md]
|
||||
trim_trailing_whitespace = false
|
||||
@@ -0,0 +1,9 @@
|
||||
node_modules/
|
||||
dist/
|
||||
*.log
|
||||
.DS_Store
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
.npmrc
|
||||
dist-cjs/
|
||||
@@ -0,0 +1,6 @@
|
||||
stages: [test]
|
||||
|
||||
test:
|
||||
stage: test
|
||||
image: node:22-alpine
|
||||
script: [npm ci, npm run build, npm test]
|
||||
@@ -0,0 +1,15 @@
|
||||
# Changelog
|
||||
|
||||
## 0.2.0 - 2026-09-28
|
||||
|
||||
First public release under MIT.
|
||||
|
||||
- `evaluatePolicy()` and `evaluateRule()` run data-defined policy rules over
|
||||
a set of files. Pure, with no I/O and no git.
|
||||
- A rule no file was subject to is reported as NOT ASSESSED, never as a pass.
|
||||
- Patterns with the `g` or `y` flag are tested through a fresh copy, so
|
||||
`lastIndex` cannot skip files.
|
||||
- Exemptions use a comment marker and are counted and cited.
|
||||
- Violations cite the file and line; passes name the files they checked.
|
||||
- Rule factories: `orgScopingRule`, `inOverflowRule`, `claimRule`.
|
||||
- `toReport()` and `formatReport()` produce an evidence-record report.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Contributing
|
||||
|
||||
Issues and merge requests are welcome at
|
||||
https://gitlab.com/extant2000/policy-enforced.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- A test that cannot fail proves nothing. If you fix a bug, add a test and
|
||||
check that it fails against the unfixed code before you submit.
|
||||
- Measure, don't assume. Two modules with the same line count can still be
|
||||
different programs.
|
||||
- Keep dependencies minimal. Every new runtime dependency needs a reason.
|
||||
- Explain why in comments, not what. The what is already in the code.
|
||||
- A breaking change needs a major version bump and a note in CHANGELOG.md.
|
||||
|
||||
## Before you open a merge request
|
||||
|
||||
Run both of these and make sure they pass:
|
||||
|
||||
npm run build
|
||||
npm test
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Extant 2000 LLC
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,119 @@
|
||||
# PolicyEnforced
|
||||
|
||||
Check that a stated policy is enforced in the code, not only written down.
|
||||
|
||||
PolicyEnforced is a small policy gate engine. A rule is data: which paths it
|
||||
covers, what content makes a file subject to it, what content satisfies it,
|
||||
and how an author documents an exemption. The engine is pure. You pass in the
|
||||
files (usually the ones a change touched) and it returns what was checked,
|
||||
what passed, what was exempted and what failed.
|
||||
|
||||
## What it checks and why
|
||||
|
||||
Policy gates written as shell scripts tend to get copied from app to app and
|
||||
edited in place. Some copies drift for no reason. Others have to differ: an
|
||||
org-scoping check for an app that filters by an `org_id` column would flag
|
||||
every handler in an app that has no tenant columns and authorises through
|
||||
RBAC capability checks. One shared script cannot serve both, and a separate
|
||||
copy per app is the wrong fix. So the engine is shared and the patterns are
|
||||
parameters.
|
||||
|
||||
```ts
|
||||
// An app that scopes rows by tenant column
|
||||
orgScopingRule({
|
||||
guards: /requireOrg|resolveOrg/,
|
||||
inlineTenantFilter: /\.eq\((['"])(organization_id|org_id|owner_id)\1/,
|
||||
})
|
||||
|
||||
// An app with no tenant columns that uses RBAC checks
|
||||
orgScopingRule({
|
||||
serviceClients: /createServiceClient|createClient\(/,
|
||||
guards: /requireAuth|requireProjectCap|requireAdminCap/,
|
||||
})
|
||||
```
|
||||
|
||||
The engine is built against three failure modes.
|
||||
|
||||
1. A vacuous run is not a pass. `passed` and `vacuous` are separate fields,
|
||||
and a rule that no file was subject to is reported as NOT ASSESSED. "No
|
||||
changed file was covered by this rule" is not evidence that the policy
|
||||
holds. A gate that prints a check mark for both is indistinguishable from
|
||||
one that does nothing, which is the same defect as a test that cannot fail.
|
||||
2. Regex state can skip files. A rule built with a `/g` pattern carries
|
||||
`lastIndex` from one file to the next, so later files can pass unexamined
|
||||
and the gate goes green while covering nothing. Every pattern is tested
|
||||
through a fresh non-global copy.
|
||||
3. A gate with no escape hatch gets switched off. The first legitimate
|
||||
cross-tenant cron job would force someone to disable it. Exemptions are a
|
||||
documented comment marker, and they are counted and cited in the report so
|
||||
they stay visible.
|
||||
|
||||
A pass names the files it checked, and a violation cites the file and line
|
||||
where the rule's trigger matched. A result of "3 files checked" that does not
|
||||
say which files is an asserted result, which is the kind of claim this library
|
||||
exists to catch.
|
||||
|
||||
One limit to keep in mind: the org-scoping rule confirms that a guard is
|
||||
present. It does not prove the guard filters to the caller's own rows. That
|
||||
still needs tests and review.
|
||||
|
||||
## Rules included
|
||||
|
||||
- `orgScopingRule` flags a handler that uses a service-role client (row-level
|
||||
security bypassed) with no authorization guard, tenant filter or exemption.
|
||||
- `inOverflowRule` flags a dynamic `.in('col', ids)` filter with no chunking
|
||||
helper. With supabase-js, a long id list goes into the query string, the
|
||||
request fails, and `data || []` turns that into an empty result with no
|
||||
error. A literal array is bounded and is not flagged.
|
||||
- `claimRule` flags a public claim (for example "99.9% uptime SLA" on a
|
||||
pricing page) when nothing in the code enforces it.
|
||||
|
||||
You can also write a `PolicyRule` object directly.
|
||||
|
||||
## Usage
|
||||
|
||||
```ts
|
||||
import { evaluatePolicy, formatReport, orgScopingRule, inOverflowRule } from '@extant2000/policy-enforced'
|
||||
|
||||
const rules = [
|
||||
orgScopingRule({ guards: /requireOrg|resolveOrg/, inlineTenantFilter: /\.eq\((['"])(org_id|owner_id)\1/ }),
|
||||
inOverflowRule({ chunkHelpers: /selectInChunks/ }),
|
||||
]
|
||||
|
||||
const files = [
|
||||
{ path: 'server/api/invoices.get.ts', content: "const db = createServiceClient()\nreturn db.from('invoices').select()" },
|
||||
{ path: 'server/api/contacts.get.ts', content: "await requireOrg(event)\nconst db = createServiceClient()" },
|
||||
]
|
||||
|
||||
const report = evaluatePolicy(rules, files)
|
||||
console.log({ passed: report.passed, vacuous: report.vacuous })
|
||||
for (const v of report.violations) console.log(`${v.ruleId} ${v.path}:${v.line} ${v.excerpt}`)
|
||||
|
||||
// The full report for a CI log, with remedies under each violation
|
||||
console.log(formatReport(report))
|
||||
```
|
||||
|
||||
`toReport(report)` returns the same result as a `CapabilityReport` from
|
||||
`@extant2000/evidence-record`. `evaluateRule(rule, files)` runs a single rule.
|
||||
|
||||
## Sample output
|
||||
|
||||
Produced by running the example above (first two lines):
|
||||
|
||||
```
|
||||
{ passed: false, vacuous: false }
|
||||
org-scoping server/api/invoices.get.ts:1 const db = createServiceClient()
|
||||
```
|
||||
|
||||
In the full report, `in-overflow` shows as NOT ASSESSED, since neither file
|
||||
used a dynamic `.in()` filter.
|
||||
|
||||
## Install
|
||||
|
||||
```
|
||||
npm install @extant2000/policy-enforced
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT. Copyright (c) 2026 Extant 2000 LLC. See [LICENSE](LICENSE).
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
# Security policy
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Please report vulnerabilities privately by email to security@extant2000.com.
|
||||
Do not open a public issue or merge request for a security problem.
|
||||
|
||||
Include the affected version, a description of the issue, and steps or a test
|
||||
that reproduce it if you have them.
|
||||
|
||||
We aim to acknowledge every report within 5 business days.
|
||||
Generated
+1504
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,43 @@
|
||||
{
|
||||
"name": "@extant2000/policy-enforced",
|
||||
"version": "0.2.0",
|
||||
"private": false,
|
||||
"description": "Verify that a stated policy is enforced in the code, not only written in a document.",
|
||||
"license": "MIT",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
"files": [
|
||||
"dist",
|
||||
"src",
|
||||
"README.md",
|
||||
"LICENSE"
|
||||
],
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://gitlab.com/extant2000/policy-enforced.git"
|
||||
},
|
||||
"types": "dist/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./dist/index.d.ts",
|
||||
"import": "./dist/index.js"
|
||||
}
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"test": "vitest run",
|
||||
"prepublishOnly": "npm run build"
|
||||
},
|
||||
"devDependencies": {
|
||||
"typescript": "^5.6.0",
|
||||
"vitest": "^2.1.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@extant2000/evidence-record": "^0.1.2"
|
||||
},
|
||||
"author": "Extant 2000 LLC",
|
||||
"homepage": "https://gitlab.com/extant2000/policy-enforced",
|
||||
"bugs": {
|
||||
"url": "https://gitlab.com/extant2000/policy-enforced/-/issues"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,98 @@
|
||||
import type {
|
||||
PolicyReport,
|
||||
PolicyRule,
|
||||
RuleResult,
|
||||
SourceFile,
|
||||
Violation,
|
||||
} from './types.js'
|
||||
|
||||
function matchesPath(path: string, includes: string[]): boolean {
|
||||
return includes.some(i => path.includes(i))
|
||||
}
|
||||
|
||||
/**
|
||||
* A regex with the `g` flag carries `lastIndex` between `.test()` calls, so the
|
||||
* same pattern reused across files silently starts matching from wherever the
|
||||
* previous file left off — producing false passes that look like clean runs.
|
||||
* Test against a fresh, non-global copy every time.
|
||||
*/
|
||||
function test(re: RegExp, content: string): boolean {
|
||||
return new RegExp(re.source, re.flags.replace(/[gy]/g, '')).test(content)
|
||||
}
|
||||
|
||||
/**
|
||||
* Locate the trigger match so a finding can cite a line, not just a file.
|
||||
*
|
||||
* Scans line by line rather than counting newlines before a whole-file match
|
||||
* index, because the trigger may legitimately span nothing more than one line
|
||||
* and this keeps the excerpt exactly what the reader will see in the editor.
|
||||
*/
|
||||
function locate(re: RegExp, content: string): { line: number, excerpt: string } | undefined {
|
||||
const bare = new RegExp(re.source, re.flags.replace(/[gy]/g, ''))
|
||||
const lines = content.split('\n')
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
if (bare.test(lines[i]!)) return { line: i + 1, excerpt: lines[i]!.trim().slice(0, 120) }
|
||||
}
|
||||
// The trigger matched the file but not any single line — a multi-line
|
||||
// pattern. Cite the file without a line rather than inventing one.
|
||||
return undefined
|
||||
}
|
||||
|
||||
/** Evaluate one rule over already-loaded files. Pure — no I/O, no git. */
|
||||
export function evaluateRule(rule: PolicyRule, files: SourceFile[]): RuleResult {
|
||||
let considered = 0
|
||||
let subject = 0
|
||||
let exempted = 0
|
||||
const violations: Violation[] = []
|
||||
const compliantPaths: string[] = []
|
||||
const exemptedPaths: string[] = []
|
||||
|
||||
for (const f of files) {
|
||||
if (!matchesPath(f.path, rule.pathIncludes)) continue
|
||||
considered++
|
||||
|
||||
if (!test(rule.trigger, f.content)) continue
|
||||
subject++
|
||||
|
||||
if (rule.allow.some(a => test(a, f.content))) {
|
||||
compliantPaths.push(f.path)
|
||||
continue
|
||||
}
|
||||
|
||||
if (f.content.includes(rule.exemptMarker)) {
|
||||
exempted++
|
||||
exemptedPaths.push(f.path)
|
||||
continue
|
||||
}
|
||||
|
||||
const at = locate(rule.trigger, f.content)
|
||||
violations.push({
|
||||
ruleId: rule.id,
|
||||
path: f.path,
|
||||
reason: rule.description,
|
||||
line: at?.line,
|
||||
excerpt: at?.excerpt,
|
||||
})
|
||||
}
|
||||
|
||||
return { rule, considered, subject, exempted, violations, compliantPaths, exemptedPaths }
|
||||
}
|
||||
|
||||
/**
|
||||
* Evaluate every rule.
|
||||
*
|
||||
* `vacuous` is reported separately from `passed`. A run where no changed file
|
||||
* was subject to any rule proves nothing — treating it as a pass is how a gate
|
||||
* ends up green for months while covering nothing, which is the same defect as
|
||||
* a test that cannot fail.
|
||||
*/
|
||||
export function evaluatePolicy(rules: PolicyRule[], files: SourceFile[]): PolicyReport {
|
||||
const results = rules.map(r => evaluateRule(r, files))
|
||||
const violations = results.flatMap(r => r.violations)
|
||||
return {
|
||||
results,
|
||||
violations,
|
||||
passed: violations.length === 0,
|
||||
vacuous: results.every(r => r.subject === 0),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
export * from './types.js'
|
||||
export * from './evaluate.js'
|
||||
export * from './rules.js'
|
||||
export * from './report.js'
|
||||
@@ -0,0 +1,90 @@
|
||||
import {
|
||||
type CapabilityReport,
|
||||
type Finding,
|
||||
evidence,
|
||||
formatReport as renderReport,
|
||||
type FormatOptions,
|
||||
} from '@extant2000/evidence-record'
|
||||
import type { PolicyReport, RuleResult } from './types.js'
|
||||
|
||||
/**
|
||||
* One finding per rule, in the shared vocabulary.
|
||||
*
|
||||
* The three-way split is the point of this library and now has three-way
|
||||
* types to carry it:
|
||||
*
|
||||
* nothing was subject to the rule -> not-assessed (used to print VACUOUS)
|
||||
* files were subject and complied -> pass, citing the files it checked
|
||||
* files were subject and did not -> fail, citing file:line and the line
|
||||
*
|
||||
* "Nothing changed that this rule covers" and "the policy holds" are different
|
||||
* statements, and a gate that prints ✓ for both is indistinguishable from one
|
||||
* that does nothing.
|
||||
*/
|
||||
function findingFor(r: RuleResult): Finding {
|
||||
const doc = evidence.document(r.rule.description, r.rule.id)
|
||||
|
||||
if (r.violations.length > 0) {
|
||||
return {
|
||||
id: r.rule.id,
|
||||
summary: `${r.rule.id} — ${r.violations.length} file(s) subject to the rule did not satisfy it`,
|
||||
determination: 'fail',
|
||||
severity: 'high',
|
||||
detail: r.rule.remedy.join(' '),
|
||||
evidence: [
|
||||
...r.violations.map(v => evidence.file(v.path, v.line, v.excerpt, v.reason)),
|
||||
doc,
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
if (r.subject === 0) {
|
||||
return {
|
||||
id: r.rule.id,
|
||||
summary: `${r.rule.id} — ${r.considered} file(s) examined, none subject to the rule`,
|
||||
determination: 'not-assessed',
|
||||
severity: 'medium',
|
||||
detail: 'Nothing was checked against this rule. That is not evidence the policy holds.',
|
||||
// Deliberately empty: the whole content of this finding is that there
|
||||
// was nothing to observe.
|
||||
evidence: [],
|
||||
}
|
||||
}
|
||||
|
||||
const ex = r.exempted ? `, ${r.exempted} exempted` : ''
|
||||
return {
|
||||
id: r.rule.id,
|
||||
summary: `${r.rule.id} — ${r.subject} file(s) checked and compliant${ex}`,
|
||||
determination: 'pass',
|
||||
severity: 'info',
|
||||
evidence: [
|
||||
...r.compliantPaths.map(p => evidence.file(p, undefined, undefined, 'satisfied the rule')),
|
||||
// Exemptions are evidence too, and printing them is what keeps an
|
||||
// exemption from being a quiet way to pass.
|
||||
...r.exemptedPaths.map(p => evidence.file(p, undefined, undefined, `exempted via "${r.rule.exemptMarker}"`)),
|
||||
doc,
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
/** Convert a policy run into the portfolio-standard report shape. */
|
||||
export function toReport(report: PolicyReport): CapabilityReport {
|
||||
const considered = report.results.reduce((n, r) => n + r.considered, 0)
|
||||
return {
|
||||
capability: 'PolicyEnforced',
|
||||
scope: `${report.results.length} rule(s) over ${considered} file(s)`,
|
||||
examined: considered,
|
||||
findings: report.results.map(findingFor),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a report for a CI log.
|
||||
*
|
||||
* Delegates to the shared renderer. The distinction that used to be carried
|
||||
* by the word VACUOUS is now carried by NOT ASSESSED plus a sentence, which
|
||||
* means the same thing without requiring a glossary.
|
||||
*/
|
||||
export function formatReport(report: PolicyReport, opts: FormatOptions = {}): string {
|
||||
return renderReport(toReport(report), opts)
|
||||
}
|
||||
+110
@@ -0,0 +1,110 @@
|
||||
import type { PolicyRule } from './types.js'
|
||||
|
||||
/**
|
||||
* Rule factories for three common gates.
|
||||
*
|
||||
* Each takes the parts that vary per app. An org-scoping gate differs between
|
||||
* apps precisely because these lists differ (one app guards with
|
||||
* `requireOrg` and tenant columns, another with RBAC capability checks), so
|
||||
* they are parameters, not constants.
|
||||
*/
|
||||
|
||||
export interface OrgScopingOptions {
|
||||
/** Accessors that BYPASS row-level security. */
|
||||
serviceClients?: RegExp
|
||||
/** Named guards that make a service-role handler safe. */
|
||||
guards: RegExp
|
||||
/** Optional: inline tenant-column filter that also counts as scoped. */
|
||||
inlineTenantFilter?: RegExp
|
||||
pathIncludes?: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* A changed handler using a service-role client (RLS bypassed) must carry an
|
||||
* authorization check or a documented exemption.
|
||||
*
|
||||
* Diff-scoped on purpose: it does not police pre-existing unguarded endpoints
|
||||
* (track those separately), only new or changed code where the author is right there.
|
||||
* A gate nobody trusts is worse than no gate.
|
||||
*
|
||||
* LIMITATION, inherited and worth restating: this confirms a guard is PRESENT,
|
||||
* not that it filters the caller's own rows. Value-correctness is for tests and
|
||||
* review.
|
||||
*/
|
||||
export function orgScopingRule(opts: OrgScopingOptions): PolicyRule {
|
||||
const allow = [opts.guards]
|
||||
if (opts.inlineTenantFilter) allow.push(opts.inlineTenantFilter)
|
||||
|
||||
return {
|
||||
id: 'org-scoping',
|
||||
description: 'service-role handler with no authorization check or exemption',
|
||||
pathIncludes: opts.pathIncludes ?? ['server/api/'],
|
||||
trigger: opts.serviceClients
|
||||
?? /useServerSupabase|serverSupabaseServiceRole|createServiceClient|createClient\(/,
|
||||
allow,
|
||||
exemptMarker: 'org-scoping-exempt:',
|
||||
remedy: [
|
||||
'Add an authorization guard, or scope the query by a tenant column.',
|
||||
'If the route is genuinely cross-tenant (cron, token, webhook, admin),',
|
||||
'document it: // org-scoping-exempt: <reason>',
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A changed file adding a dynamic `.in()` filter must chunk it.
|
||||
*
|
||||
* The bug: supabase-js puts every id in the query string, so past roughly 400
|
||||
* UUIDs the request exceeds the header limit and dies — returning null, which
|
||||
* `data || []` turns into a convincing empty state on top of real rows. It does
|
||||
* not error. That is why this is a gate and not a code review note.
|
||||
*/
|
||||
export function inOverflowRule(opts: { chunkHelpers: RegExp, pathIncludes?: string[] }): PolicyRule {
|
||||
return {
|
||||
id: 'in-overflow',
|
||||
description: 'dynamic .in() filter with no chunking helper or exemption',
|
||||
pathIncludes: opts.pathIncludes ?? ['server/', 'app/'],
|
||||
// `.in('col', someVariable)` — a literal array is bounded and fine.
|
||||
trigger: /\.in\(\s*['"][^'"]+['"]\s*,\s*(?!\[)[A-Za-z_$]/,
|
||||
allow: [opts.chunkHelpers],
|
||||
exemptMarker: 'in-overflow-ok:',
|
||||
remedy: [
|
||||
'Chunk the filter (selectInChunks or equivalent), or document why the',
|
||||
'list is bounded: // in-overflow-ok: <reason>',
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
/** A rule asserting a stated claim has an enforcement mechanism behind it. */
|
||||
export interface ClaimOptions {
|
||||
id: string
|
||||
/** Where the claim is made (marketing copy, docs, policy pages). */
|
||||
claimPaths: string[]
|
||||
/** The claim text. */
|
||||
claim: RegExp
|
||||
/** Evidence that it is actually enforced. */
|
||||
enforcedBy: RegExp
|
||||
remedy?: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* The general "advertised == implemented" shape.
|
||||
*
|
||||
* The typical case: uptime or SLA language appears on pricing and marketing
|
||||
* pages with no server-side enforcement anywhere. A public claim with nothing
|
||||
* behind it is a compliance risk, not just a documentation gap.
|
||||
*/
|
||||
export function claimRule(opts: ClaimOptions): PolicyRule {
|
||||
return {
|
||||
id: opts.id,
|
||||
description: 'public claim with no enforcement behind it',
|
||||
pathIncludes: opts.claimPaths,
|
||||
trigger: opts.claim,
|
||||
allow: [opts.enforcedBy],
|
||||
exemptMarker: `${opts.id}-exempt:`,
|
||||
remedy: opts.remedy ?? [
|
||||
'Either implement the enforcement, or remove/soften the claim.',
|
||||
`If enforcement lives elsewhere, document it: // ${opts.id}-exempt: <where>`,
|
||||
],
|
||||
}
|
||||
}
|
||||
+103
@@ -0,0 +1,103 @@
|
||||
/**
|
||||
* A policy that must hold in the running codebase, not merely in a document.
|
||||
*
|
||||
* Policy gates written as shell scripts tend to share one engine and differ
|
||||
* only in their patterns. An org-scoping gate copied across several apps can
|
||||
* legitimately end up with a distinct copy per app, and that divergence is
|
||||
* CORRECT, not drift: one app scopes by an `org_id` tenant column, another has
|
||||
* no tenant columns at all and authorises via RBAC capability checks. Forcing
|
||||
* one script would break one of them.
|
||||
*
|
||||
* So the engine is shared and the policy is data.
|
||||
*/
|
||||
|
||||
export interface PolicyRule {
|
||||
/** Stable id, used in output and to reference a rule. */
|
||||
id: string
|
||||
/** One line: what must be true. */
|
||||
description: string
|
||||
|
||||
/**
|
||||
* Only files whose path matches one of these substrings/patterns are
|
||||
* considered. Kept deliberately simple — a rule that needs a real glob
|
||||
* engine is usually a rule that wants narrowing instead.
|
||||
*/
|
||||
pathIncludes: string[]
|
||||
|
||||
/**
|
||||
* The file is SUBJECT to this rule only if its content matches. This is the
|
||||
* risky construct — a service-role client, an unbounded `.in()`, and so on.
|
||||
*/
|
||||
trigger: RegExp
|
||||
|
||||
/**
|
||||
* Any match here makes the file compliant. Multiple entries are OR-ed,
|
||||
* because there is usually more than one legitimate way to satisfy a policy.
|
||||
*/
|
||||
allow: RegExp[]
|
||||
|
||||
/**
|
||||
* A comment marker that documents a deliberate exemption, e.g.
|
||||
* `org-scoping-exempt:`. Exemptions are part of a workable gate: without one,
|
||||
* the legitimate cross-tenant cron job forces someone to disable the check
|
||||
* entirely.
|
||||
*/
|
||||
exemptMarker: string
|
||||
|
||||
/** Lines printed under a violation telling the author how to fix it. */
|
||||
remedy: string[]
|
||||
}
|
||||
|
||||
export interface SourceFile {
|
||||
path: string
|
||||
content: string
|
||||
}
|
||||
|
||||
export interface Violation {
|
||||
ruleId: string
|
||||
path: string
|
||||
/** Why it failed, in one line. */
|
||||
reason: string
|
||||
/**
|
||||
* 1-indexed line where the trigger matched — the construct that made this
|
||||
* file subject to the rule. A violation that names only the file sends the
|
||||
* reader hunting; this points at the line that did it.
|
||||
*/
|
||||
line?: number
|
||||
/** The matched line, trimmed. */
|
||||
excerpt?: string
|
||||
}
|
||||
|
||||
export interface RuleResult {
|
||||
rule: PolicyRule
|
||||
/** Files the rule actually examined (matched pathIncludes). */
|
||||
considered: number
|
||||
/** Files that were subject to it (matched the trigger). */
|
||||
subject: number
|
||||
/** Files excused by an exemption marker. */
|
||||
exempted: number
|
||||
violations: Violation[]
|
||||
/**
|
||||
* Paths that were subject to the rule and satisfied it.
|
||||
*
|
||||
* Retained so a PASS can cite what it checked. A gate that reports "✓ 3
|
||||
* files checked" without naming them is an asserted result, and this
|
||||
* library's whole premise is that written policy and enforced policy are
|
||||
* different things.
|
||||
*/
|
||||
compliantPaths: string[]
|
||||
/** Paths excused by the exemption marker — counted, never hidden. */
|
||||
exemptedPaths: string[]
|
||||
}
|
||||
|
||||
export interface PolicyReport {
|
||||
results: RuleResult[]
|
||||
violations: Violation[]
|
||||
passed: boolean
|
||||
/**
|
||||
* True when there was nothing to check. Distinguished from `passed` on
|
||||
* purpose: "no changed files matched" is not evidence the policy holds, and
|
||||
* a report that conflates them is the same lie as a test that cannot fail.
|
||||
*/
|
||||
vacuous: boolean
|
||||
}
|
||||
@@ -0,0 +1,211 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { overall, validateReport } from '@extant2000/evidence-record'
|
||||
import {
|
||||
claimRule,
|
||||
evaluatePolicy,
|
||||
evaluateRule,
|
||||
formatReport,
|
||||
toReport,
|
||||
inOverflowRule,
|
||||
orgScopingRule,
|
||||
type SourceFile,
|
||||
} from '../src/index.js'
|
||||
|
||||
const f = (path: string, content: string): SourceFile => ({ path, content })
|
||||
|
||||
// An app that scopes rows by tenant columns.
|
||||
const tenantOrg = orgScopingRule({
|
||||
guards: /requireOrg|resolveOrg|assertContactInOrg/,
|
||||
inlineTenantFilter: /\.eq\((['"])(organization_id|org_id|owner_id|user_id)\1/,
|
||||
})
|
||||
|
||||
// An app with no tenant columns at all, using RBAC capability checks instead.
|
||||
const rbacOrg = orgScopingRule({
|
||||
serviceClients: /createServiceClient|createClient\(/,
|
||||
guards: /requireAuth|requireProjectCap|requireAdminCap|getRequestClient/,
|
||||
})
|
||||
|
||||
describe('orgScopingRule — the divergence that made this a library', () => {
|
||||
it('tenant-column app: a service-role handler with no guard is a violation', () => {
|
||||
const r = evaluateRule(tenantOrg, [
|
||||
f('server/api/x.ts', 'const s = createServiceClient()\ns.from("t").select()'),
|
||||
])
|
||||
expect(r.violations).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('tenant-column app: an inline tenant filter satisfies it', () => {
|
||||
const r = evaluateRule(tenantOrg, [
|
||||
f('server/api/x.ts', `const s = createServiceClient()\ns.from("t").select().eq('org_id', id)`),
|
||||
])
|
||||
expect(r.violations).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('RBAC app: a capability check satisfies it, with NO tenant column', () => {
|
||||
// This is why one shared script could not work. The RBAC app has no org_id
|
||||
// anywhere; under the tenant-column rule this same file would be a false
|
||||
// violation.
|
||||
const rbacFile = f('server/api/y.ts',
|
||||
'await requireProjectCap(event, id, "view")\nconst s = createServiceClient()')
|
||||
expect(evaluateRule(rbacOrg, [rbacFile]).violations).toHaveLength(0)
|
||||
expect(evaluateRule(tenantOrg, [rbacFile]).violations).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('a documented exemption excuses the file and is counted', () => {
|
||||
const r = evaluateRule(tenantOrg, [
|
||||
f('server/api/cron.ts', 'createServiceClient()\n// org-scoping-exempt: nightly cron, cross-tenant by design'),
|
||||
])
|
||||
expect(r.violations).toHaveLength(0)
|
||||
expect(r.exempted).toBe(1)
|
||||
})
|
||||
|
||||
it('ignores files outside the path scope', () => {
|
||||
const r = evaluateRule(tenantOrg, [f('app/pages/x.vue', 'createServiceClient()')])
|
||||
expect(r.considered).toBe(0)
|
||||
expect(r.violations).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('ignores a file that never touches a service-role client', () => {
|
||||
const r = evaluateRule(tenantOrg, [f('server/api/x.ts', 'const s = getRequestClient(event)')])
|
||||
expect(r.considered).toBe(1)
|
||||
expect(r.subject).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('inOverflowRule', () => {
|
||||
const rule = inOverflowRule({ chunkHelpers: /selectInChunks/ })
|
||||
|
||||
it('flags a dynamic .in() with no chunking', () => {
|
||||
const r = evaluateRule(rule, [f('server/x.ts', `supabase.from('t').select().in('id', ids)`)])
|
||||
expect(r.violations).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('accepts a chunked call', () => {
|
||||
const r = evaluateRule(rule, [
|
||||
f('server/x.ts', `selectInChunks(ids, p => supabase.from('t').select().in('id', p))`),
|
||||
])
|
||||
expect(r.violations).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('does NOT flag a literal array — that list is bounded', () => {
|
||||
const r = evaluateRule(rule, [f('server/x.ts', `.in('status', ['a','b'])`)])
|
||||
expect(r.subject).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('claimRule — advertised == implemented', () => {
|
||||
// The classic gap: uptime or SLA language in marketing copy with nothing
|
||||
// in the code that enforces it.
|
||||
const rule = claimRule({
|
||||
id: 'uptime-claim',
|
||||
claimPaths: ['src/views/', 'app/pages/'],
|
||||
claim: /99\.\d+%\s*uptime|uptime\s*SLA/i,
|
||||
enforcedBy: /uptimeCredit|slaEnforcer/,
|
||||
})
|
||||
|
||||
it('flags an uptime claim with no enforcement', () => {
|
||||
const r = evaluateRule(rule, [f('src/views/Pricing.vue', '<p>99.9% uptime SLA</p>')])
|
||||
expect(r.violations).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('passes when enforcement is present', () => {
|
||||
const r = evaluateRule(rule, [
|
||||
f('src/views/Pricing.vue', '<p>99.9% uptime SLA</p>\nimport { uptimeCredit } from "~/lib"'),
|
||||
])
|
||||
expect(r.violations).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('regex state safety', () => {
|
||||
it('a global-flag rule does not skip files via lastIndex', () => {
|
||||
// A `g` regex carries lastIndex between .test() calls. Reusing one across
|
||||
// files makes later files silently pass — a green gate covering nothing.
|
||||
const sticky = orgScopingRule({
|
||||
serviceClients: /createServiceClient/g,
|
||||
guards: /requireOrg/g,
|
||||
})
|
||||
const r = evaluateRule(sticky, [
|
||||
f('server/api/a.ts', 'createServiceClient()'),
|
||||
f('server/api/b.ts', 'createServiceClient()'),
|
||||
f('server/api/c.ts', 'createServiceClient()'),
|
||||
])
|
||||
expect(r.violations).toHaveLength(3)
|
||||
})
|
||||
})
|
||||
|
||||
describe('evaluatePolicy + formatReport', () => {
|
||||
it('distinguishes "nothing was checked" from a real pass', () => {
|
||||
// "nothing changed that this covers" is not "the policy holds".
|
||||
const nothing = evaluatePolicy([tenantOrg], [f('server/api/x.ts', 'export default 1')])
|
||||
expect(nothing.passed).toBe(true)
|
||||
expect(nothing.vacuous).toBe(true)
|
||||
expect(overall(toReport(nothing))).toBe('not-assessed')
|
||||
const nothingText = formatReport(nothing)
|
||||
expect(nothingText).toContain('NOT ASSESSED')
|
||||
expect(nothingText).toContain('not evidence the policy holds')
|
||||
// The word it replaced needed a glossary.
|
||||
expect(nothingText).not.toContain('VACUOUS')
|
||||
|
||||
const real = evaluatePolicy([tenantOrg], [
|
||||
f('server/api/x.ts', `createServiceClient()\n.eq('org_id', o)`),
|
||||
])
|
||||
expect(real.passed).toBe(true)
|
||||
expect(real.vacuous).toBe(false)
|
||||
expect(overall(toReport(real))).toBe('pass')
|
||||
})
|
||||
|
||||
it('a pass cites the files it actually checked', () => {
|
||||
// A gate that reports "3 files checked" without naming them is an
|
||||
// asserted result, which is the defect this library exists to find.
|
||||
const real = evaluatePolicy([tenantOrg], [
|
||||
f('server/api/x.ts', `createServiceClient()\n.eq('org_id', o)`),
|
||||
])
|
||||
expect(toReport(real).findings[0]!.evidence.length).toBeGreaterThan(0)
|
||||
expect(formatReport(real, { evidence: 'full' })).toContain('server/api/x.ts')
|
||||
})
|
||||
|
||||
it('aggregates violations across rules and prints remedies', () => {
|
||||
const rep = evaluatePolicy([tenantOrg, inOverflowRule({ chunkHelpers: /selectInChunks/ })], [
|
||||
f('server/api/a.ts', 'createServiceClient()'),
|
||||
f('server/b.ts', `.in('id', ids)`),
|
||||
])
|
||||
expect(rep.passed).toBe(false)
|
||||
expect(rep.violations).toHaveLength(2)
|
||||
const text = formatReport(rep, { evidence: 'full' })
|
||||
expect(text).toContain('org-scoping')
|
||||
expect(text).toContain('in-overflow')
|
||||
expect(text).toContain('org-scoping-exempt:')
|
||||
expect(overall(toReport(rep))).toBe('fail')
|
||||
})
|
||||
|
||||
it('cites the line that triggered the rule, not just the file', () => {
|
||||
const rep = evaluatePolicy([tenantOrg], [
|
||||
f('server/api/a.ts', 'const x = 1\nconst y = 2\ncreateServiceClient()'),
|
||||
])
|
||||
expect(rep.violations[0]!.line).toBe(3)
|
||||
expect(rep.violations[0]!.excerpt).toBe('createServiceClient()')
|
||||
expect(formatReport(rep)).toContain('server/api/a.ts:3')
|
||||
})
|
||||
|
||||
it('counts exemptions as evidence rather than hiding them', () => {
|
||||
const rep = evaluatePolicy([tenantOrg], [
|
||||
f('server/api/a.ts', 'createServiceClient() // org-scoping-exempt: cross-tenant cron'),
|
||||
])
|
||||
expect(rep.passed).toBe(true)
|
||||
const text = formatReport(rep, { evidence: 'full' })
|
||||
expect(text).toContain('exempted via')
|
||||
expect(text).toContain('server/api/a.ts')
|
||||
})
|
||||
|
||||
it('every conclusion carries a citation', () => {
|
||||
const rep = evaluatePolicy([tenantOrg, inOverflowRule({ chunkHelpers: /selectInChunks/ })], [
|
||||
f('server/api/a.ts', 'createServiceClient()'),
|
||||
f('server/b.ts', `.in('id', ids)`),
|
||||
])
|
||||
expect(validateReport(toReport(rep))).toEqual([])
|
||||
})
|
||||
|
||||
it('an empty file set is vacuous, never a pass claim', () => {
|
||||
const rep = evaluatePolicy([tenantOrg], [])
|
||||
expect(rep.vacuous).toBe(true)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022", "module": "ES2022", "moduleResolution": "bundler",
|
||||
"declaration": true, "outDir": "dist", "rootDir": "src",
|
||||
"strict": true, "skipLibCheck": true
|
||||
},
|
||||
"include": ["src/**/*.ts"]
|
||||
}
|
||||
Reference in New Issue
Block a user