Paul HittandClaude Opus 5.5
339129990c
It needs zod, which this package does not depend on, so the import now says to install it. assertFields threw on the drift the example shows on purpose, which stopped the script before the coverage part; it is now a commented alternative to the expectFields report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
105 lines
4.5 KiB
Markdown
105 lines
4.5 KiB
Markdown
# ContractCheck
|
|
|
|
Catches interface drift between two systems: a response whose shape changed,
|
|
a requested field that never came back, and endpoints with no contract check.
|
|
It is used in Hitt Hosting products.
|
|
|
|
## What it does and why
|
|
|
|
**A response can be perfectly valid while the field you asked for is
|
|
missing.** Stripe removed `invoice.payment_intent` in its 2025-03-31 (Basil)
|
|
API version, and Stripe ignores an unknown `expand` path instead of rejecting
|
|
it. A stale expand keeps succeeding and returns a well-formed invoice without
|
|
the field. There is no error and no warning; the client secret is absent and
|
|
the payment form cannot mount, while typecheck, lint and the test suite all
|
|
pass. A schema validator cannot catch this, because the invoice is valid. The
|
|
only way to detect it is to check that what you requested actually came back.
|
|
|
|
ContractCheck has three parts.
|
|
|
|
- `assertShape()` validates a payload against any schema with a zod-style
|
|
`safeParse` and throws a `ContractDriftError` naming the context and the
|
|
offending paths. It fails loudly because the bug it targets is an endpoint
|
|
dropping a field the UI needs: nothing errors, the page renders, and the
|
|
number is just wrong. A 500 naming the path beats a plausible screen. The
|
|
error is a plain `Error`, not tied to a web framework, so you map it to your
|
|
own transport.
|
|
- `expectFields()` and `assertFields()` check that each requested dot path is
|
|
present in the response. An explicit `null` counts as present: `null` means
|
|
the provider answered "no value", while `undefined` means it never
|
|
acknowledged the request. A `why` note on each expectation is quoted in the
|
|
failure, so it says what breaks and not only what is missing.
|
|
- `coverage()` measures how much of an API surface is contract-checked,
|
|
against a floor you can raise over time so coverage cannot slip back. It
|
|
lists the uncovered endpoints, because a bare percentage tells you there is
|
|
a gap, not where it is.
|
|
|
|
The `Parser` type accepts Zod 4 schemas, whose issue paths may include
|
|
symbols, and symbol path segments are rendered without crashing.
|
|
|
|
## Reports
|
|
|
|
`fieldReport()` and `formatFields()` make each requested path its own
|
|
finding, so "the response is a valid object" and "the field you asked for is
|
|
absent" never collapse into one verdict. If no fields were asserted, the
|
|
report is not assessed.
|
|
|
|
`coverageReport()` and `formatCoverage()` report an empty surface as not
|
|
assessed, not 0%. "0% covered" for a surface with no endpoints describes a
|
|
failure that did not happen. Uncovered endpoints are listed one by one as not
|
|
assessed. Reports use the `@extant2000/evidence-record` format.
|
|
|
|
## Usage
|
|
|
|
```ts
|
|
import { assertShape, expectFields, assertFields, formatFields, coverage, formatCoverage } from '@extant2000/contract-check'
|
|
import { z } from 'zod' // npm install zod, or use any parser with a zod-style safeParse
|
|
|
|
// 1. Response shape
|
|
const payload: unknown = { id: 'm1', name: 'Survey' } // e.g. a parsed response body
|
|
const Mission = z.object({ id: z.string(), name: z.string() })
|
|
const mission = assertShape(Mission, payload, 'GET /missions/:id')
|
|
|
|
// 2. Requested fields actually returned
|
|
const subscription = { id: 'sub_1', latest_invoice: { id: 'in_1' } } // response from the provider
|
|
const expectations = [
|
|
{ path: 'latest_invoice.payment_intent', why: 'the client secret is needed to mount the payment form' },
|
|
'id',
|
|
]
|
|
const result = expectFields(subscription, expectations)
|
|
console.log(formatFields(result, { context: 'stripe.subscriptions.create', expectations }))
|
|
// assertFields(subscription, expectations, 'stripe.subscriptions.create') // throws ContractDriftError here instead
|
|
|
|
// 3. Coverage against a floor
|
|
const c = coverage(['GET /a', 'GET /b', 'POST /c', 'GET /d'], ['GET /a', 'GET /b', 'POST /c'], 80)
|
|
console.log(formatCoverage(c, 80))
|
|
```
|
|
|
|
For a subscription whose invoice lacks `payment_intent`, the field report
|
|
fails at critical severity on `latest_invoice.payment_intent`, quotes the
|
|
`why` note, and passes `id`. The coverage example prints:
|
|
|
|
```
|
|
✗ HIGH 3 of 4 endpoints contract-checked (75%) [NOT MET]
|
|
Below the ratchet floor of 80%.
|
|
evidence: Contract coverage = 75% (4 samples)
|
|
· MEDIUM GET /d has no contract check [NOT ASSESSED]
|
|
```
|
|
|
|
ContractCheck does not depend on zod. Any object with a compatible
|
|
`safeParse` works.
|
|
|
|
## Install
|
|
|
|
The package is published to the GitLab package registry, so point the
|
|
`@extant2000` scope there first:
|
|
|
|
```
|
|
npm config set @extant2000:registry https://gitlab.com/api/v4/packages/npm/
|
|
npm install @extant2000/contract-check
|
|
```
|
|
|
|
## License
|
|
|
|
MIT. See [LICENSE](LICENSE).
|