Check the content envelope against envelopes the iPhone emitted
The content layer had no vectors. Everything from section 6.6 onward was guarded only by same-code round trips: this engine encodes and parses through its own codec and would pass unchanged if it and the iPhone disagreed about every key. That is how SPEC 6.18 specified `ment` for as long as it has existed while both clients have always sent `men`. `content-v1.json` holds fifteen envelopes generated by RUNNING the iPhone's encoders, and this suite parses each one and asserts field by field. "It did not crash" is what the old tests already proved. One of them pins something easy to get wrong twice: the iPhone writes `fwd` as the integer 1, not as `true`, and `optBoolean` does not coerce a number. A parser reading only the boolean answers false and the message quietly loses the fact that it was passed on. The suite also fails if the vector file grows a case nobody asserts on. Separately, and the reason any of this is trustworthy: the test task did not track the vector files at all. They arrive by environment variable, Gradle does not track one as an input, so a vector could change under a build directory that already had results and the task reported UP-TO-DATE without running. That is not hypothetical here, it is how twenty vector tests once passed in CI for weeks without executing, on this same runner, whose build directory survives between runs. All six vector files are now declared inputs. Proven rather than asserted: with the inputs declared, flipping `men` to `ment` in one vector turns the suite red without `--rerun-tasks`. Before the fix the same change left it UP-TO-DATE and green. Full suite: 1294 tests, no failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
5d84ad180c
commit
19add570cb
@@ -58,6 +58,36 @@ tasks.test {
|
||||
events("passed", "failed", "skipped")
|
||||
showStandardStreams = true
|
||||
}
|
||||
|
||||
// The vector files arrive by environment variable, and Gradle does not
|
||||
// track an environment variable as an input. So a vector could move under
|
||||
// a build directory that already had results in it and the task would
|
||||
// report UP-TO-DATE without running a thing.
|
||||
//
|
||||
// That is not hypothetical. Twenty vector tests once "passed" in CI for
|
||||
// weeks without ever executing, because the runner is a developer machine
|
||||
// whose build directory survives between runs, and a replayed local pass
|
||||
// looks exactly like a real one in the log.
|
||||
//
|
||||
// Declaring each file as an input is what makes a changed vector
|
||||
// invalidate the task. Proven by changing one key in one vector and
|
||||
// watching the suite go red without `--rerun-tasks`.
|
||||
listOf(
|
||||
"FLAMENET_VECTORS",
|
||||
"FLAMENET_PQ_VECTORS",
|
||||
"FLAMENET_ACCOUNT_VECTORS",
|
||||
"FLAMENET_BACKUP_VECTORS",
|
||||
"FLAMENET_BACKUP_PAYLOAD_VECTORS",
|
||||
"FLAMENET_CONTENT_VECTORS",
|
||||
).forEach { name ->
|
||||
val path = System.getenv(name)
|
||||
if (path != null && file(path).exists()) {
|
||||
inputs
|
||||
.file(path)
|
||||
.withPropertyName("vector-$name")
|
||||
.withPathSensitivity(PathSensitivity.NONE)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Coverage for this module. `:engine` and `:client` are plain JVM libraries
|
||||
|
||||
Reference in New Issue
Block a user