From 625b4516c784b41cfc141ac1b63a3175eeedb528 Mon Sep 17 00:00:00 2001 From: Paul Hitt Date: Sun, 6 Sep 2026 17:21:44 -0400 Subject: [PATCH] Pair the e2e pin with a loud advisory at HEAD MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I pinned the vector clone earlier today on flamenet-relay#56's evidence, and stopped a step short. flamenet-relay did not: it pins AND keeps `conformance-head` / `doc-pages-head` running against HEAD with `allow_failure: true`, and its `.e2e-ref.md` says why in a sentence I had read and not acted on — "a relay tested against a pinned old harness is tested against a pinned old spec, and that argument still holds." A pin alone is worse than HEAD for exactly that reason. HEAD couples this board to another repository's merges; a bare pin means the vectors can move and this suite goes on checking last month's protocol, reporting green, with nothing anywhere saying so. The first failure is loud and wrong; the second is quiet and wrong, and quiet is worse in a conformance suite whose whole job is to notice a divergence. So: `.e2e-ref` holds the commit, `unit-tests` checks against it and blocks, and `vectors-head` runs the same vector suites against HEAD and is allowed to fail. Being behind is loud and non-blocking. Being broken is still a hard stop. Verified it would actually notice, rather than assuming — pin and HEAD are the same commit today, so a green advisory run proves nothing. With one expected value in `account-kdf-v1.json` mutated, `AccountKdfTest` reddens. `.e2e-ref.md` records the procedure and the deadlock story behind it. ⚠️ `vectors-head` runs `:engine:test` only. The rest of the suite has nothing to say about whether flamenet-e2e moved. Co-Authored-By: Claude Opus 5 --- .e2e-ref | 1 + .e2e-ref.md | 32 ++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 47 ++++++++++++++++++++++++++++++++++++++++++++--- 3 files changed, 77 insertions(+), 3 deletions(-) create mode 100644 .e2e-ref create mode 100644 .e2e-ref.md diff --git a/.e2e-ref b/.e2e-ref new file mode 100644 index 0000000..b1041c0 --- /dev/null +++ b/.e2e-ref @@ -0,0 +1 @@ +9775fbfe08c823921b6493b0fd39a0e4a13c8aa2 diff --git a/.e2e-ref.md b/.e2e-ref.md new file mode 100644 index 0000000..6a7ca53 --- /dev/null +++ b/.e2e-ref.md @@ -0,0 +1,32 @@ +# Why this repository pins flamenet-e2e + +`.e2e-ref` is the commit of [flamenet-e2e](https://gitlab.com/paulhitt/flamenet-e2e) +whose protocol vectors the `unit-tests` job checks against. `AccountKdfTest`, +`PQVectorTest` and `PublishedVectorsTest` all read them. + +🔴 **It exists because a merge in that repository could stop all work in this +one.** flamenet-relay learned this the hard way: its jobs cloned `main` at HEAD, +and the switch got thrown three times in twelve hours (flamenet-relay#56). Once +it was a deadlock with no way out from inside either repository — that repo +expected a change this one had not made, so every pipeline failed, *including +the pipeline for the branch that made it*. See `.e2e-ref.md` there for the full +account; the conclusion was that a pin makes that situation impossible rather +than survivable. + +⚠️ **A pin alone is worse than HEAD, though.** The argument for HEAD is real: a +client tested against a pinned old harness is tested against a pinned old spec, +and if the vectors move and nothing says so, this suite quietly stops checking +the protocol anybody is actually running. That is why `vectors-head` exists and +is `allow_failure: true`. + +**Being behind is LOUD and NON-BLOCKING. Being broken is a hard stop.** A red +`vectors-head` means an update is waiting, not that anything is wrong. + +## Taking an update + +One MR, both parts together: + +1. Put the new e2e commit in `.e2e-ref`. +2. If the vectors themselves changed, read what changed before accepting it — + a moved vector is a protocol change, and this app is one of the two ends + that has to agree about it. diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 795edad..6c497af 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -41,11 +41,19 @@ unit-tests: # different project and had no reason to look. # # A pin makes the same divergence a deliberate, reviewable bump here instead - # of a surprise failure in a repo nobody was thinking about. Move it when - # the protocol moves, and read what changed on the way past. + # of a surprise failure in a repo nobody was thinking about. The commit + # lives in `.e2e-ref`; `.e2e-ref.md` says how to take an update. + # + # ⚠️ **A pin ALONE is worse than HEAD**, and that is what the first version + # of this got wrong. Pinning silently means the vectors can move and this + # suite goes on checking last month's protocol, reporting green — the exact + # "tested against a pinned old spec" failure the relay's own note warns + # about. `vectors-head` below is the other half: it runs the same checks at + # HEAD and is allowed to fail, so being behind is LOUD and non-blocking + # while being broken stays a hard stop. # # `flamenet-e2e` is public, so no credentials are needed. - - FLAMENET_E2E_PIN=9775fbfe08c823921b6493b0fd39a0e4a13c8aa2 + - FLAMENET_E2E_PIN=$(cat "$CI_PROJECT_DIR/.e2e-ref") - rm -rf "$CI_PROJECT_DIR/.ci-vectors" - git clone -q https://gitlab.com/paulhitt/flamenet-e2e.git "$CI_PROJECT_DIR/.ci-vectors" - git -C "$CI_PROJECT_DIR/.ci-vectors" checkout -q "$FLAMENET_E2E_PIN" @@ -119,3 +127,36 @@ unit-tests: sys.exit(1 if bad else 0) PY timeout: 30m + +# 🔴 **The pin's conscience.** Runs the vector suites against flamenet-e2e's +# actual HEAD, and is allowed to fail. Without it, `.e2e-ref` is permission to +# fall behind forever with a green board — which is the failure mode a pin +# introduces and the reason the relay pairs its pin with the same job. +# +# A red run here means an update is waiting. Take it per `.e2e-ref.md`. +vectors-head: + stage: test + tags: [mac-local] + allow_failure: true + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: '$CI_COMMIT_BRANCH == "main"' + variables: + GIT_DEPTH: "1" + OBFS4_INTEROP_OPTOUT: "1" + script: + - export JAVA_HOME=/opt/homebrew/opt/openjdk@21 + - export ANDROID_HOME=$HOME/Library/Android/sdk + - rm -rf "$CI_PROJECT_DIR/.ci-vectors-head" + - git clone --depth 1 https://gitlab.com/paulhitt/flamenet-e2e.git "$CI_PROJECT_DIR/.ci-vectors-head" + - | + echo "pinned: $(cat "$CI_PROJECT_DIR/.e2e-ref")" + echo "at HEAD: $(git -C "$CI_PROJECT_DIR/.ci-vectors-head" rev-parse HEAD)" + - export FLAMENET_VECTORS="$CI_PROJECT_DIR/.ci-vectors-head/test/vectors/protocol-v1.json" + - export FLAMENET_PQ_VECTORS="$CI_PROJECT_DIR/.ci-vectors-head/test/vectors/pqx3dh-v2.json" + - export FLAMENET_ACCOUNT_VECTORS="$CI_PROJECT_DIR/.ci-vectors-head/test/vectors/account-kdf-v1.json" + # ⚠️ Only the module that reads vectors. The rest of the suite has nothing + # to say about whether flamenet-e2e moved, and running it here would double + # the board's cost to answer a question it cannot answer. + - ./gradlew --no-daemon --rerun-tasks :engine:test + timeout: 20m