The instrumented suite could not sign up, and said nothing about it

🔴 **Sign-up requires a year of birth and no instrumented test filled it.**
`submit` is `enabled = … && (!registering || birthYear.length == 4)`, so
the button was DISABLED — and `performClick()` on a disabled Compose node
does not throw, it does nothing. Every sign-up in this suite has been
tapping a dead button since the age gate landed, failing two minutes later
with 'nav-contacts did not appear' and no error anywhere on screen.

Nothing was wrong with the app. The suite simply stopped being able to
create an account, and a failing instrumented suite reads as a flaky one.

Also fixes the skip counter in scripts/instrumented.sh: it grepped a bare
SKIPPED, which matches Gradle's own `Task … SKIPPED` lines, so a verdict
written to stop a skip being read as a pass was inventing skips instead.
This commit is contained in:
Paul Hitt
2026-09-05 16:15:29 -04:00
parent 1a7c25386d
commit 77a7f3981c
5 changed files with 37 additions and 1 deletions
@@ -177,6 +177,14 @@ class ChatFlowTest {
compose.onNodeWithTag("email").performTextInput("$handle@example.invalid")
compose.onNodeWithTag("handle").performTextInput(handle)
compose.onNodeWithTag("passphrase").performTextInput("a-long-enough-passphrase")
// 🔴 **The age gate, which nothing here filled.** `submit` is
// `enabled = … && (!registering || birthYear.length == 4)`, so without
// this the button is DISABLED — and `performClick()` on a disabled node
// does not throw, it does nothing. Every sign-up in this suite has been
// silently tapping a dead button since the gate landed, and the failure
// surfaced two minutes later as "nav-contacts did not appear" with no
// error anywhere on screen.
compose.onNodeWithTag("birth-year").performTextInput("1990")
compose.onNodeWithTag("submit").performClick()
awaitTag("device-id", 120_000)
}
@@ -68,6 +68,14 @@ class CrossPlatformTest {
compose.onNodeWithTag("email").performTextInput("$handle@example.invalid")
compose.onNodeWithTag("handle").performTextInput(handle)
compose.onNodeWithTag("passphrase").performTextInput("a-long-enough-passphrase")
// 🔴 **The age gate, which nothing here filled.** `submit` is
// `enabled = … && (!registering || birthYear.length == 4)`, so without
// this the button is DISABLED — and `performClick()` on a disabled node
// does not throw, it does nothing. Every sign-up in this suite has been
// silently tapping a dead button since the gate landed, and the failure
// surfaced two minutes later as "nav-contacts did not appear" with no
// error anywhere on screen.
compose.onNodeWithTag("birth-year").performTextInput("1990")
compose.onNodeWithTag("submit").performClick()
awaitTag("nav-contacts", 150_000)
android.util.Log.i("fnxplat", "ANDROID_HANDLE=$handle")
@@ -231,6 +231,14 @@ class CrudSurfaceTest {
compose.onNodeWithTag("email").performTextInput("$handle@example.invalid")
compose.onNodeWithTag("handle").performTextInput(handle)
compose.onNodeWithTag("passphrase").performTextInput("a-long-enough-passphrase")
// 🔴 **The age gate, which nothing here filled.** `submit` is
// `enabled = … && (!registering || birthYear.length == 4)`, so without
// this the button is DISABLED — and `performClick()` on a disabled node
// does not throw, it does nothing. Every sign-up in this suite has been
// silently tapping a dead button since the gate landed, and the failure
// surfaced two minutes later as "nav-contacts did not appear" with no
// error anywhere on screen.
compose.onNodeWithTag("birth-year").performTextInput("1990")
compose.onNodeWithTag("submit").performClick()
awaitTag("nav-contacts", 120_000)
}
@@ -230,6 +230,14 @@ class FeatureUseTest {
compose.onNodeWithTag("email").performTextInput("$handle@example.invalid")
compose.onNodeWithTag("handle").performTextInput(handle)
compose.onNodeWithTag("passphrase").performTextInput("a-long-enough-passphrase")
// 🔴 **The age gate, which nothing here filled.** `submit` is
// `enabled = … && (!registering || birthYear.length == 4)`, so without
// this the button is DISABLED — and `performClick()` on a disabled node
// does not throw, it does nothing. Every sign-up in this suite has been
// silently tapping a dead button since the gate landed, and the failure
// surfaced two minutes later as "nav-contacts did not appear" with no
// error anywhere on screen.
compose.onNodeWithTag("birth-year").performTextInput("1990")
compose.onNodeWithTag("submit").performClick()
awaitTag("nav-contacts", 120_000)
}
+5 -1
View File
@@ -88,7 +88,11 @@ set -e
# 🔴 **A skip is not a pass.** Every one of these suites opens with
# `assumeTrue(relayReachable())`, so an unreachable relay produces a green
# build that exercised nothing. Say which happened.
SKIPS=$(grep -c "SKIPPED" /tmp/fnmsg-instrumented.log || true)
# ⚠️ Anchored to the TEST line, not a bare "SKIPPED". Gradle prints
# `> Task :app:preDebugAndroidTestBuild SKIPPED` for up-to-date tasks, so the
# loose grep reported four skipped tests on a run where none were skipped —
# a verdict written to stop a skip reading as a pass, inventing skips instead.
SKIPS=$(grep -cE "^io\.flamenet\..*SKIPPED" /tmp/fnmsg-instrumented.log || true)
RAN=$(grep -oE "Finished [0-9]+ test" /tmp/fnmsg-instrumented.log | tail -1 || echo "")
echo
echo "### $RAN, $SKIPS skipped"