Paul Hitt
d8d48d8651
🔴 **The relay probe had no deadline and no way to say it failed.** `discovered` is null both while the probe is in flight and after it has come back with nothing, so `identityDescription` shows "Checking what this server uses…" in both cases. A relay that never answers leaves the sign-up form on that sentence permanently — no error, no retry, nothing to act on. Found by the instrumented suite on a clean emulator: nine of ten tests failed with `nav-contacts did not appear within 120000ms`, and the screen dump showed the form still checking. ⚠️ **The reason it never answered is worth writing down.** The app was pointed at `http://10.0.2.2:18810`, and Android has blocked cleartext HTTP by default since API 28 — there is no `usesCleartextTraffic` and no network security config here. The request never left the device. 🔴 That refusal is CORRECT and is not what changed. `SiteConfig` already says why: a bare hostname becoming `http://` would hand key material to anyone on the path "and it would work, which is the worst outcome." What was wrong is that it failed silently, so somebody self-hosting who omits the scheme gets a dead screen and no idea why. Now: bounded at fifteen seconds — far longer than a healthy relay needs — and a failed probe is a distinct state that says "Couldn't reach that server. Check the address — and note it has to be https, not http." Same defect as android#19 one screen earlier, which I fixed this morning without noticing the probe had it too. Also adds `scripts/instrumented.sh`, which closes two ways an on-device run silently points at production: the suites DEFAULT to live hosts when no relay is named, and a saved emulator snapshot restores whatever the AVD was last doing — observed today as a snapshot signed in as `fnreviewbeta` against relay.flamenet.io, under a run told to use a local relay. The script has no default, refuses both live hosts by name, cold boots with `-no-snapshot-load`, and exits 3 on a skip because a skip is not a pass.
99 lines
4.2 KiB
Bash
Executable File
99 lines
4.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Run the on-device suites against a relay you named, on an emulator with no
|
|
# history.
|
|
#
|
|
# 🔴 **Two ways an instrumented run silently points at PRODUCTION, and this
|
|
# script closes both.**
|
|
#
|
|
# 1. The suites DEFAULT to a live host when no relay argument is passed —
|
|
# `FeatureUseTest` to relay1.flamenet.io, `LiveSignUpFleetTest` to
|
|
# relay.flamenet.io. A plain `./gradlew connectedAndroidTest` therefore
|
|
# signs accounts up on somebody's real server, and nothing in the output
|
|
# says so.
|
|
# 2. A SAVED EMULATOR SNAPSHOT restores whatever the AVD was last doing.
|
|
# Observed 2026-09-05: a snapshot signed in as `fnreviewbeta` against
|
|
# relay.flamenet.io, restored under a test run that had been told to use a
|
|
# local relay. Every one of the ten tests failed — they could not find the
|
|
# sign-up screen, because the app was already signed in — and the reason
|
|
# was invisible until the failure's screen dump was read.
|
|
#
|
|
# Usage: scripts/instrumented.sh 10.0.2.2:18810 [ClassName ...]
|
|
set -euo pipefail
|
|
|
|
RELAY="${1:-}"
|
|
[ -n "$RELAY" ] || {
|
|
cat >&2 <<'MSG'
|
|
usage: scripts/instrumented.sh <relay-host:port> [TestClass ...]
|
|
|
|
Name the relay. There is deliberately no default: the suites have their own,
|
|
and theirs are live servers.
|
|
|
|
10.0.2.2:18810 the host machine's local relay, as seen from the emulator
|
|
MSG
|
|
exit 2
|
|
}
|
|
shift || true
|
|
|
|
# 🔴 Refused rather than warned about. A run that creates accounts on the
|
|
# production relay cannot be undone by noticing afterwards.
|
|
case "$RELAY" in
|
|
*relay.flamenet.io*|*relay1.flamenet.io*)
|
|
echo "REFUSED: $RELAY is a live relay. These suites create accounts." >&2
|
|
echo "Point them at a local relay, or edit this check if you truly mean it." >&2
|
|
exit 1;;
|
|
esac
|
|
|
|
CLASSES="${*:-io.flamenet.messenger.app.FeatureUseTest,io.flamenet.messenger.app.CrudSurfaceTest,io.flamenet.messenger.app.ChatFlowTest}"
|
|
CLASSES="${CLASSES// /,}"
|
|
|
|
export ANDROID_HOME="${ANDROID_HOME:-$HOME/Library/Android/sdk}"
|
|
export JAVA_HOME="${JAVA_HOME:-/opt/homebrew/opt/openjdk@21}"
|
|
ADB="$ANDROID_HOME/platform-tools/adb"
|
|
AVD="${FN_AVD:-fnmsg-android-test}"
|
|
|
|
# ⚠️ `-no-snapshot-load` is the point of this block, and `-no-snapshot-save`
|
|
# keeps the next run clean too. A warm boot is faster and carries the last
|
|
# session's account with it.
|
|
if ! "$ADB" shell getprop sys.boot_completed 2>/dev/null | grep -q 1; then
|
|
echo "### cold-booting $AVD (no snapshot)"
|
|
nohup "$ANDROID_HOME/emulator/emulator" -avd "$AVD" \
|
|
-no-snapshot-load -no-snapshot-save -no-boot-anim \
|
|
-netdelay none -netspeed full >/tmp/fnmsg-emu.log 2>&1 &
|
|
until "$ADB" shell getprop sys.boot_completed 2>/dev/null | grep -q 1; do sleep 5; done
|
|
else
|
|
echo "### emulator already up — clearing the app rather than trusting it"
|
|
"$ADB" uninstall io.flamenet.messenger >/dev/null 2>&1 || true
|
|
"$ADB" uninstall io.flamenet.messenger.test >/dev/null 2>&1 || true
|
|
fi
|
|
|
|
echo "### signed-in account on the device before we start:"
|
|
"$ADB" shell pm list packages 2>/dev/null | grep -i flamenet || echo " (nothing installed — good)"
|
|
|
|
echo "### relay $RELAY · classes $CLASSES"
|
|
set +e
|
|
./gradlew --no-daemon :app:connectedDebugAndroidTest \
|
|
-Pandroid.testInstrumentationRunnerArguments.relay="$RELAY" \
|
|
-Pandroid.testInstrumentationRunnerArguments.class="$CLASSES" 2>&1 \
|
|
| tee /tmp/fnmsg-instrumented.log \
|
|
| grep -E "FAILED|PASSED|SKIPPED|BUILD|Finished [0-9]+ test|Tests on"
|
|
STATUS=${PIPESTATUS[0]}
|
|
set -e
|
|
|
|
# 🔴 **A skip is not a pass.** Every one of these suites opens with
|
|
# `assumeTrue(relayReachable())`, so an unreachable relay produces a green
|
|
# build that exercised nothing. Say which happened.
|
|
SKIPS=$(grep -c "SKIPPED" /tmp/fnmsg-instrumented.log || true)
|
|
RAN=$(grep -oE "Finished [0-9]+ test" /tmp/fnmsg-instrumented.log | tail -1 || echo "")
|
|
echo
|
|
echo "### $RAN, $SKIPS skipped"
|
|
if [ "$SKIPS" != "0" ] && [ "$STATUS" = "0" ]; then
|
|
echo "### VERDICT: SKIPPED — the relay was unreachable. This is NOT a pass."
|
|
exit 3
|
|
elif [ "$STATUS" = "0" ]; then
|
|
echo "### VERDICT: PASSED on device"
|
|
else
|
|
echo "### VERDICT: FAILED — see /tmp/fnmsg-instrumented.log"
|
|
echo "### and app/build/reports/androidTests/connected/debug/index.html"
|
|
fi
|
|
exit $STATUS
|