Make the instrumented tests capable of failing
Three separate reasons these suites could never catch a regression, all
fixed here. None of them are about the app; all of them are about the tests
being unable to report the truth.
1. A skipped test reported as a pass. Thirteen guards were bare returns —
`if (!relayReachable()) return` — so with no relay the method returned
immediately and the runner recorded PASSED. Now `assumeTrue`, which
reports SKIPPED, and the run says "(2 skipped)" where it used to say
nothing at all. Only the guards that abandon a whole test were changed;
helper loop exits like `if (found) return` are control flow and were
left alone.
2. The relay was hardcoded to relay1.flamenet.io — production. Every run
that did work created real accounts there. Now `-e relay <host>`, still
defaulting to production so an unqualified run is unchanged.
3. Android blocks cleartext HTTP, so a debug build could not reach a relay
on the developer's machine even when one was running. A debug-only
network security config permits cleartext to 10.0.2.2, 127.0.0.1 and
localhost, and nothing else. src/debug, so no release APK is affected.
Together these took the sign-up suite from "2 skipped, reported green" to
"2 run, 2 failed" — which is worth more, because it is true. What it
fails on now (no compose hierarchy at launch) is the next thing to fix.
./gradlew :app:connectedDebugAndroidTest \
-Pandroid.testInstrumentationRunnerArguments.relay=10.0.2.2:18810
This commit is contained in:
@@ -0,0 +1,12 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!--
|
||||
Debug-only overlay. Points debug builds at a network security config that
|
||||
permits cleartext to the loopback addresses, so a local relay is reachable
|
||||
for end-to-end tests. Release builds never see this file.
|
||||
-->
|
||||
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
|
||||
xmlns:tools="http://schemas.android.com/tools">
|
||||
<application
|
||||
android:networkSecurityConfig="@xml/network_security_config"
|
||||
tools:replace="android:networkSecurityConfig" />
|
||||
</manifest>
|
||||
@@ -0,0 +1,22 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!--
|
||||
🔴 DEBUG BUILDS ONLY. This file lives in src/debug and is not part of a
|
||||
release APK, so nothing shipped to anybody is loosened by it.
|
||||
|
||||
Android blocks cleartext HTTP by default, which is right — but it also
|
||||
means a debug build cannot talk to a relay running on the developer's own
|
||||
machine, and so every end-to-end test either points at PRODUCTION or skips.
|
||||
Both of those are worse than a scoped exception: one creates real accounts
|
||||
for every test run, the other reports a pass for a test that never ran.
|
||||
|
||||
Scoped to the loopback addresses only. 10.0.2.2 is how the emulator reaches
|
||||
the host; 127.0.0.1 and localhost are the device itself. Nothing else is
|
||||
permitted, so this cannot quietly enable cleartext to a real server.
|
||||
-->
|
||||
<network-security-config>
|
||||
<domain-config cleartextTrafficPermitted="true">
|
||||
<domain includeSubdomains="false">10.0.2.2</domain>
|
||||
<domain includeSubdomains="false">127.0.0.1</domain>
|
||||
<domain includeSubdomains="false">localhost</domain>
|
||||
</domain-config>
|
||||
</network-security-config>
|
||||
Reference in New Issue
Block a user