obfs4 for Android, part 6: live interop with real lyrebird, both directions
scripts/obfs4-interop.sh builds a real lyrebird bridge and a real lyrebird client from the flamenet-elligator2 checkout and runs the Kotlin port against both. The bridge accepts the Kotlin client's handshake and echoes data; lyrebird's own client accepts the Kotlin bridge and its echo. 61 tests. Both directions are needed, and the falsification shows why: swapping the framing key halves by role leaves the CLIENT test green -- a client takes the first half either way -- and fails only the reverse one. Encoding the epoch hour as binary rather than a decimal string fails both. The probe-resistance test, which asserts the bridge stays silent for garbage, passes throughout, as it should. Both clients are pinned to the SAME reference rather than to each other. Two independent implementations agreeing with lyrebird is a real statement; agreeing with each other is not, and would hide anything they share. The runner's name check earned its keep twice in ten minutes. First it reported failure while all three tests had passed, because Gradle prints `name() PASSED` with parentheses and the pattern omitted them. Then, with that fixed, it caught the tests not running AT ALL: Gradle's up-to-date check does not know the Go counterparty changed, so a second run skips the task silently. Hence --rerun-tasks. An exit code alone would have called both of those green. The tests never skip quietly: absent configuration is a hard failure unless OBFS4_INTEROP_OPTOUT=1 is set explicitly. Verified both ways -- the bare run fails, the opted-out run passes 61. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hitt
co-authored by
Claude Opus 5
Claude Opus 5
parent
ac5802a839
commit
a7ce010403
@@ -0,0 +1,229 @@
|
||||
package io.flamenet.obfs4
|
||||
|
||||
import java.io.File
|
||||
import java.net.InetSocketAddress
|
||||
import java.net.ServerSocket
|
||||
import java.net.Socket
|
||||
import java.net.SocketTimeoutException
|
||||
import java.security.SecureRandom
|
||||
import org.json.JSONObject
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertEquals
|
||||
import kotlin.test.assertNotNull
|
||||
import kotlin.test.assertTrue
|
||||
|
||||
/**
|
||||
* Handshakes against a **real lyrebird bridge**, and a real lyrebird client
|
||||
* against this one.
|
||||
*
|
||||
* 🔴 **These are the only tests here that prove interoperability.** Everything
|
||||
* else compares against recorded reference output or against itself, and a
|
||||
* Kotlin-to-Kotlin test cannot catch a mistake both ends share — two
|
||||
* identically-wrong implementations agree perfectly. Here the counterparty is
|
||||
* the Tor Project's own code, deciding for itself whether to answer.
|
||||
*
|
||||
* ⚠️ **A bug shows up as SILENCE, not an error.** obfs4 is probe-resistant: a
|
||||
* bridge that dislikes a handshake says nothing at all. So every read has a
|
||||
* timeout and a timeout is reported as *"the bridge refused to answer"* — the
|
||||
* one diagnosis that is actually true — rather than as a network fault.
|
||||
*
|
||||
* ⚠️ **Both directions are needed.** Swapping the framing key halves by role is
|
||||
* fatal for a server and leaves the client test green, because a client takes
|
||||
* the first half either way. Only the reverse direction catches it.
|
||||
*
|
||||
* Run with `scripts/obfs4-interop.sh`, which builds and starts the counterparty.
|
||||
*/
|
||||
class InteropTest {
|
||||
|
||||
private val random = SecureRandom()
|
||||
|
||||
/**
|
||||
* 🔴 **Never skips quietly.** A silent skip is how a suite reports success
|
||||
* for a test that has not run in months. Absent configuration is a hard
|
||||
* failure unless the environment explicitly opts out, which forces the
|
||||
* decision to be visible in the runner rather than buried here.
|
||||
*/
|
||||
private fun env(name: String): String? {
|
||||
System.getenv(name)?.let { return it }
|
||||
if (System.getenv("OBFS4_INTEROP_OPTOUT") == "1") return null
|
||||
throw AssertionError(
|
||||
"$name is not set and OBFS4_INTEROP_OPTOUT=1 was not given either. " +
|
||||
"Run scripts/obfs4-interop.sh, or opt out explicitly.")
|
||||
}
|
||||
|
||||
private fun hex(s: String) = ByteArray(s.length / 2) {
|
||||
s.substring(it * 2, it * 2 + 2).toInt(16).toByte()
|
||||
}
|
||||
private fun hex(b: ByteArray) = b.joinToString("") { "%02x".format(it) }
|
||||
|
||||
private fun representableKeypair(): Triple<ByteArray, ByteArray, ByteArray> {
|
||||
// 🔴 Retried with FRESH entropy, never by re-tweaking a key that had no
|
||||
// representative: representability is a property of the key, so
|
||||
// retrying the same one differently would bias which keys get used.
|
||||
var attempts = 0
|
||||
while (true) {
|
||||
attempts++
|
||||
val candidate = ByteArray(32).also { random.nextBytes(it) }
|
||||
val pub = Ladder.dirtyPublicKey(candidate)
|
||||
val tweak = random.nextInt(256)
|
||||
val repr = Elligator2.representativeForPublicKey(pub, tweak)
|
||||
if (repr != null) return Triple(candidate, pub, repr)
|
||||
assertTrue(attempts < 100, "representable keys should arrive in a few tries")
|
||||
}
|
||||
}
|
||||
|
||||
private fun readSome(socket: Socket, stage: String): ByteArray {
|
||||
val buf = ByteArray(16384)
|
||||
val n = try { socket.getInputStream().read(buf) } catch (e: SocketTimeoutException) { -1 }
|
||||
if (n <= 0) throw AssertionError(
|
||||
"the bridge refused to answer during $stage. obfs4 bridges stay silent when a " +
|
||||
"handshake is invalid, so this is what a wire-format bug looks like — not a " +
|
||||
"network fault.")
|
||||
return buf.copyOfRange(0, n)
|
||||
}
|
||||
|
||||
// MARK: - Kotlin client → real lyrebird bridge
|
||||
|
||||
@Test
|
||||
fun `a handshake and an echo against a real lyrebird bridge`() {
|
||||
val paramsPath = env("OBFS4_INTEROP_PARAMS") ?: return
|
||||
val params = JSONObject(File(paramsPath).readText())
|
||||
val nodeId = hex(params.getString("node_id"))
|
||||
val identity = hex(params.getString("public_key"))
|
||||
val address = params.getString("address").split(":")
|
||||
|
||||
val (priv, pub, repr) = representableKeypair()
|
||||
val handshake = Obfs4ClientHandshake(
|
||||
nodeId, identity, priv, pub, repr, Obfs4Handshake.currentEpochHour())
|
||||
|
||||
val padLength = Obfs4Handshake.CLIENT_MIN_PAD_LENGTH + random.nextInt(500)
|
||||
val padding = ByteArray(padLength).also { random.nextBytes(it) }
|
||||
|
||||
Socket().use { socket ->
|
||||
socket.connect(InetSocketAddress(address[0], address[1].toInt()), 8000)
|
||||
// ⚠️ Without a read timeout a rejected handshake hangs the suite
|
||||
// forever, because the bridge's correct behaviour is to say nothing.
|
||||
socket.soTimeout = 8000
|
||||
socket.getOutputStream().write(handshake.opening(padding))
|
||||
socket.getOutputStream().flush()
|
||||
|
||||
var established: Obfs4ClientHandshake.Established? = null
|
||||
for (i in 0 until 8) {
|
||||
established = handshake.receive(readSome(socket, "the server handshake"))
|
||||
if (established != null) break
|
||||
}
|
||||
assertNotNull(established, "the bridge never completed its handshake")
|
||||
|
||||
val keys = Obfs4Session.directionKeys(established.keySeed, isClient = true)
|
||||
val plane = Obfs4DataPlane(keys.first, keys.second)
|
||||
// ⚠️ Whatever followed the handshake in the same read is already
|
||||
// framed traffic — usually the bridge's PRNG seed. Dropping it
|
||||
// desynchronises the frame decoder immediately.
|
||||
plane.receive(established.leftover, isClient = true)
|
||||
|
||||
val message = "flamenet obfs4 kotlin interop".toByteArray()
|
||||
socket.getOutputStream().write(plane.send(message, 97))
|
||||
socket.getOutputStream().flush()
|
||||
|
||||
var echoed = ByteArray(0)
|
||||
for (i in 0 until 8) {
|
||||
if (echoed.size >= message.size) break
|
||||
echoed += plane.receive(readSome(socket, "the echo"), isClient = true)
|
||||
}
|
||||
assertEquals(hex(message), hex(echoed),
|
||||
"the bridge accepted the handshake but the data plane disagreed")
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* ⚠️ The mirror image: a handshake the bridge should refuse. It must go
|
||||
* silent rather than answer, which is what probe resistance means.
|
||||
*/
|
||||
@Test
|
||||
fun `a bridge ignores a handshake it cannot verify`() {
|
||||
val paramsPath = env("OBFS4_INTEROP_PARAMS") ?: return
|
||||
val params = JSONObject(File(paramsPath).readText())
|
||||
val address = params.getString("address").split(":")
|
||||
|
||||
Socket().use { socket ->
|
||||
socket.connect(InetSocketAddress(address[0], address[1].toInt()), 8000)
|
||||
socket.soTimeout = 6000
|
||||
socket.getOutputStream().write(ByteArray(700).also { random.nextBytes(it) })
|
||||
socket.getOutputStream().flush()
|
||||
val n = try { socket.getInputStream().read(ByteArray(1024)) }
|
||||
catch (e: SocketTimeoutException) { -1 }
|
||||
assertTrue(n <= 0,
|
||||
"the bridge replied to an invalid handshake with $n bytes. A bridge that " +
|
||||
"answers unverified clients is one a scanner can confirm exists.")
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Real lyrebird client → Kotlin bridge
|
||||
|
||||
/**
|
||||
* 🔴 **The only thing that can prove the server role.** The client test
|
||||
* stays green through bugs that are fatal for a server.
|
||||
*/
|
||||
@Test
|
||||
fun `a real lyrebird client accepts the kotlin bridge`() {
|
||||
val clientPath = env("OBFS4_INTEROP_CLIENT") ?: return
|
||||
|
||||
val identityPrivate = ByteArray(32).also { random.nextBytes(it) }
|
||||
// ⚠️ CLEAN, not dirty. Only session keys keep their cofactor, because
|
||||
// only those travel as representatives.
|
||||
val identityPublic = Ladder.publicKey(identityPrivate)
|
||||
val nodeId = ByteArray(20).also { random.nextBytes(it) }
|
||||
val (sessionPriv, sessionPub, sessionRepr) = representableKeypair()
|
||||
val filter = ReplayFilter()
|
||||
|
||||
ServerSocket(0, 4, java.net.InetAddress.getByName("127.0.0.1")).use { listener ->
|
||||
val cert = java.util.Base64.getEncoder()
|
||||
.encodeToString(nodeId + identityPublic).dropLast(2)
|
||||
val message = "kotlin bridge, real client"
|
||||
val process = ProcessBuilder(
|
||||
clientPath, "-address", "127.0.0.1:${listener.localPort}",
|
||||
"-cert", cert, "-message", message,
|
||||
).redirectErrorStream(true).start()
|
||||
|
||||
listener.accept().use { conn ->
|
||||
conn.soTimeout = 15000
|
||||
val handshake = Obfs4ServerHandshake(
|
||||
nodeId, identityPrivate, identityPublic,
|
||||
sessionPriv, sessionPub, sessionRepr, Obfs4Handshake.currentEpochHour())
|
||||
val padding = ByteArray(random.nextInt(400)).also { random.nextBytes(it) }
|
||||
|
||||
var accepted: Obfs4ServerHandshake.Accepted? = null
|
||||
for (i in 0 until 8) {
|
||||
val chunk = readSome(conn, "the client handshake")
|
||||
accepted = handshake.receive(chunk, padding, filter)
|
||||
if (accepted != null) break
|
||||
}
|
||||
assertNotNull(accepted, "the Kotlin bridge never accepted the client's handshake")
|
||||
|
||||
conn.getOutputStream().write(accepted.reply)
|
||||
val keys = Obfs4Session.directionKeys(accepted.keySeed, isClient = false)
|
||||
val plane = Obfs4DataPlane(keys.first, keys.second)
|
||||
// Faithful to the reference: a bridge's first frame hands the
|
||||
// client a seed so both ends pad from the same distribution.
|
||||
conn.getOutputStream().write(
|
||||
plane.sendPrngSeed(ByteArray(24).also { random.nextBytes(it) }))
|
||||
conn.getOutputStream().flush()
|
||||
|
||||
var echoed = 0
|
||||
while (echoed < message.length) {
|
||||
val payload = plane.receive(readSome(conn, "the client's traffic"),
|
||||
isClient = false)
|
||||
if (payload.isNotEmpty()) {
|
||||
conn.getOutputStream().write(plane.send(payload, 63))
|
||||
conn.getOutputStream().flush()
|
||||
echoed += payload.size
|
||||
}
|
||||
}
|
||||
assertEquals(message.length, echoed, "the bridge did not echo the whole message")
|
||||
}
|
||||
|
||||
val output = process.inputStream.readBytes().decodeToString()
|
||||
assertEquals(0, process.waitFor(), "lyrebird rejected the Kotlin bridge:\n$output")
|
||||
}
|
||||
}
|
||||
}
|
||||
Executable
+91
@@ -0,0 +1,91 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run the Kotlin obfs4 port against real lyrebird, in both directions.
|
||||
#
|
||||
# 🔴 The only check that proves interoperability. Everything else in `:obfs4`
|
||||
# compares against recorded vectors or against itself, and two identically-wrong
|
||||
# ends agree perfectly.
|
||||
#
|
||||
# ⚠️ A wire-format bug shows up as SILENCE, not an error: obfs4 bridges do not
|
||||
# answer handshakes they cannot verify. The Kotlin side uses read timeouts and
|
||||
# reports them as "the bridge refused to answer".
|
||||
#
|
||||
# ⚠️ The Go counterparty lives in the flamenet-elligator2 checkout, which also
|
||||
# holds the Swift port. Both clients are pinned to the SAME reference rather
|
||||
# than to each other — two independent implementations agreeing with lyrebird is
|
||||
# a real statement; agreeing with each other is not.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
REFERENCE="${FN_OBFS4_REFERENCE:-../flamenet-elligator2/Reference}"
|
||||
if [[ ! -d "$REFERENCE" ]]; then
|
||||
echo "Cannot find the Go reference at $REFERENCE."
|
||||
echo "Clone flamenet-elligator2 beside this repo, or set FN_OBFS4_REFERENCE."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
WORK="$(mktemp -d)"
|
||||
PARAMS="$WORK/params.json"
|
||||
SERVER_LOG="$WORK/server.log"
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "${SERVER_PID:-}" ]] && kill -0 "$SERVER_PID" 2>/dev/null; then
|
||||
kill "$SERVER_PID" 2>/dev/null || true
|
||||
wait "$SERVER_PID" 2>/dev/null || true
|
||||
fi
|
||||
# The bridge log is the other half of every diagnosis: it says whether the
|
||||
# handshake was accepted or refused, which the silent client cannot.
|
||||
if [[ -s "$SERVER_LOG" ]]; then echo "--- bridge log ---"; cat "$SERVER_LOG"; fi
|
||||
rm -rf "$WORK"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
echo "building the lyrebird bridge and client..."
|
||||
(cd "$REFERENCE" && go build -o "$WORK/interopserver" ./interopserver)
|
||||
(cd "$REFERENCE" && go build -o "$WORK/interopclient" ./interopclient)
|
||||
|
||||
"$WORK/interopserver" -params "$PARAMS" -addr 127.0.0.1:0 >/dev/null 2>"$SERVER_LOG" &
|
||||
SERVER_PID=$!
|
||||
|
||||
# ⚠️ Poll for the params file rather than sleeping. A fixed sleep is either
|
||||
# flaky or slow, and on a loaded machine it is both.
|
||||
for _ in $(seq 1 100); do
|
||||
[[ -f "$PARAMS" ]] && break
|
||||
kill -0 "$SERVER_PID" 2>/dev/null || { echo "the bridge died on startup"; exit 1; }
|
||||
sleep 0.1
|
||||
done
|
||||
[[ -f "$PARAMS" ]] || { echo "the bridge never published its parameters"; exit 1; }
|
||||
|
||||
# ⚠️ sed, not python3 or jq: the CI images for this repo cannot be assumed to
|
||||
# have either, and that assumption has cost red pipelines before.
|
||||
ADDRESS="$(sed -n 's/.*"address"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$PARAMS")"
|
||||
[[ -n "$ADDRESS" ]] || { echo "could not read the bridge address from $PARAMS"; exit 1; }
|
||||
echo "bridge up at $ADDRESS"
|
||||
|
||||
# 🔴 `--rerun-tasks`, because Gradle's up-to-date check does not know the
|
||||
# counterparty changed. Without it a second run skips the task entirely and
|
||||
# prints nothing — the tests do not fail, they simply never happen. Caught by
|
||||
# the name check below, which is the second time that check has earned its keep
|
||||
# in ten minutes.
|
||||
OBFS4_INTEROP_PARAMS="$PARAMS" OBFS4_INTEROP_CLIENT="$WORK/interopclient" \
|
||||
./gradlew :obfs4:test --tests '*InteropTest*' --rerun-tasks --console=plain 2>&1 \
|
||||
| tee "$WORK/test.log"
|
||||
|
||||
# 🔴 A Gradle filter that matches nothing still exits 0, which would make "no
|
||||
# interop coverage at all" look exactly like success.
|
||||
for name in "a handshake and an echo against a real lyrebird bridge" \
|
||||
"a bridge ignores a handshake it cannot verify" \
|
||||
"a real lyrebird client accepts the kotlin bridge"; do
|
||||
# ⚠️ Gradle prints `name() PASSED`, with parentheses. Matching on
|
||||
# "$name PASSED" silently matches nothing — which this check caught on
|
||||
# itself, reporting failure while all three tests had passed.
|
||||
grep -qF "$name() PASSED" "$WORK/test.log" || {
|
||||
echo
|
||||
echo "FAILED: '$name' did not pass. A filter that matches nothing exits 0,"
|
||||
echo "so each test is checked by name rather than trusting the exit code."
|
||||
exit 1
|
||||
}
|
||||
done
|
||||
|
||||
echo
|
||||
echo "ANDROID INTEROP OK: both directions -- a real lyrebird bridge accepted the"
|
||||
echo "Kotlin client, and a real lyrebird client accepted the Kotlin bridge."
|
||||
Reference in New Issue
Block a user