extant2000/flamenet-messenger-android
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

Android: contacts, conversations, and a message that made the round trip

Browse Source 
The app now holds a conversation. A message typed into it reaches a different
program through the real relay, that program answers, and the answer appears in
the thread -- verified three times, end to end, encrypted the whole way.

96 JVM tests, 37 cross-engine, 11 on-device, plus live round trips against the
throwaway fleet.

What landed: a fan-out send that reaches every device a person has, inbox
polling, message history encrypted at rest, contacts, presence, and a session
that survives a restart. Plus `:interop:peer` -- a Flamenet account you can
drive from a shell, so the correspondent on the other end is not this app
talking to itself.

Three wire shapes were wrong, and only a real relay could say so:

  - `prekeys`, not `opks`. The relay answered 200 and stored nothing, so the
    device was published and could not be started with.
  - The send body names the recipient once at the top, not per envelope.
  - The buddy list is `id`/`presence`/`screen_name`, not `user_id`/`status`.

The second and third at least failed loudly. The first did not, which is why
the live tests exist.

Decisions worth stating:

A send returns partial success rather than throwing. One of someone's three
devices being unreachable must not stop the message reaching the other two, and
the UI says "delivered to 2 of 3" rather than "sent".

The inbox watermark advances per message, after it is decrypted AND stored --
never once for the batch, which would lose everything after a crash. An
undecryptable message advances it too: it is never going to decrypt, and
leaving it would re-fail on every poll for ever.

A sent line is recorded only once a device took it. Writing it first produces a
thread showing messages the recipient never got, which is worse than an error --
the sender believes they said something.

Opening a thread marks it read locally and tells the relay nothing. A read
receipt is a disclosure, and this product does not make one by default.

History is keyed by person, not device. Someone's phone and tablet are one
conversation, and filing by device would split it into threads each holding
half the story.

Also improves how instrumented failures report: the assertion now prints what
the screen was showing. That turned a bare 90-second timeout into "No value for
user_id" and then "Failed to connect to relay1" -- one a bug, one the emulator's
network, and previously indistinguishable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hitt
2026-08-23 17:55:49 -04:00
co-authored by Claude Opus 5
parent 5f983c1455
commit d30a77dd1e
13 changed files with 1512 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
client/src/main/kotlin/io/flamenet/client/Conversations.kt
+141
View File
@@ -0,0 +1,141 @@
package io.flamenet.client
import io.flamenet.e2e.DeviceIdentity
import io.flamenet.e2e.E2ESession
import io.flamenet.e2e.Envelope
import io.flamenet.e2e.PeerBundle
import io.flamenet.e2e.Vault
/**
* Sending and receiving, on top of sessions the vault keeps.
*
* ⚠️ **A person is not a device.** Everyone may be signed in on a phone, a
* tablet and the web at once, and a message encrypted to one of them is
* invisible on the other two. So a send is a fan-out: one envelope per
* recipient device, each under its own ratchet. A client that encrypts to
* "the" device works perfectly in testing, where everybody has one.
*
* ⚠️ **Every send and every receive moves ratchet state, and unsaved ratchet
* state is a broken conversation.** Each operation persists before it returns.
* That costs a write per message and buys the difference between an app that
* survives being killed and one that has to be reinstalled.
*/
class Conversations(
private val identity: DeviceIdentity,
private val vault: Vault,
private val relay: RelayClient,
) {
/** One decrypted message, and where it came from. */
data class Received(
val id: Long,
val fromUser: Long,
val fromDevice: String,
val text: String,
)
/** What happened to one send. Partial success is the normal case, not an error. */
data class Sent(val delivered: List<String>, val failed: Map<String, String>) {
val reachedSomeone: Boolean get() = delivered.isNotEmpty()
}
/**
* Send to every device the recipient has.
*
* ⚠️ Returns partial success rather than throwing. One of someone's three
* devices being unreachable — a drained prekey pool, a stale bundle — must
* not stop the message reaching the other two. Throwing on the first
* failure would turn "your tablet did not get it" into "the message was
* not sent", which is both wrong and unrecoverable.
*/
fun send(toUserId: Long, text: String): Sent {
val bundles = relay.fetchKeys(toUserId)
val delivered = mutableListOf<String>()
val failed = LinkedHashMap<String, String>()
val outgoing = mutableListOf<RelayClient.ToDevice>()
for (bundle in bundles) {
if (bundle.deviceId == identity.deviceId) continue // never talk to ourselves
try {
val session = sessionFor(bundle)
val envelope = session.encrypt(text)
// Saved BEFORE the network call. If the relay accepts the
// envelope and we then crash, the peer has a message our
// ratchet has not advanced past, and nothing either side can
// do recovers it. The reverse — saved, not sent — costs one
// unused message key and nothing else.
vault.saveSession(bundle.deviceId, session)
outgoing += RelayClient.ToDevice(bundle.deviceId, envelope)
delivered += bundle.deviceId
} catch (e: Exception) {
failed[bundle.deviceId] = e.message ?: e::class.simpleName.orEmpty()
}
}
if (outgoing.isNotEmpty()) relay.sendEnvelopes(toUserId, outgoing, identity.deviceId)
return Sent(delivered, failed)
}
/**
* Read the inbox, decrypt what we can, and advance the watermark.
*
* ⚠️ The watermark advances **per message, after it is decrypted and its
* session is stored** — not once for the batch. Advancing for the batch up
* front means a crash halfway loses every message after the crash, and the
* relay will never offer those ids again.
*
* ⚠️ A message that will not decrypt still advances the watermark. It is
* never going to decrypt — the ratchet has moved on, or it was addressed to
* a device identity we replaced — and leaving it in place would make the
* client re-fetch and re-fail it on every poll, for ever.
*/
fun poll(limit: Int = 100): List<Received> {
val incoming = relay.pollInbox(identity.deviceId, vault.inboxWatermark, limit)
val out = mutableListOf<Received>()
for (message in incoming) {
try {
val envelope = Envelope.fromPayload(message.payload)
val session = existingSession(message.fromDevice)
?: E2ESession.respond(identity, envelope)
val text = session.decrypt(envelope)
vault.saveSession(message.fromDevice, session)
out += Received(message.id, message.fromUser, message.fromDevice, text)
} catch (_: Exception) {
// Deliberately swallowed and skipped. One undecryptable message
// is not a reason to stop reading the inbox.
}
vault.inboxWatermark = message.id
}
return out
}
/** How many prekeys we have left with the relay, and a top-up when it is low. */
fun replenishPreKeysIfLow(threshold: Int = 5, batch: Int = 20): Int {
val mine = relay.fetchKeys(myUserId ?: return 0)
.firstOrNull { it.deviceId == identity.deviceId } ?: return 0
// The relay pops one per fetch, so seeing none here means the pool was
// empty before this call, not that this call drained it.
if (mine.oneTimePreKey != null) return 0
val fresh = identity.generateOneTimePreKeys(batch)
relay.uploadPreKeys(identity.deviceId, fresh)
vault.saveIdentity(identity)
return fresh.size
}
/** Set once known; only [replenishPreKeysIfLow] needs it. */
var myUserId: Long? = null
private fun existingSession(peerDeviceId: String): E2ESession? =
runCatching { vault.loadSession(peerDeviceId) }.getOrNull()
/**
* The session for a peer device: the stored one, or a new handshake.
*
* ⚠️ Starting a new session when a stored one exists is not a fresh start,
* it is a broken conversation — the peer keeps ratcheting the old one and
* nothing either side sends will open.
*/
private fun sessionFor(bundle: PeerBundle): E2ESession =
existingSession(bundle.deviceId) ?: E2ESession.initiate(identity, bundle)
}
client/src/main/kotlin/io/flamenet/client/MessengerApi.kt
+163
View File
@@ -0,0 +1,163 @@
package io.flamenet.client
import okhttp3.HttpUrl
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import org.json.JSONArray
import org.json.JSONObject
/**
* The non-cryptographic half of the product: who your contacts are, who is
* online, what you have been notified about, who you have blocked.
*
* ⚠️ **These routes take the account credential, not the relay token.** The
* `/e2e/…` routes deliberately refuse the account password because a
* credential good for every device is too much authority for key material.
* These routes are the opposite case — they are about the account, not a
* device — so Basic is correct here and a device-scoped token would be wrong.
*/
class MessengerApi(
baseUrl: String,
private val username: String,
appPassword: String,
private val http: OkHttpClient = RelayClient.defaultHttpClient(),
) {
private val base: HttpUrl = baseUrl.trimEnd('/').toHttpUrl()
private val appPassword = appPassword.replace(" ", "")
/**
* Someone in your list.
*
* ⚠️ `handle` is the name they signed up with and the only one another
* person can type to reach them. It was null on this relay for a while,
* having started life as an OSCAR screen name, which meant you could not
* find your own contacts by the name you knew them by. Show it.
*
* `presence` is their chosen status, not an inference from activity.
*/
data class Buddy(
val userId: Long,
val displayName: String,
val handle: String,
val presence: String,
)
data class Notification(
val id: Long,
val kind: String,
val body: String,
val read: Boolean,
)
fun buddies(): List<Buddy> {
val list = get("buddies").optJSONArray("buddies") ?: JSONArray()
return (0 until list.length()).map {
val b = list.getJSONObject(it)
// ⚠️ `id`, `display_name`, `screen_name`, `presence` — this route
// does NOT use the `user_id`/`status` shape the §8 routes use, and
// guessing otherwise threw "No value for user_id" at the exact
// moment a contact was added.
Buddy(
userId = b.getLong("id"),
displayName = b.optString("display_name", ""),
handle = b.optString("screen_name", ""),
presence = b.optString("presence", "offline"),
)
}
}
/**
* ⚠️ By username, not by id. A person types a name; the relay is the only
* thing that can turn that into an id, and doing the lookup client-side
* would need a directory this product deliberately does not have.
*/
fun addBuddy(username: String) {
post("buddies", JSONObject().put("username", username))
}
fun removeBuddy(userId: Long) {
delete("buddies/$userId")
}
fun setStatus(status: String) {
post("status", JSONObject().put("status", status))
}
fun notifications(): List<Notification> {
val page = get("notifications")
val list = page.optJSONArray("notifications") ?: JSONArray()
return (0 until list.length()).map {
val n = list.getJSONObject(it)
Notification(
id = n.optLong("id"),
kind = n.optString("kind", ""),
body = n.optString("body", ""),
read = n.optBoolean("read", false),
)
}
}
fun markNotificationRead(id: Long) = post("notifications/$id/read", JSONObject()).let { }
fun markAllNotificationsRead() = post("notifications/read_all", JSONObject()).let { }
fun blocked(): List<Long> {
val list = get("blocks").optJSONArray("blocked") ?: JSONArray()
return (0 until list.length()).map { list.getJSONObject(it).getLong("user_id") }
}
fun setBlocked(userId: Long, blocked: Boolean) {
if (blocked) post("blocks/$userId", JSONObject()) else delete("blocks/$userId")
}
// -- Internals -----------------------------------------------------------
private fun url(path: String) = base.newBuilder()
.addPathSegments("api/flamenet/v1/messenger/$path").build()
private fun get(path: String) = send(
Request.Builder().url(url(path)).get().header("Accept", "application/json"))
private fun post(path: String, body: JSONObject) = send(
Request.Builder().url(url(path))
.post(body.toString().toRequestBody(JSON))
.header("Accept", "application/json"))
private fun delete(path: String) = send(
Request.Builder().url(url(path)).delete().header("Accept", "application/json"))
private fun send(builder: Request.Builder): JSONObject {
val request = builder
.header("Authorization", "Basic " + Wire.b64(
"$username:$appPassword".toByteArray(Charsets.UTF_8)))
.build()
val response = try {
http.newCall(request).execute()
} catch (e: Exception) {
throw RelayException.Transport(e.message ?: e.toString(), e)
}
response.use {
val text = it.body?.string().orEmpty()
if (it.isSuccessful) {
return if (text.isBlank()) JSONObject() else runCatching { JSONObject(text) }
.getOrElse { throw RelayException.Malformed("the relay's reply was not JSON") }
}
val message = runCatching { JSONObject(text).optString("message") }
.getOrNull().orEmpty().ifBlank { text }
throw when (it.code) {
401, 403 -> RelayException.Unauthorized(message)
404 -> RelayException.NotFound(message.ifBlank { request.url.encodedPath })
429 -> RelayException.RateLimited(it.header("Retry-After")?.toIntOrNull())
else -> RelayException.Http(it.code, message)
}
}
}
private companion object {
val JSON = "application/json".toMediaType()
}
}
client/src/main/kotlin/io/flamenet/client/RelayClient.kt
+22 -13
View File
@@ -110,21 +110,30 @@ class RelayClient(
// -- Messages ------------------------------------------------------------
/** One addressed envelope, as `POST /e2e/messages` carries it. */
data class Addressed(val toUser: Long, val toDevice: String, val payload: String) {
constructor(toUser: Long, toDevice: String, envelope: Envelope) :
this(toUser, toDevice, envelope.toPayload())
/** One envelope for one of the recipient's devices. */
data class ToDevice(val deviceId: String, val payload: String) {
constructor(deviceId: String, envelope: Envelope) : this(deviceId, envelope.toPayload())
}
fun sendEnvelopes(envelopes: List<Addressed>): JSONObject =
postJson("/e2e/messages", JSONObject().put("messages", JSONArray().also { array ->
envelopes.forEach {
array.put(JSONObject()
.put("to_user", it.toUser)
.put("to_device", it.toDevice)
.put("payload", it.payload))
}
}))
/**
* `POST /e2e/messages` — one recipient, one envelope per device of theirs.
*
* ⚠️ The recipient is named **once, at the top**, not per envelope. A body
* that repeats `to_user` inside each message is rejected outright as a
* malformed body, which at least fails loudly — the shape being wrong in a
* way the relay tolerates is the version that costs a week.
*/
fun sendEnvelopes(toUser: Long, envelopes: List<ToDevice>, fromDevice: String): JSONObject =
postJson("/e2e/messages", JSONObject()
.put("to", toUser)
.put("from_device", fromDevice)
.put("messages", JSONArray().also { array ->
envelopes.forEach {
array.put(JSONObject()
.put("to_device", it.deviceId)
.put("payload", it.payload))
}
}))
/** One envelope from the inbox, with the server id that acknowledges it. */
data class Incoming(val id: Long, val fromUser: Long, val fromDevice: String, val payload: String)
client/src/test/kotlin/io/flamenet/client/LiveConversationTest.kt
+183
View File
@@ -0,0 +1,183 @@
package io.flamenet.client
import io.flamenet.e2e.DeviceIdentity
import io.flamenet.e2e.Vault
import javax.crypto.KeyGenerator
import javax.crypto.SecretKey
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
/**
* Two accounts, one relay, a real conversation.
*
* This is the test the whole stack exists to pass. Everything below it —
* X3DH, the ratchet, the vault, the transport — has been proven in isolation;
* this is the first thing that proves they were assembled correctly.
*
* ⚠️ Opt-in via `FN_LIVE_FLEET`, and pointed at the throwaway fleet. It creates
* accounts and sends mail.
*
* FN_LIVE_FLEET=https://relay1.flamenet.io ./gradlew :client:test
*/
class LiveConversationTest {
private val fleet: String? = System.getenv("FN_LIVE_FLEET")
/** A signed-up account with a registered device, ready to talk. */
private class Person(
val handle: String,
val userId: Long,
val identity: DeviceIdentity,
val vault: Vault,
val relay: RelayClient,
val conversations: Conversations,
val api: MessengerApi,
)
private class MapStorage : Vault.Storage {
private val records = LinkedHashMap<String, ByteArray>()
override fun read(name: String) = records[name]
override fun write(name: String, bytes: ByteArray) { records[name] = bytes }
override fun delete(name: String) { records.remove(name) }
override fun names() = records.keys.toList()
}
private fun key(): SecretKey = KeyGenerator.getInstance("AES").apply { init(256) }.generateKey()
private fun signUp(base: String, tag: String, preKeys: Int = 20): Person {
val handle = "live$tag" + (System.nanoTime() % 10_000_000)
val account = RelayAccounts(base).signUp(handle, "$handle@example.invalid", "a-long-enough-passphrase")
val identity = DeviceIdentity.create()
val vault = Vault(MapStorage(), key())
vault.saveIdentity(identity)
val relay = RelayClient(base, account.username, account.appPassword)
relay.useRelayAuth(identity)
relay.registerDevice(identity, identity.generateOneTimePreKeys(preKeys))
vault.saveIdentity(identity)
return Person(
handle = handle,
userId = account.userId,
identity = identity,
vault = vault,
relay = relay,
conversations = Conversations(identity, vault, relay).also { it.myUserId = account.userId },
api = MessengerApi(base, account.username, account.appPassword),
)
}
@Test
fun `two people exchange messages through a real relay`() {
val base = fleet ?: return
val alice = signUp(base, "a")
val bob = signUp(base, "b")
// Alice opens the conversation. Bob has never heard of her, so this is
// a prekey envelope and it has to stand on its own.
val sent = alice.conversations.send(bob.userId, "hello from the Android engine 👋")
assertTrue(sent.reachedSomeone, "nothing was delivered: ${sent.failed}")
assertEquals(listOf(bob.identity.deviceId), sent.delivered)
val received = bob.conversations.poll()
assertEquals(1, received.size, "Bob's inbox had ${received.size} messages")
assertEquals("hello from the Android engine 👋", received[0].text)
assertEquals(alice.userId, received[0].fromUser)
assertEquals(alice.identity.deviceId, received[0].fromDevice)
// Bob replies on the session he just established, which is the step
// that turns a handshake into a conversation.
assertTrue(bob.conversations.send(alice.userId, "and hello back").reachedSomeone)
val back = alice.conversations.poll()
assertEquals(1, back.size)
assertEquals("and hello back", back[0].text)
// Several turns, so the ratchet is exercised past the handshake.
for (turn in 1..3) {
alice.conversations.send(bob.userId, "alice $turn")
assertEquals("alice $turn", bob.conversations.poll().single().text)
bob.conversations.send(alice.userId, "bob $turn")
assertEquals("bob $turn", alice.conversations.poll().single().text)
}
}
/**
* ⚠️ The watermark must not re-deliver. A client that re-reads its inbox
* from zero shows every message again on every poll, and the second
* decryption fails because the ratchet has moved — so the user sees their
* history replaced by errors.
*/
@Test
fun `polling twice does not deliver the same message twice`() {
val base = fleet ?: return
val alice = signUp(base, "c")
val bob = signUp(base, "d")
alice.conversations.send(bob.userId, "only once please")
assertEquals(1, bob.conversations.poll().size)
assertEquals(0, bob.conversations.poll().size, "the relay re-delivered a read message")
}
/**
* A person signed in on two devices must receive on both.
*
* ⚠️ This is the failure that looks fine in every test where everyone has
* one device: encrypting to "the" recipient device delivers to whichever
* came back first and silently drops the rest.
*/
@Test
fun `a message reaches every device a person has`() {
val base = fleet ?: return
val alice = signUp(base, "e")
val bob = signUp(base, "f")
// Bob's second device: same account, a new identity of its own.
val second = DeviceIdentity.create()
val secondVault = Vault(MapStorage(), key())
secondVault.saveIdentity(second)
val secondRelay = RelayClient(base, bob.handle, appPasswordOf(bob))
secondRelay.useRelayAuth(second)
secondRelay.registerDevice(second, second.generateOneTimePreKeys(20))
secondVault.saveIdentity(second)
val secondConversations = Conversations(second, secondVault, secondRelay)
val sent = alice.conversations.send(bob.userId, "to both of your devices")
assertEquals(2, sent.delivered.size, "only reached ${sent.delivered}; failed: ${sent.failed}")
assertEquals("to both of your devices", bob.conversations.poll().single().text)
assertEquals("to both of your devices", secondConversations.poll().single().text)
}
/**
* The relay stores ciphertext and nothing else.
*
* ⚠️ Worth asserting rather than assuming: the entire claim this product
* makes is that the operator cannot read the mail, and the cheapest way to
* break it is a client that sends a plaintext field "just for the preview".
*/
@Test
fun `the relay never sees the plaintext`() {
val base = fleet ?: return
val alice = signUp(base, "g")
val bob = signUp(base, "h")
val secret = "a-distinctive-plaintext-marker-9f2a"
alice.conversations.send(bob.userId, secret)
val raw = bob.relay.pollInbox(bob.identity.deviceId, 0, 10)
assertTrue(raw.isNotEmpty(), "nothing arrived to inspect")
for (message in raw) {
val decoded = String(Wire.unb64(message.payload), Charsets.UTF_8)
assertTrue(!decoded.contains(secret), "the plaintext was on the wire: $decoded")
}
}
private val appPasswords = HashMap<String, String>()
private fun appPasswordOf(person: Person): String =
appPasswords.getOrPut(person.handle) {
RelayAccounts(fleet!!).signIn(person.handle, "a-long-enough-passphrase").appPassword
}
}