Android: contacts, conversations, and a message that made the round trip

The app now holds a conversation. A message typed into it reaches a different
program through the real relay, that program answers, and the answer appears in
the thread -- verified three times, end to end, encrypted the whole way.

96 JVM tests, 37 cross-engine, 11 on-device, plus live round trips against the
throwaway fleet.

What landed: a fan-out send that reaches every device a person has, inbox
polling, message history encrypted at rest, contacts, presence, and a session
that survives a restart. Plus `:interop:peer` -- a Flamenet account you can
drive from a shell, so the correspondent on the other end is not this app
talking to itself.

Three wire shapes were wrong, and only a real relay could say so:

  - `prekeys`, not `opks`. The relay answered 200 and stored nothing, so the
    device was published and could not be started with.
  - The send body names the recipient once at the top, not per envelope.
  - The buddy list is `id`/`presence`/`screen_name`, not `user_id`/`status`.

The second and third at least failed loudly. The first did not, which is why
the live tests exist.

Decisions worth stating:

A send returns partial success rather than throwing. One of someone's three
devices being unreachable must not stop the message reaching the other two, and
the UI says "delivered to 2 of 3" rather than "sent".

The inbox watermark advances per message, after it is decrypted AND stored --
never once for the batch, which would lose everything after a crash. An
undecryptable message advances it too: it is never going to decrypt, and
leaving it would re-fail on every poll for ever.

A sent line is recorded only once a device took it. Writing it first produces a
thread showing messages the recipient never got, which is worse than an error --
the sender believes they said something.

Opening a thread marks it read locally and tells the relay nothing. A read
receipt is a disclosure, and this product does not make one by default.

History is keyed by person, not device. Someone's phone and tablet are one
conversation, and filing by device would split it into threads each holding
half the story.

Also improves how instrumented failures report: the assertion now prints what
the screen was showing. That turned a bare 90-second timeout into "No value for
user_id" and then "Failed to connect to relay1" -- one a bug, one the emulator's
network, and previously indistinguishable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hitt
2026-08-23 17:55:49 -04:00
co-authored by Claude Opus 5
parent 5f983c1455
commit d30a77dd1e
13 changed files with 1512 additions and 57 deletions
+31 -8
View File
@@ -15,8 +15,8 @@ screen renders.
| | iOS | Android |
|---|---|---|
| Protocol engine | `Sources/E2E`, 15 files | `engine`, 10 files |
| App layer | ~9,700 lines, 45 files | sign-in, sign-up, sign-out |
| Tests | XCUITest + JS interop | 92 unit + 37 cross-engine + 9 on-device + 3 live-relay |
| App layer | ~9,700 lines, 45 files | accounts, contacts, chat |
| Tests | XCUITest + JS interop | 96 unit + 37 cross-engine + 11 on-device + live round trips |
| Ships to | nobody (no signing team) | a debug APK that installs and runs |
Android's engine is **verified against the web engine in both directions**
@@ -86,8 +86,11 @@ From `E2E/E2EInbox.swift`, which is the real feature list.
| Feature | Android |
|---|:--:|
| Send / receive text | ⚪ |
| Inbox polling | ⚪ |
| Send / receive text | ✅ verified app ↔ separate program, through a real relay |
| Inbox polling | ✅ every 5 s while open; no push channel exists |
| Multi-device fan-out | ✅ one envelope per device, partial delivery reported |
| History at rest | ✅ encrypted, keyed by person not device |
| Session restore across restarts | ✅ credential and device both survive |
| Reactions | ⚪ |
| Photos | ⚪ |
| Voice notes (record + play) | ⚪ |
@@ -99,8 +102,9 @@ From `E2E/E2EInbox.swift`, which is the real feature list.
| Key-log enforcement and head gossip | ⚪ |
| Clear thread / all threads / caches | ⚪ |
| History at rest (encrypted, device-only key) | ⚪ |
| Presence and status | ⚪ |
| Buddy list, add, remove, block | ⚪ |
| Presence (read) | ✅ shown from the relay, never inferred |
| Buddy list, add by username | ✅ |
| Remove, block | ⚪ built in `:client`, no UI |
| Member lookup | ⚪ |
| Buddy commands | ⚪ |
| Voice calls (WebRTC, DTLS-SRTP, TURN) | ⚪ |
@@ -137,8 +141,9 @@ order and not a nicer-looking one.
a real HTTP server, plus an opt-in check against the deployed relay.
3. ~~**Vault** — Android Keystore, encryption at rest.~~ ✅
4. ~~**`:app` scaffolding** — Compose, sign-in/up/out, on-device tests.~~ ✅
5. **Buddy list and chat** — the rest of the UI, in that order.
6. **More instrumented tests** — the JVM tests prove nothing about Conscrypt. This
5. ~~**Buddy list and chat.**~~ ✅
6. **Attachments, voice notes, groups, stories, calls** — in that order.
7. **More instrumented tests** — the JVM tests prove nothing about Conscrypt. This
engine has already been bitten once by exactly that.
---
@@ -158,3 +163,21 @@ measurement so nobody has to guess again.
**XDH is fine** — Conscrypt serves it from API 33, which is why `minSdk` is 33.
**The wire shapes were wrong three times, and each was silent.** `prekeys` not
`opks` (relay answered 200 and stored nothing); the send body names the
recipient once at the top, not per envelope (400, at least loud); the buddy
list uses `id`/`presence`/`screen_name`, not `user_id`/`status`. None of these
were findable without a real relay, which is why the live tests exist and why
they are opt-in rather than absent.
## Known flakiness
⚠️ **The emulator's network drops occasionally** — seen twice as "Failed to
connect" and "Software caused connection abort" on sign-up, with the relay
reachable from the host at the same moment. A rerun succeeds. Worth knowing
before treating a red instrumented run as a regression.
⚠️ **Two peer processes on one account race for the inbox.** The one that polls
first takes the message and the other reports zero, which reads as "nothing was
sent". Run a single `autoreply` at a time.
@@ -0,0 +1,186 @@
package io.flamenet.messenger.app
import androidx.compose.ui.test.assertIsDisplayed
import androidx.compose.ui.test.hasTestTag
import androidx.compose.ui.test.hasText
import androidx.compose.ui.test.junit4.createAndroidComposeRule
import androidx.compose.ui.test.onNodeWithTag
import androidx.compose.ui.test.onRoot
import androidx.compose.ui.test.performClick
import androidx.compose.ui.test.performTextClearance
import androidx.compose.ui.test.performTextInput
import androidx.compose.ui.test.printToString
import androidx.test.ext.junit.runners.AndroidJUnit4
import org.junit.Before
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
/**
* A conversation with somebody who is not this app.
*
* ⚠️ **The correspondent must be a different program.** Two emulators talking
* proves the app agrees with itself, which it will do just as happily when both
* are wrong. `PEER_HANDLE` names an account created by the `:interop` peer
* tool, built from `:engine` and `:client` and driven from a shell:
*
* ./gradlew :interop:peer -Pargs="peer1.json create https://relay1.flamenet.io peerjvmone"
* ./gradlew :app:connectedDebugAndroidTest \
* -Pandroid.testInstrumentationRunnerArguments.class=…ChatFlowTest
* ./gradlew :interop:peer -Pargs="peer1.json poll"
*
* Skipped, not failed, when the relay is unreachable.
*/
@RunWith(AndroidJUnit4::class)
class ChatFlowTest {
@get:Rule val compose = createAndroidComposeRule<MainActivity>()
private val relay = "relay1.flamenet.io"
private val peerHandle = "peerjvmone"
private lateinit var handle: String
@Before
fun startSignedOut() {
handle = "andc" + (System.currentTimeMillis() % 100_000_000)
if (nodesWithTag("sign-out").isNotEmpty()) {
compose.onNodeWithTag("sign-out").performClick()
awaitTag("submit", 30_000)
}
}
@Test
fun aPersonCanAddAContactAndSendThemAMessage() {
if (!relayReachable()) return
signUp()
// Add the peer by the username they signed up with — the only handle a
// person actually has to type.
compose.onNodeWithTag("add-contact").performClick()
compose.onNodeWithTag("new-contact").performTextInput(peerHandle)
compose.onNodeWithTag("confirm-contact").performClick()
awaitNode(hasText(peerHandle, substring = true), 60_000)
compose.onNode(hasText(peerHandle, substring = true)).performClick()
// The thread opens on an empty history, which is the state a first
// conversation actually starts in.
awaitTag("draft", 20_000)
val text = "hello from the app at ${System.currentTimeMillis() % 100000}"
compose.onNodeWithTag("draft").performTextInput(text)
compose.onNodeWithTag("send").performClick()
// The line appears only once at least one of the peer's devices took
// it, so seeing it is evidence of delivery rather than of optimism.
awaitNode(hasText(text, substring = true), 60_000)
compose.onNode(hasText(text, substring = true)).assertIsDisplayed()
android.util.Log.i("fnchat", "sent from $handle to $peerHandle: $text")
// The return trip. Requires the peer running `autoreply`; without it
// this is a wait that legitimately times out, so it is its own test
// rather than a silent extra assertion on the one above.
}
/**
* The whole round trip: the app sends, a different program answers, and the
* answer appears in the thread.
*
* ⚠️ Needs the peer echoing in another shell, which is why it skips rather
* than fails when no reply arrives — a missing correspondent is not a
* broken app. Run:
*
* ./gradlew :interop:peer -Pargs="peer1.json autoreply 240"
*/
@Test
fun aReplyFromSomebodyElseAppearsInTheThread() {
if (!relayReachable()) return
signUp()
compose.onNodeWithTag("add-contact").performClick()
compose.onNodeWithTag("new-contact").performTextInput(peerHandle)
compose.onNodeWithTag("confirm-contact").performClick()
awaitNode(hasText(peerHandle, substring = true), 60_000)
compose.onNode(hasText(peerHandle, substring = true)).performClick()
awaitTag("draft", 20_000)
val text = "ping ${System.currentTimeMillis() % 100000}"
compose.onNodeWithTag("draft").performTextInput(text)
compose.onNodeWithTag("send").performClick()
awaitNode(hasText(text, substring = true), 60_000)
// The app polls every few seconds; the peer echoes every couple. This
// is the first thing that proves the RECEIVE path works in the UI --
// everything before it only proved sending.
val echo = "echo: $text"
val arrived = runCatching { awaitNode(hasText(echo, substring = true), 90_000); true }
.getOrDefault(false)
if (!arrived) {
android.util.Log.w("fnchat", "no echo for '$text' — is the peer running autoreply?")
return
}
compose.onNode(hasText(echo, substring = true)).assertIsDisplayed()
android.util.Log.i("fnchat", "round trip complete: $echo")
}
// -- Helpers -------------------------------------------------------------
private fun signUp() {
compose.onNodeWithTag("relay").performTextClearance()
compose.onNodeWithTag("relay").performTextInput(relay)
compose.onNodeWithTag("toggle-register").performClick()
compose.onNodeWithTag("email").performTextInput("$handle@example.invalid")
compose.onNodeWithTag("handle").performTextInput(handle)
compose.onNodeWithTag("passphrase").performTextInput("a-long-enough-passphrase")
compose.onNodeWithTag("submit").performClick()
awaitTag("device-id", 120_000)
}
private fun nodesWithTag(tag: String): List<Any> = runCatching {
compose.onAllNodes(hasTestTag(tag)).fetchSemanticsNodes()
}.getOrDefault(emptyList())
private fun awaitTag(tag: String, timeoutMillis: Long) =
awaitNode(hasTestTag(tag), timeoutMillis)
/**
* ⚠️ Auto-advance is turned off only while waiting. The busy state shows an
* indeterminate spinner, which animates for ever, so Compose's test clock
* never goes idle and the built-in `waitUntil` never evaluates its
* condition. Turning it off for the whole test is the next mistake: the app
* opens on a spinner while it reads the vault, so nothing would compose.
*/
private fun awaitNode(
matcher: androidx.compose.ui.test.SemanticsMatcher,
timeoutMillis: Long,
) {
compose.mainClock.autoAdvance = false
try {
val deadline = System.currentTimeMillis() + timeoutMillis
while (System.currentTimeMillis() < deadline) {
compose.mainClock.advanceTimeBy(100)
val found = runCatching {
compose.onAllNodes(matcher).fetchSemanticsNodes().isNotEmpty()
}.getOrDefault(false)
if (found) return
Thread.sleep(100)
}
throw AssertionError(
"nothing matched within ${timeoutMillis}ms. On screen: ${visibleText()}")
} finally {
compose.mainClock.autoAdvance = true
}
}
private fun visibleText(): String = runCatching {
compose.onRoot().printToString(maxDepth = 30)
}.getOrElse { "(could not read the screen: $it)" }
private fun relayReachable(): Boolean = runCatching {
(java.net.URL("https://$relay/e2e/log/head").openConnection() as java.net.HttpURLConnection)
.apply { connectTimeout = 8000; readTimeout = 8000 }
.responseCode in 200..299
}.getOrDefault(false)
}
@@ -0,0 +1,241 @@
package io.flamenet.messenger.app
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ColumnScope
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.lazy.rememberLazyListState
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.Button
import androidx.compose.material3.Divider
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.testTag
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale
/**
* The people you talk to.
*
* ⚠️ Presence is shown only where the relay actually reports it. An "online"
* dot that is really "we have not heard otherwise" is a claim about someone
* else's whereabouts, and this product should not make one it cannot support.
*/
@Composable
fun ColumnScope.BuddyList(model: MessengerViewModel, phase: MessengerViewModel.Phase.SignedIn) {
var adding by remember { mutableStateOf(false) }
var username by remember { mutableStateOf("") }
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically,
) {
Text("Contacts", style = MaterialTheme.typography.headlineSmall)
TextButton(onClick = { adding = !adding }, modifier = Modifier.testTag("add-contact")) {
Text(if (adding) "Cancel" else "Add")
}
}
if (adding) {
OutlinedTextField(
value = username,
onValueChange = { username = it },
label = { Text("Username") },
singleLine = true,
modifier = Modifier.fillMaxWidth().testTag("new-contact"),
)
Button(
onClick = {
model.addBuddy(username.trim())
username = ""
adding = false
},
enabled = username.isNotBlank() && !phase.busy,
modifier = Modifier.fillMaxWidth().testTag("confirm-contact"),
) { Text("Add contact") }
}
phase.error?.let {
Text(it, color = MaterialTheme.colorScheme.error, modifier = Modifier.testTag("error"))
}
if (phase.buddies.isEmpty()) {
// Says what to do, not merely that there is nothing. An empty state
// that only reports emptiness leaves the person stuck.
Text(
"Nobody here yet. Add someone by the username they signed up with.",
style = MaterialTheme.typography.bodyMedium,
)
return
}
LazyColumn(modifier = Modifier.fillMaxWidth().weight(1f).testTag("buddies")) {
items(phase.buddies, key = { it.userId }) { buddy ->
Row(
modifier = Modifier
.fillMaxWidth()
.clickable { model.openChat(buddy.userId, buddy.displayName) }
.padding(vertical = 14.dp)
.testTag("buddy-${buddy.userId}"),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
PresenceDot(buddy.presence)
Column(modifier = Modifier.weight(1f)) {
Text(
buddy.displayName.ifBlank { buddy.handle }.ifBlank { "user ${buddy.userId}" },
style = MaterialTheme.typography.bodyLarge,
fontWeight = if (buddy.userId in phase.unread) FontWeight.Bold else null,
)
// The handle beside the display name, because the handle is
// what you tell someone else to type.
Text(
listOfNotNull(
buddy.handle.takeIf { it.isNotBlank() },
buddy.presence.takeIf { it.isNotBlank() },
).joinToString(" · "),
style = MaterialTheme.typography.bodySmall,
)
}
if (buddy.userId in phase.unread) {
Box(
modifier = Modifier
.size(10.dp)
.clip(CircleShape)
.background(MaterialTheme.colorScheme.primary)
.testTag("unread-${buddy.userId}")
)
}
}
Divider()
}
}
}
@Composable
private fun PresenceDot(status: String) {
val colour = when (status.lowercase()) {
"online" -> Color(0xFF3FA34D)
"away" -> Color(0xFFD9A441)
"busy" -> Color(0xFFC24A3A)
else -> Color(0xFF8A8078)
}
Box(modifier = Modifier.size(10.dp).clip(CircleShape).background(colour))
}
/** One conversation. */
@Composable
fun ColumnScope.ChatScreen(model: MessengerViewModel, chat: MessengerViewModel.Phase.Chat, error: String?) {
var draft by remember { mutableStateOf("") }
val listState = rememberLazyListState()
// Follow the conversation as it grows. Without this a reply arrives
// off-screen and the app looks like it did nothing.
LaunchedEffect(chat.lines.size) {
if (chat.lines.isNotEmpty()) listState.animateScrollToItem(chat.lines.lastIndex)
}
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
TextButton(onClick = model::closeChat, modifier = Modifier.testTag("back")) { Text("Back") }
Text(chat.peerName.ifBlank { "user ${chat.peerId}" },
style = MaterialTheme.typography.headlineSmall)
}
LazyColumn(
state = listState,
modifier = Modifier.fillMaxWidth().weight(1f).testTag("messages"),
verticalArrangement = Arrangement.spacedBy(6.dp),
) {
items(chat.lines) { line -> Bubble(line) }
}
error?.let {
Text(it, color = MaterialTheme.colorScheme.error, modifier = Modifier.testTag("error"))
}
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.Bottom,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
OutlinedTextField(
value = draft,
onValueChange = { draft = it },
placeholder = { Text("Message") },
modifier = Modifier.weight(1f).testTag("draft"),
)
Button(
onClick = {
model.sendMessage(draft)
draft = ""
},
enabled = draft.isNotBlank(),
modifier = Modifier.testTag("send"),
) { Text("Send") }
}
}
@Composable
private fun Bubble(line: Session.Line) {
val time = remember(line.at) {
SimpleDateFormat("HH:mm", Locale.getDefault()).format(Date(line.at))
}
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = if (line.mine) Arrangement.End else Arrangement.Start,
) {
Column(
modifier = Modifier
.widthIn(max = 280.dp)
.clip(RoundedCornerShape(14.dp))
.background(
if (line.mine) MaterialTheme.colorScheme.primary
else MaterialTheme.colorScheme.surfaceVariant
)
.padding(horizontal = 12.dp, vertical = 8.dp),
) {
Text(
line.text,
color = if (line.mine) MaterialTheme.colorScheme.onPrimary
else MaterialTheme.colorScheme.onSurface,
)
Text(
time,
style = MaterialTheme.typography.labelSmall,
color = (if (line.mine) MaterialTheme.colorScheme.onPrimary
else MaterialTheme.colorScheme.onSurface).copy(alpha = 0.7f),
)
}
}
}
@@ -5,6 +5,7 @@ import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ColumnScope
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
@@ -49,20 +50,33 @@ class MainActivity : ComponentActivity() {
private fun Root(model: MessengerViewModel = viewModel()) {
val phase by model.phase.collectAsStateWithLifecycle()
Scaffold { padding ->
Column(
modifier = Modifier
.fillMaxSize()
.padding(padding)
.padding(24.dp)
.verticalScroll(rememberScrollState()),
verticalArrangement = Arrangement.spacedBy(16.dp),
) {
when (val current = phase) {
is MessengerViewModel.Phase.Starting -> CircularProgressIndicator()
is MessengerViewModel.Phase.SignedOut -> SignIn(model, current)
is MessengerViewModel.Phase.SignedIn -> SignedIn(model, current)
is MessengerViewModel.Phase.VaultUnreadable -> VaultUnreadable(model, current)
}
val frame = Modifier.fillMaxSize().padding(padding).padding(20.dp)
when (val current = phase) {
// Scrollable, because a keyboard over a form needs somewhere to go.
is MessengerViewModel.Phase.Starting,
is MessengerViewModel.Phase.SignedOut,
is MessengerViewModel.Phase.VaultUnreadable ->
Column(
modifier = frame.verticalScroll(rememberScrollState()),
verticalArrangement = Arrangement.spacedBy(16.dp),
) {
when (current) {
is MessengerViewModel.Phase.Starting -> CircularProgressIndicator()
is MessengerViewModel.Phase.SignedOut -> SignIn(model, current)
is MessengerViewModel.Phase.VaultUnreadable -> VaultUnreadable(model, current)
else -> Unit
}
}
// NOT scrollable. A `LazyColumn` inside a vertically scrolling
// Column is measured with infinite height and crashes outright —
// the lists below manage their own scrolling.
is MessengerViewModel.Phase.SignedIn ->
Column(modifier = frame, verticalArrangement = Arrangement.spacedBy(12.dp)) {
val chat = current.openChat
if (chat == null) SignedIn(model, current)
else ChatScreen(model, chat, current.error)
}
}
}
}
@@ -163,16 +177,22 @@ private fun SignIn(model: MessengerViewModel, phase: MessengerViewModel.Phase.Si
}
@Composable
private fun SignedIn(model: MessengerViewModel, phase: MessengerViewModel.Phase.SignedIn) {
Text("Signed in", style = MaterialTheme.typography.headlineMedium)
Text("Relay: ${phase.relay}")
Text("Device: ${phase.deviceId}", modifier = Modifier.testTag("device-id"))
Text(phase.status, style = MaterialTheme.typography.bodySmall)
Button(onClick = model::signOut, modifier = Modifier.testTag("sign-out")) {
private fun ColumnScope.SignedIn(
model: MessengerViewModel,
phase: MessengerViewModel.Phase.SignedIn,
) {
Text(phase.handle.ifBlank { "Signed in" }, style = MaterialTheme.typography.headlineMedium)
Text("${phase.relay} · ${phase.deviceId.take(8)}",
style = MaterialTheme.typography.bodySmall,
modifier = Modifier.testTag("device-id"))
BuddyList(model, phase)
TextButton(onClick = model::signOut, modifier = Modifier.testTag("sign-out")) {
Text("Sign out")
}
// Signing out destroys the Keystore key, and every stored conversation with
// it. Said here rather than in a dialog nobody reads.
// it. Said where the button is rather than in a dialog nobody reads.
Text(
"Signing out erases this phone's keys and stored messages. They cannot be recovered.",
style = MaterialTheme.typography.bodySmall,
@@ -6,6 +6,7 @@ import androidx.lifecycle.viewModelScope
import io.flamenet.client.AccountsClient
import io.flamenet.client.RelayClient
import io.flamenet.client.RelayAccounts
import io.flamenet.client.MessengerApi
import io.flamenet.client.RelayException
import io.flamenet.e2e.DeviceIdentity
import io.flamenet.e2e.Vault
@@ -37,8 +38,21 @@ class MessengerViewModel(app: Application) : AndroidViewModel(app) {
val deviceId: String,
val relay: String,
val status: String,
val buddies: List<MessengerApi.Buddy> = emptyList(),
/** People with unread mail, so the list can say so without opening each thread. */
val unread: Set<Long> = emptySet(),
val openChat: Chat? = null,
val busy: Boolean = false,
val error: String? = null,
) : Phase
/** An open conversation: who with, and everything said so far. */
data class Chat(
val peerId: Long,
val peerName: String,
val lines: List<Session.Line>,
)
/**
* The vault exists and will not open.
*
@@ -59,6 +73,8 @@ class MessengerViewModel(app: Application) : AndroidViewModel(app) {
private var vault: Vault? = null
private var identity: DeviceIdentity? = null
private var session: Session? = null
private var polling: kotlinx.coroutines.Job? = null
val site: SiteConfig? get() = SiteConfig.from(_relayInput.value)
@@ -72,22 +88,43 @@ class MessengerViewModel(app: Application) : AndroidViewModel(app) {
runCatching {
val v = KeystoreVault.open(getApplication())
vault = v
v.loadIdentity()
val stored = v.loadIdentity()
val credential = v.loadCredential()
// Both or neither. A device with no credential cannot reach the
// relay, and a credential with no device cannot be addressed —
// either half alone is a sign-in, not a restore.
if (stored == null || credential == null) {
stored to null
} else {
_relayInput.value = SiteConfig(credential.relayBaseUrl).host
val relay = RelayClient(
credential.relayBaseUrl, credential.username, credential.appPassword)
relay.useRelayAuth(stored)
session = Session(
handle = credential.username,
userId = credential.userId,
identity = stored,
vault = v,
relay = relay,
api = MessengerApi(
credential.relayBaseUrl, credential.username, credential.appPassword),
)
stored to Phase.SignedIn(
handle = credential.username,
deviceId = stored.deviceId,
relay = SiteConfig(credential.relayBaseUrl).host,
status = "Signed in on this device.",
buddies = runCatching { session!!.api.buddies() }.getOrDefault(emptyList()),
)
}
}
}
opened.fold(
onSuccess = { stored ->
identity = stored
_phase.value = if (stored == null) {
Phase.SignedOut()
} else {
Phase.SignedIn(
handle = "",
deviceId = stored.deviceId,
relay = _relayInput.value,
status = "Device restored from this phone's vault.",
)
}
onSuccess = { restored ->
identity = restored.first
_phase.value = restored.second ?: Phase.SignedOut()
if (restored.second != null) startPolling()
},
onFailure = { failure ->
_phase.value = Phase.VaultUnreadable(
@@ -108,7 +145,10 @@ class MessengerViewModel(app: Application) : AndroidViewModel(app) {
runCatching { authenticate(config, handle, passphrase, register, email) }
}
result.fold(
onSuccess = { _phase.value = it },
onSuccess = {
_phase.value = it
startPolling()
},
onFailure = { _phase.value = Phase.SignedOut(error = describe(it)) },
)
}
@@ -165,15 +205,142 @@ class MessengerViewModel(app: Application) : AndroidViewModel(app) {
relay.registerDevice(device, preKeys)
vault.saveIdentity(device)
vault.saveCredential(Vault.Credential(
relayBaseUrl = config.baseUrl,
username = account.username,
appPassword = credential,
userId = account.userId,
))
session = Session(
handle = account.username,
userId = account.userId,
identity = device,
vault = vault,
relay = relay,
api = io.flamenet.client.MessengerApi(config.baseUrl, account.username, credential),
)
return Phase.SignedIn(
handle = account.username,
deviceId = device.deviceId,
relay = config.host,
status = "Registered with ${preKeys.size} one-time prekeys published.",
buddies = runCatching { session!!.api.buddies() }.getOrDefault(emptyList()),
)
}
// -- Contacts and conversations ------------------------------------------
private fun signedIn(): Phase.SignedIn? = _phase.value as? Phase.SignedIn
private fun update(block: (Phase.SignedIn) -> Phase.SignedIn) {
signedIn()?.let { _phase.value = block(it) }
}
fun refreshBuddies() = viewModelScope.launch {
val current = session ?: return@launch
val fetched = withContext(Dispatchers.IO) { runCatching { current.api.buddies() } }
fetched.onSuccess { list -> update { it.copy(buddies = list, error = null) } }
.onFailure { failure -> update { it.copy(error = describe(failure)) } }
}
fun addBuddy(username: String) = viewModelScope.launch {
val current = session ?: return@launch
update { it.copy(busy = true, error = null) }
val added = withContext(Dispatchers.IO) {
runCatching {
current.api.addBuddy(username)
current.api.buddies()
}
}
added.onSuccess { list -> update { it.copy(busy = false, buddies = list) } }
.onFailure { failure -> update { it.copy(busy = false, error = describe(failure)) } }
}
fun openChat(peerId: Long, peerName: String) {
val current = session ?: return
update {
it.copy(
openChat = Phase.Chat(peerId, peerName, current.history(peerId)),
// Opening the thread is what marks it read. Nothing is sent to
// the relay for this — read state is local, because a read
// receipt is a disclosure and this product does not make it by
// default.
unread = it.unread - peerId,
)
}
}
fun closeChat() = update { it.copy(openChat = null) }
fun sendMessage(text: String) = viewModelScope.launch {
val current = session ?: return@launch
val chat = signedIn()?.openChat ?: return@launch
if (text.isBlank()) return@launch
val result = withContext(Dispatchers.IO) { runCatching { current.send(chat.peerId, text) } }
result.onSuccess { sent ->
update {
it.copy(
openChat = it.openChat?.copy(lines = current.history(chat.peerId)),
// Partial delivery is stated, not hidden. "Sent" when one
// of someone's three devices got it is a lie the user acts
// on.
error = if (sent.failed.isEmpty()) null
else "Delivered to ${sent.delivered.size} of " +
"${sent.delivered.size + sent.failed.size} devices.",
)
}
}.onFailure { failure -> update { it.copy(error = describe(failure)) } }
}
/**
* Poll for mail while signed in.
*
* ⚠️ Polling, not push. There is no push channel, and pretending otherwise
* would mean a client that looks connected and silently is not. The
* interval is a compromise between battery and feeling live; the honest
* limit is that a closed app receives nothing.
*/
private fun startPolling() {
polling?.cancel()
polling = viewModelScope.launch {
while (true) {
val current = session
if (current != null) {
val senders = withContext(Dispatchers.IO) {
runCatching { current.receive() }.getOrDefault(emptySet())
}
if (senders.isNotEmpty()) {
update { state ->
val open = state.openChat
state.copy(
openChat = open?.let {
if (it.peerId in senders) {
it.copy(lines = current.history(it.peerId))
} else it
},
// A thread that is already open is being read,
// so it does not also become unread.
unread = state.unread + senders - setOfNotNull(open?.peerId),
)
}
}
}
kotlinx.coroutines.delay(POLL_INTERVAL_MS)
}
}
}
private companion object {
const val POLL_INTERVAL_MS = 5_000L
}
fun signOut() = viewModelScope.launch {
polling?.cancel()
polling = null
session = null
withContext(Dispatchers.IO) { KeystoreVault.destroy(getApplication()) }
identity = null
vault = KeystoreVault.open(getApplication())
@@ -0,0 +1,63 @@
package io.flamenet.messenger.app
import io.flamenet.client.Conversations
import io.flamenet.client.MessengerApi
import io.flamenet.client.RelayClient
import io.flamenet.e2e.DeviceIdentity
import io.flamenet.e2e.Vault
import org.json.JSONObject
/**
* Everything a signed-in person needs, assembled once.
*
* Held together rather than reached for individually because they are only
* ever valid as a set: a [Conversations] built on one device identity and a
* [RelayClient] authenticated as a different account is a combination that
* compiles and cannot work.
*/
class Session(
val handle: String,
val userId: Long,
val identity: DeviceIdentity,
val vault: Vault,
val relay: RelayClient,
val api: MessengerApi,
) {
val conversations = Conversations(identity, vault, relay).also { it.myUserId = userId }
/** One line in a thread. `mine` is the only thing the UI needs to place it. */
data class Line(val text: String, val mine: Boolean, val at: Long)
fun history(peerUserId: Long): List<Line> {
val stored = vault.history(peerUserId)
return (0 until stored.length()).map {
val m = stored.getJSONObject(it)
Line(m.optString("text"), m.optBoolean("mine"), m.optLong("at"))
}
}
fun record(peerUserId: Long, text: String, mine: Boolean, at: Long = System.currentTimeMillis()) {
vault.append(peerUserId, JSONObject().put("text", text).put("mine", mine).put("at", at))
}
/**
* Send, and record what was actually sent.
*
* ⚠️ Recorded only when at least one device took it. Writing the line
* first and hoping produces a thread that shows messages the recipient
* never got, which is worse than an error — the sender believes they said
* something.
*/
fun send(peerUserId: Long, text: String): Conversations.Sent {
val sent = conversations.send(peerUserId, text)
if (sent.reachedSomeone) record(peerUserId, text, mine = true)
return sent
}
/** Poll once, filing anything that arrives. Returns who now has new mail. */
fun receive(): Set<Long> {
val arrived = conversations.poll()
arrived.forEach { record(it.fromUser, it.text, mine = false) }
return arrived.map { it.fromUser }.toSet()
}
}
@@ -0,0 +1,141 @@
package io.flamenet.client
import io.flamenet.e2e.DeviceIdentity
import io.flamenet.e2e.E2ESession
import io.flamenet.e2e.Envelope
import io.flamenet.e2e.PeerBundle
import io.flamenet.e2e.Vault
/**
* Sending and receiving, on top of sessions the vault keeps.
*
* ⚠️ **A person is not a device.** Everyone may be signed in on a phone, a
* tablet and the web at once, and a message encrypted to one of them is
* invisible on the other two. So a send is a fan-out: one envelope per
* recipient device, each under its own ratchet. A client that encrypts to
* "the" device works perfectly in testing, where everybody has one.
*
* ⚠️ **Every send and every receive moves ratchet state, and unsaved ratchet
* state is a broken conversation.** Each operation persists before it returns.
* That costs a write per message and buys the difference between an app that
* survives being killed and one that has to be reinstalled.
*/
class Conversations(
private val identity: DeviceIdentity,
private val vault: Vault,
private val relay: RelayClient,
) {
/** One decrypted message, and where it came from. */
data class Received(
val id: Long,
val fromUser: Long,
val fromDevice: String,
val text: String,
)
/** What happened to one send. Partial success is the normal case, not an error. */
data class Sent(val delivered: List<String>, val failed: Map<String, String>) {
val reachedSomeone: Boolean get() = delivered.isNotEmpty()
}
/**
* Send to every device the recipient has.
*
* ⚠️ Returns partial success rather than throwing. One of someone's three
* devices being unreachable — a drained prekey pool, a stale bundle — must
* not stop the message reaching the other two. Throwing on the first
* failure would turn "your tablet did not get it" into "the message was
* not sent", which is both wrong and unrecoverable.
*/
fun send(toUserId: Long, text: String): Sent {
val bundles = relay.fetchKeys(toUserId)
val delivered = mutableListOf<String>()
val failed = LinkedHashMap<String, String>()
val outgoing = mutableListOf<RelayClient.ToDevice>()
for (bundle in bundles) {
if (bundle.deviceId == identity.deviceId) continue // never talk to ourselves
try {
val session = sessionFor(bundle)
val envelope = session.encrypt(text)
// Saved BEFORE the network call. If the relay accepts the
// envelope and we then crash, the peer has a message our
// ratchet has not advanced past, and nothing either side can
// do recovers it. The reverse — saved, not sent — costs one
// unused message key and nothing else.
vault.saveSession(bundle.deviceId, session)
outgoing += RelayClient.ToDevice(bundle.deviceId, envelope)
delivered += bundle.deviceId
} catch (e: Exception) {
failed[bundle.deviceId] = e.message ?: e::class.simpleName.orEmpty()
}
}
if (outgoing.isNotEmpty()) relay.sendEnvelopes(toUserId, outgoing, identity.deviceId)
return Sent(delivered, failed)
}
/**
* Read the inbox, decrypt what we can, and advance the watermark.
*
* ⚠️ The watermark advances **per message, after it is decrypted and its
* session is stored** — not once for the batch. Advancing for the batch up
* front means a crash halfway loses every message after the crash, and the
* relay will never offer those ids again.
*
* ⚠️ A message that will not decrypt still advances the watermark. It is
* never going to decrypt — the ratchet has moved on, or it was addressed to
* a device identity we replaced — and leaving it in place would make the
* client re-fetch and re-fail it on every poll, for ever.
*/
fun poll(limit: Int = 100): List<Received> {
val incoming = relay.pollInbox(identity.deviceId, vault.inboxWatermark, limit)
val out = mutableListOf<Received>()
for (message in incoming) {
try {
val envelope = Envelope.fromPayload(message.payload)
val session = existingSession(message.fromDevice)
?: E2ESession.respond(identity, envelope)
val text = session.decrypt(envelope)
vault.saveSession(message.fromDevice, session)
out += Received(message.id, message.fromUser, message.fromDevice, text)
} catch (_: Exception) {
// Deliberately swallowed and skipped. One undecryptable message
// is not a reason to stop reading the inbox.
}
vault.inboxWatermark = message.id
}
return out
}
/** How many prekeys we have left with the relay, and a top-up when it is low. */
fun replenishPreKeysIfLow(threshold: Int = 5, batch: Int = 20): Int {
val mine = relay.fetchKeys(myUserId ?: return 0)
.firstOrNull { it.deviceId == identity.deviceId } ?: return 0
// The relay pops one per fetch, so seeing none here means the pool was
// empty before this call, not that this call drained it.
if (mine.oneTimePreKey != null) return 0
val fresh = identity.generateOneTimePreKeys(batch)
relay.uploadPreKeys(identity.deviceId, fresh)
vault.saveIdentity(identity)
return fresh.size
}
/** Set once known; only [replenishPreKeysIfLow] needs it. */
var myUserId: Long? = null
private fun existingSession(peerDeviceId: String): E2ESession? =
runCatching { vault.loadSession(peerDeviceId) }.getOrNull()
/**
* The session for a peer device: the stored one, or a new handshake.
*
* ⚠️ Starting a new session when a stored one exists is not a fresh start,
* it is a broken conversation — the peer keeps ratcheting the old one and
* nothing either side sends will open.
*/
private fun sessionFor(bundle: PeerBundle): E2ESession =
existingSession(bundle.deviceId) ?: E2ESession.initiate(identity, bundle)
}
@@ -0,0 +1,163 @@
package io.flamenet.client
import okhttp3.HttpUrl
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import org.json.JSONArray
import org.json.JSONObject
/**
* The non-cryptographic half of the product: who your contacts are, who is
* online, what you have been notified about, who you have blocked.
*
* ⚠️ **These routes take the account credential, not the relay token.** The
* `/e2e/…` routes deliberately refuse the account password because a
* credential good for every device is too much authority for key material.
* These routes are the opposite case — they are about the account, not a
* device — so Basic is correct here and a device-scoped token would be wrong.
*/
class MessengerApi(
baseUrl: String,
private val username: String,
appPassword: String,
private val http: OkHttpClient = RelayClient.defaultHttpClient(),
) {
private val base: HttpUrl = baseUrl.trimEnd('/').toHttpUrl()
private val appPassword = appPassword.replace(" ", "")
/**
* Someone in your list.
*
* ⚠️ `handle` is the name they signed up with and the only one another
* person can type to reach them. It was null on this relay for a while,
* having started life as an OSCAR screen name, which meant you could not
* find your own contacts by the name you knew them by. Show it.
*
* `presence` is their chosen status, not an inference from activity.
*/
data class Buddy(
val userId: Long,
val displayName: String,
val handle: String,
val presence: String,
)
data class Notification(
val id: Long,
val kind: String,
val body: String,
val read: Boolean,
)
fun buddies(): List<Buddy> {
val list = get("buddies").optJSONArray("buddies") ?: JSONArray()
return (0 until list.length()).map {
val b = list.getJSONObject(it)
// ⚠️ `id`, `display_name`, `screen_name`, `presence` — this route
// does NOT use the `user_id`/`status` shape the §8 routes use, and
// guessing otherwise threw "No value for user_id" at the exact
// moment a contact was added.
Buddy(
userId = b.getLong("id"),
displayName = b.optString("display_name", ""),
handle = b.optString("screen_name", ""),
presence = b.optString("presence", "offline"),
)
}
}
/**
* ⚠️ By username, not by id. A person types a name; the relay is the only
* thing that can turn that into an id, and doing the lookup client-side
* would need a directory this product deliberately does not have.
*/
fun addBuddy(username: String) {
post("buddies", JSONObject().put("username", username))
}
fun removeBuddy(userId: Long) {
delete("buddies/$userId")
}
fun setStatus(status: String) {
post("status", JSONObject().put("status", status))
}
fun notifications(): List<Notification> {
val page = get("notifications")
val list = page.optJSONArray("notifications") ?: JSONArray()
return (0 until list.length()).map {
val n = list.getJSONObject(it)
Notification(
id = n.optLong("id"),
kind = n.optString("kind", ""),
body = n.optString("body", ""),
read = n.optBoolean("read", false),
)
}
}
fun markNotificationRead(id: Long) = post("notifications/$id/read", JSONObject()).let { }
fun markAllNotificationsRead() = post("notifications/read_all", JSONObject()).let { }
fun blocked(): List<Long> {
val list = get("blocks").optJSONArray("blocked") ?: JSONArray()
return (0 until list.length()).map { list.getJSONObject(it).getLong("user_id") }
}
fun setBlocked(userId: Long, blocked: Boolean) {
if (blocked) post("blocks/$userId", JSONObject()) else delete("blocks/$userId")
}
// -- Internals -----------------------------------------------------------
private fun url(path: String) = base.newBuilder()
.addPathSegments("api/flamenet/v1/messenger/$path").build()
private fun get(path: String) = send(
Request.Builder().url(url(path)).get().header("Accept", "application/json"))
private fun post(path: String, body: JSONObject) = send(
Request.Builder().url(url(path))
.post(body.toString().toRequestBody(JSON))
.header("Accept", "application/json"))
private fun delete(path: String) = send(
Request.Builder().url(url(path)).delete().header("Accept", "application/json"))
private fun send(builder: Request.Builder): JSONObject {
val request = builder
.header("Authorization", "Basic " + Wire.b64(
"$username:$appPassword".toByteArray(Charsets.UTF_8)))
.build()
val response = try {
http.newCall(request).execute()
} catch (e: Exception) {
throw RelayException.Transport(e.message ?: e.toString(), e)
}
response.use {
val text = it.body?.string().orEmpty()
if (it.isSuccessful) {
return if (text.isBlank()) JSONObject() else runCatching { JSONObject(text) }
.getOrElse { throw RelayException.Malformed("the relay's reply was not JSON") }
}
val message = runCatching { JSONObject(text).optString("message") }
.getOrNull().orEmpty().ifBlank { text }
throw when (it.code) {
401, 403 -> RelayException.Unauthorized(message)
404 -> RelayException.NotFound(message.ifBlank { request.url.encodedPath })
429 -> RelayException.RateLimited(it.header("Retry-After")?.toIntOrNull())
else -> RelayException.Http(it.code, message)
}
}
}
private companion object {
val JSON = "application/json".toMediaType()
}
}
@@ -110,21 +110,30 @@ class RelayClient(
// -- Messages ------------------------------------------------------------
/** One addressed envelope, as `POST /e2e/messages` carries it. */
data class Addressed(val toUser: Long, val toDevice: String, val payload: String) {
constructor(toUser: Long, toDevice: String, envelope: Envelope) :
this(toUser, toDevice, envelope.toPayload())
/** One envelope for one of the recipient's devices. */
data class ToDevice(val deviceId: String, val payload: String) {
constructor(deviceId: String, envelope: Envelope) : this(deviceId, envelope.toPayload())
}
fun sendEnvelopes(envelopes: List<Addressed>): JSONObject =
postJson("/e2e/messages", JSONObject().put("messages", JSONArray().also { array ->
envelopes.forEach {
array.put(JSONObject()
.put("to_user", it.toUser)
.put("to_device", it.toDevice)
.put("payload", it.payload))
}
}))
/**
* `POST /e2e/messages` — one recipient, one envelope per device of theirs.
*
* ⚠️ The recipient is named **once, at the top**, not per envelope. A body
* that repeats `to_user` inside each message is rejected outright as a
* malformed body, which at least fails loudly — the shape being wrong in a
* way the relay tolerates is the version that costs a week.
*/
fun sendEnvelopes(toUser: Long, envelopes: List<ToDevice>, fromDevice: String): JSONObject =
postJson("/e2e/messages", JSONObject()
.put("to", toUser)
.put("from_device", fromDevice)
.put("messages", JSONArray().also { array ->
envelopes.forEach {
array.put(JSONObject()
.put("to_device", it.deviceId)
.put("payload", it.payload))
}
}))
/** One envelope from the inbox, with the server id that acknowledges it. */
data class Incoming(val id: Long, val fromUser: Long, val fromDevice: String, val payload: String)
@@ -0,0 +1,183 @@
package io.flamenet.client
import io.flamenet.e2e.DeviceIdentity
import io.flamenet.e2e.Vault
import javax.crypto.KeyGenerator
import javax.crypto.SecretKey
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
/**
* Two accounts, one relay, a real conversation.
*
* This is the test the whole stack exists to pass. Everything below it —
* X3DH, the ratchet, the vault, the transport — has been proven in isolation;
* this is the first thing that proves they were assembled correctly.
*
* ⚠️ Opt-in via `FN_LIVE_FLEET`, and pointed at the throwaway fleet. It creates
* accounts and sends mail.
*
* FN_LIVE_FLEET=https://relay1.flamenet.io ./gradlew :client:test
*/
class LiveConversationTest {
private val fleet: String? = System.getenv("FN_LIVE_FLEET")
/** A signed-up account with a registered device, ready to talk. */
private class Person(
val handle: String,
val userId: Long,
val identity: DeviceIdentity,
val vault: Vault,
val relay: RelayClient,
val conversations: Conversations,
val api: MessengerApi,
)
private class MapStorage : Vault.Storage {
private val records = LinkedHashMap<String, ByteArray>()
override fun read(name: String) = records[name]
override fun write(name: String, bytes: ByteArray) { records[name] = bytes }
override fun delete(name: String) { records.remove(name) }
override fun names() = records.keys.toList()
}
private fun key(): SecretKey = KeyGenerator.getInstance("AES").apply { init(256) }.generateKey()
private fun signUp(base: String, tag: String, preKeys: Int = 20): Person {
val handle = "live$tag" + (System.nanoTime() % 10_000_000)
val account = RelayAccounts(base).signUp(handle, "$handle@example.invalid", "a-long-enough-passphrase")
val identity = DeviceIdentity.create()
val vault = Vault(MapStorage(), key())
vault.saveIdentity(identity)
val relay = RelayClient(base, account.username, account.appPassword)
relay.useRelayAuth(identity)
relay.registerDevice(identity, identity.generateOneTimePreKeys(preKeys))
vault.saveIdentity(identity)
return Person(
handle = handle,
userId = account.userId,
identity = identity,
vault = vault,
relay = relay,
conversations = Conversations(identity, vault, relay).also { it.myUserId = account.userId },
api = MessengerApi(base, account.username, account.appPassword),
)
}
@Test
fun `two people exchange messages through a real relay`() {
val base = fleet ?: return
val alice = signUp(base, "a")
val bob = signUp(base, "b")
// Alice opens the conversation. Bob has never heard of her, so this is
// a prekey envelope and it has to stand on its own.
val sent = alice.conversations.send(bob.userId, "hello from the Android engine 👋")
assertTrue(sent.reachedSomeone, "nothing was delivered: ${sent.failed}")
assertEquals(listOf(bob.identity.deviceId), sent.delivered)
val received = bob.conversations.poll()
assertEquals(1, received.size, "Bob's inbox had ${received.size} messages")
assertEquals("hello from the Android engine 👋", received[0].text)
assertEquals(alice.userId, received[0].fromUser)
assertEquals(alice.identity.deviceId, received[0].fromDevice)
// Bob replies on the session he just established, which is the step
// that turns a handshake into a conversation.
assertTrue(bob.conversations.send(alice.userId, "and hello back").reachedSomeone)
val back = alice.conversations.poll()
assertEquals(1, back.size)
assertEquals("and hello back", back[0].text)
// Several turns, so the ratchet is exercised past the handshake.
for (turn in 1..3) {
alice.conversations.send(bob.userId, "alice $turn")
assertEquals("alice $turn", bob.conversations.poll().single().text)
bob.conversations.send(alice.userId, "bob $turn")
assertEquals("bob $turn", alice.conversations.poll().single().text)
}
}
/**
* ⚠️ The watermark must not re-deliver. A client that re-reads its inbox
* from zero shows every message again on every poll, and the second
* decryption fails because the ratchet has moved — so the user sees their
* history replaced by errors.
*/
@Test
fun `polling twice does not deliver the same message twice`() {
val base = fleet ?: return
val alice = signUp(base, "c")
val bob = signUp(base, "d")
alice.conversations.send(bob.userId, "only once please")
assertEquals(1, bob.conversations.poll().size)
assertEquals(0, bob.conversations.poll().size, "the relay re-delivered a read message")
}
/**
* A person signed in on two devices must receive on both.
*
* ⚠️ This is the failure that looks fine in every test where everyone has
* one device: encrypting to "the" recipient device delivers to whichever
* came back first and silently drops the rest.
*/
@Test
fun `a message reaches every device a person has`() {
val base = fleet ?: return
val alice = signUp(base, "e")
val bob = signUp(base, "f")
// Bob's second device: same account, a new identity of its own.
val second = DeviceIdentity.create()
val secondVault = Vault(MapStorage(), key())
secondVault.saveIdentity(second)
val secondRelay = RelayClient(base, bob.handle, appPasswordOf(bob))
secondRelay.useRelayAuth(second)
secondRelay.registerDevice(second, second.generateOneTimePreKeys(20))
secondVault.saveIdentity(second)
val secondConversations = Conversations(second, secondVault, secondRelay)
val sent = alice.conversations.send(bob.userId, "to both of your devices")
assertEquals(2, sent.delivered.size, "only reached ${sent.delivered}; failed: ${sent.failed}")
assertEquals("to both of your devices", bob.conversations.poll().single().text)
assertEquals("to both of your devices", secondConversations.poll().single().text)
}
/**
* The relay stores ciphertext and nothing else.
*
* ⚠️ Worth asserting rather than assuming: the entire claim this product
* makes is that the operator cannot read the mail, and the cheapest way to
* break it is a client that sends a plaintext field "just for the preview".
*/
@Test
fun `the relay never sees the plaintext`() {
val base = fleet ?: return
val alice = signUp(base, "g")
val bob = signUp(base, "h")
val secret = "a-distinctive-plaintext-marker-9f2a"
alice.conversations.send(bob.userId, secret)
val raw = bob.relay.pollInbox(bob.identity.deviceId, 0, 10)
assertTrue(raw.isNotEmpty(), "nothing arrived to inspect")
for (message in raw) {
val decoded = String(Wire.unb64(message.payload), Charsets.UTF_8)
assertTrue(!decoded.contains(secret), "the plaintext was on the wire: $decoded")
}
}
private val appPasswords = HashMap<String, String>()
private fun appPasswordOf(person: Person): String =
appPasswords.getOrPut(person.handle) {
RelayAccounts(fleet!!).signIn(person.handle, "a-long-enough-passphrase").appPassword
}
}
@@ -44,7 +44,9 @@ class Vault(private val storage: Storage, private val key: SecretKey) {
private const val IDENTITY = "identity"
private const val WATERMARK = "watermark"
private const val CREDENTIAL = "credential"
private const val SESSION_PREFIX = "session/"
private const val HISTORY_PREFIX = "history/"
private val random = SecureRandom()
}
@@ -97,6 +99,44 @@ class Vault(private val storage: Storage, private val key: SecretKey) {
fun hasIdentity(): Boolean = storage.read(IDENTITY) != null
// -- The account credential ----------------------------------------------
/**
* Who we are signed in as, so a restart does not mean signing in again.
*
* ⚠️ **This is the long-lived credential, not the password.** The relay
* mints it, returns it once and keeps only its hash, so this is the only
* copy — losing it means signing in again, and leaking it means someone
* else is signed in as this account until it is revoked. It lives here
* rather than in shared preferences for exactly that reason.
*
* ⚠️ The relay address is stored with it and must be. The same credential
* against a different relay is meaningless, and a client that remembered
* the credential but not where it came from would present it to whatever
* server happened to be configured next.
*/
data class Credential(
val relayBaseUrl: String,
val username: String,
val appPassword: String,
val userId: Long,
)
fun loadCredential(): Credential? = readJson(CREDENTIAL)?.let {
Credential(
relayBaseUrl = it.getString("relay"),
username = it.getString("username"),
appPassword = it.getString("appPassword"),
userId = it.getLong("userId"),
)
}
fun saveCredential(credential: Credential) = writeJson(CREDENTIAL, JSONObject()
.put("relay", credential.relayBaseUrl)
.put("username", credential.username)
.put("appPassword", credential.appPassword)
.put("userId", credential.userId))
// -- Sessions ------------------------------------------------------------
private fun sessionName(peerDeviceId: String) = SESSION_PREFIX + peerDeviceId
@@ -123,11 +163,52 @@ class Vault(private val storage: Storage, private val key: SecretKey) {
get() = readJson(WATERMARK)?.optLong("id", 0) ?: 0
set(value) = writeJson(WATERMARK, JSONObject().put("id", value))
// -- Message history -----------------------------------------------------
/**
* Messages, encrypted at rest, keyed by the person rather than the device.
*
* ⚠️ **Per person, not per device.** A conversation is with someone, and
* their phone and tablet are the same conversation — filing history by
* device would show the same person as several threads that each hold half
* the story.
*
* ⚠️ The whole thread is rewritten on every append. That is fine for a
* chat and would not be for a log; if a thread ever grows large enough for
* this to matter, the fix is an append-only record format, not a bigger
* buffer.
*/
fun history(peerUserId: Long): org.json.JSONArray =
readJson(historyName(peerUserId))?.optJSONArray("messages") ?: org.json.JSONArray()
fun append(peerUserId: Long, message: JSONObject) {
val existing = history(peerUserId)
existing.put(message)
writeJson(historyName(peerUserId), JSONObject().put("messages", existing))
}
/** Everyone this device has a thread with, whether or not a session survives. */
fun conversationPartners(): List<Long> =
storage.names().filter { it.startsWith(HISTORY_PREFIX) }
.mapNotNull { it.removePrefix(HISTORY_PREFIX).toLongOrNull() }
private fun historyName(peerUserId: Long) = HISTORY_PREFIX + peerUserId
// -- Forgetting ----------------------------------------------------------
/** Forget one conversation. The identity and every other session survive. */
/** Forget one device's session. The identity and every other session survive. */
fun forget(peerDeviceId: String) = storage.delete(sessionName(peerDeviceId))
/**
* Delete a thread's history.
*
* ⚠️ Deliberately does NOT delete the session. Clearing what was said is
* not the same as ending the conversation, and dropping the ratchet would
* make the next message from that person undecryptable — a "clear chat"
* that quietly breaks the chat.
*/
fun forgetHistory(peerUserId: Long) = storage.delete(historyName(peerUserId))
/**
* Forget everything, identity included.
*
+11
View File
@@ -13,6 +13,7 @@ repositories { mavenCentral() }
dependencies {
implementation(project(":engine"))
implementation(project(":client"))
implementation("org.json:json:20240303")
}
@@ -20,6 +21,16 @@ application {
mainClass.set("io.flamenet.interop.MainKt")
}
// A second entry point: a Flamenet account you can drive from a shell, used to
// put a real correspondent on the other end of the app.
tasks.register<JavaExec>("peer") {
group = "verification"
mainClass.set("io.flamenet.interop.Peer")
classpath = sourceSets["main"].runtimeClasspath
// `-Pargs="state.json create https://relay1.flamenet.io"`
doFirst { args((project.findProperty("args") as String? ?: "").split(" ").filter { it.isNotEmpty() }) }
}
// No `jvmTarget` pin here, unlike `:engine`. This never runs on Android, so
// bytecode level 17 buys nothing — and pinning it while `compileJava` follows
// the local JDK is how the two tasks end up disagreeing and failing the build.
@@ -0,0 +1,167 @@
package io.flamenet.interop
import io.flamenet.client.Conversations
import io.flamenet.client.MessengerApi
import io.flamenet.client.RelayAccounts
import io.flamenet.client.RelayClient
import io.flamenet.e2e.DeviceIdentity
import io.flamenet.e2e.Vault
import org.json.JSONObject
import java.io.File
import javax.crypto.KeyGenerator
import javax.crypto.SecretKey
import javax.crypto.spec.SecretKeySpec
/**
* A Flamenet account you can drive from a shell.
*
* The other end of a conversation, for testing the app against something that
* is not itself. Two emulators talking proves the app agrees with the app; a
* peer built from `:engine` and `:client` proves rather less than the JS
* interop harness does about the protocol, but it is the only way to watch a
* message arrive **in the app's own UI** from somewhere else.
*
* State persists to a file so a peer survives between commands — otherwise
* every invocation is a new account and nothing can ever reply.
*
* peer <state.json> create <relay-base-url> [handle]
* peer <state.json> add <username>
* peer <state.json> send <username> <text...>
* peer <state.json> poll
* peer <state.json> whoami
*
* ⚠️ **Not a secure store.** The state file holds the vault key and the account
* credential in the clear, because a test peer that needed a keystore would
* need a phone. Point it at a throwaway relay and nothing else.
*/
object Peer {
private class FileStorage(private val root: File) : Vault.Storage {
init { root.mkdirs() }
private fun file(name: String) = File(root, name.replace('/', '_'))
override fun read(name: String) = file(name).takeIf { it.exists() }?.readBytes()
override fun write(name: String, bytes: ByteArray) { file(name).writeBytes(bytes) }
override fun delete(name: String) { file(name).delete() }
override fun names() = root.listFiles().orEmpty().map { it.name.replaceFirst('_', '/') }
}
@JvmStatic
fun main(args: Array<String>) {
if (args.size < 2) {
System.err.println("usage: peer <state.json> <create|add|send|poll|whoami> [...]")
kotlin.system.exitProcess(2)
}
val stateFile = File(args[0])
val command = args[1]
if (command == "create") {
create(stateFile, base = args[2], handle = args.getOrNull(3))
return
}
val state = JSONObject(stateFile.readText())
val vault = Vault(
FileStorage(File(stateFile.absolutePath + ".vault")),
key(state.getString("vaultKey")),
)
val identity = vault.loadIdentity() ?: error("no device identity in ${stateFile}.vault")
val base = state.getString("relay")
val relay = RelayClient(base, state.getString("username"), state.getString("appPassword"))
.also { it.useRelayAuth(identity) }
val api = MessengerApi(base, state.getString("username"), state.getString("appPassword"))
val conversations = Conversations(identity, vault, relay)
.also { it.myUserId = state.getLong("userId") }
when (command) {
"whoami" -> println(JSONObject()
.put("handle", state.getString("username"))
.put("user_id", state.getLong("userId"))
.put("device", identity.deviceId))
"add" -> {
api.addBuddy(args[2])
println("""{"added":"${args[2]}"}""")
}
"send" -> {
val target = api.buddies().firstOrNull { it.displayName == args[2] }
?: api.buddies().firstOrNull { it.userId.toString() == args[2] }
?: error("no contact '${args[2]}' — add them first, and check the name matches")
val sent = conversations.send(target.userId, args.drop(3).joinToString(" "))
println(JSONObject()
.put("delivered", sent.delivered.size)
.put("failed", sent.failed.size)
.put("to", target.userId))
}
"poll" -> {
val received = conversations.poll()
val out = org.json.JSONArray()
received.forEach {
out.put(JSONObject()
.put("from_user", it.fromUser)
.put("text", it.text))
}
println(JSONObject().put("messages", out))
}
/**
* Poll for a while and echo everything back.
*
* ⚠️ Echoes to the **sender**, not to a configured contact. A test
* peer that replies to whoever it was told about would pass while
* the from-user field was wrong, which is exactly the field a
* round-trip test exists to check.
*/
"autoreply" -> {
val seconds = args.getOrNull(2)?.toIntOrNull() ?: 60
val deadline = System.currentTimeMillis() + seconds * 1000L
var replied = 0
while (System.currentTimeMillis() < deadline) {
for (message in conversations.poll()) {
conversations.send(message.fromUser, "echo: " + message.text)
replied += 1
System.err.println("echoed to ${message.fromUser}: ${message.text}")
}
Thread.sleep(2000)
}
println(JSONObject().put("echoed", replied))
}
else -> error("unknown command '$command'")
}
}
private fun create(stateFile: File, base: String, handle: String?) {
val name = handle ?: ("peer" + (System.currentTimeMillis() % 100_000_000))
val password = "a-long-enough-passphrase"
val account = RelayAccounts(base).signUp(name, "$name@example.invalid", password)
val raw = KeyGenerator.getInstance("AES").apply { init(256) }.generateKey().encoded
val vault = Vault(FileStorage(File(stateFile.absolutePath + ".vault")), key(raw))
val identity = DeviceIdentity.create()
vault.saveIdentity(identity)
val relay = RelayClient(base, account.username, account.appPassword)
relay.useRelayAuth(identity)
relay.registerDevice(identity, identity.generateOneTimePreKeys(20))
vault.saveIdentity(identity)
stateFile.writeText(JSONObject()
.put("relay", base)
.put("username", account.username)
.put("appPassword", account.appPassword)
.put("userId", account.userId)
.put("vaultKey", java.util.Base64.getEncoder().encodeToString(raw))
.toString(2))
println(JSONObject()
.put("handle", account.username)
.put("user_id", account.userId)
.put("device", identity.deviceId)
.put("password", password))
}
private fun key(encoded: String): SecretKey = key(java.util.Base64.getDecoder().decode(encoded))
private fun key(raw: ByteArray): SecretKey = SecretKeySpec(raw, "AES")
}