extant2000/flamenet-messenger-android
1
0
Fork 0
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

Android: voice notes

Browse Source 
The engine and the client already carried audio — `Attachment.AUDIO`, a
`duration` field, and a `sendAttachment` that takes both. What was missing was
entirely the phone: something to record with, something to play with, and two
controls. So this is app-layer only, and adds no dependency: `MediaRecorder`
and `MediaPlayer` are platform APIs, and a voice note is an ordinary encrypted
attachment the moment it is bytes.

AAC in an MP4 container, mono, 32 kbit/s. Speech rather than music: clearly
intelligible, and roughly a tenth the size of the stereo default nobody asked
for — which matters when every byte is uploaded encrypted and stored by
somebody paying for it.

⚠️ **Tap to start, tap to send — not hold-to-talk.** Hold-to-talk demonstrates
well and behaves badly on a real phone: the gesture competes with scrolling,
and letting go early silently discards what was said. Two deliberate taps
cannot be lost by accident.

⚠️ **`MediaRecorder.stop()` throws when it captured no frames** — a tap instead
of a hold, or a permission granted and then denied by policy. The exception is
a bare `RuntimeException` and it reads as "recording is broken" when the truth
is "there was nothing to record". Treated as an empty recording, and anything
under half a second is dropped rather than delivered: a note that plays silence
looks like a delivery bug, and the person who fumbled the button already knows
they did.

⚠️ **The microphone permission is requested when somebody asks to record**, not
at startup. A microphone prompt on first launch, before anything has been said,
is one people deny on reflex — and a denied permission is far harder to recover
from than one that was never asked for.

⚠️ Received notes are fetched and decrypted **on tap, never on render**. A
thread of twenty would otherwise download and decrypt all twenty on scroll:
bytes nobody asked for, and twenty decrypted files in the cache for a list
nobody has listened to. Playback stops on leaving the thread, because a
MediaPlayer that outlives its composable holds audio focus and a file handle.

Every temporary file — the recording, and the decrypted copy playback needs
because `MediaPlayer` cannot take a byte array — is deleted on both the success
and the failure path. Plaintext audio left in a cache directory is the exact
thing this product is not supposed to do.

`imageBytes` became `attachmentBytes`: photos and voice notes take the same
path, and a method called `imageBytes` that audio also calls is a name that has
quietly stopped being true.

Also retires buddy commands in the parity table — no longer a feature.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hitt
2026-08-25 14:37:31 -04:00
co-authored by Claude Opus 5
parent a213ad8d5d
commit 4728928d98
7 changed files with 442 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PARITY.md
+2 -2
View File
@@ -95,7 +95,7 @@ From `E2E/E2EInbox.swift`, which is the real feature list.
| Session restore across restarts | ✅ credential and device both survive |
| Reactions | 🟡 codec done, no UI |
| Photos | ✅ send via the system picker, render inline |
| Voice notes (record + play) | ⚪ |
| Voice notes (record + play) | ✅ AAC/MP4, tap to record, tap to send |
| Stories, and seen-state | ⚪ |
| Groups: create, list, members, leave | ⚪ |
| Group text and images | ⚪ |
@@ -108,7 +108,7 @@ From `E2E/E2EInbox.swift`, which is the real feature list.
| Buddy list, add by username | ✅ |
| Remove, block, unblock | ✅ |
| Member lookup | ⚪ |
| Buddy commands | ⚪ |
| Buddy commands | — retired; not a feature any more (owner, 2026-08-25) |
| Voice calls (WebRTC, DTLS-SRTP, TURN) | ⚪ |
| Notifications list, read state | ✅ |
| Services (AIM/ICQ via `flamenet-oscar`) | ⚪ |
app/src/androidTest/kotlin/io/flamenet/messenger/app/CrudSurfaceTest.kt
+4
View File
@@ -142,6 +142,10 @@ class CrudSurfaceTest {
// Identity state, on the thread it describes rather than buried in a
// settings screen. A freshly added contact is PINNED — trust on first
// use — so the state line and a way to reach the number both exist.
// Voice notes are reachable from the thread, and the control says what
// it does rather than being a bare icon.
compose.onNodeWithTag("record-voice").assertExists()
compose.onNodeWithTag("identity-state").assertExists()
compose.onNodeWithTag("show-safety-number").performClick()
compose.onNodeWithTag("safety-number").assertExists()
app/src/main/AndroidManifest.xml
+9
View File
@@ -3,6 +3,15 @@
<uses-permission android:name="android.permission.INTERNET" />
<!--
Voice notes. Requested at runtime as well: on this minSdk a dangerous
permission declared here is not granted here, and a recorder started
without it fails with a RuntimeException carrying no useful message.
-->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<!-- The app is usable without a microphone; it simply cannot record. -->
<uses-feature android:name="android.hardware.microphone" android:required="false" />
<application
android:allowBackup="false"
android:label="@string/app_name"
app/src/main/kotlin/io/flamenet/messenger/app/ChatScreens.kt
+142 -2
View File
@@ -37,6 +37,15 @@ import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.asImageBitmap
import android.Manifest
import android.content.pm.PackageManager
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.core.content.ContextCompat
import androidx.compose.ui.platform.LocalContext
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.rememberCoroutineScope
import kotlinx.coroutines.launch
import androidx.compose.ui.platform.testTag
import io.flamenet.client.IdentityTrust
import io.flamenet.e2e.SafetyNumber
@@ -188,7 +197,7 @@ private fun AttachedImage(ref: io.flamenet.e2e.Attachment.Ref, model: MessengerV
var failed by remember(ref.id) { mutableStateOf(false) }
LaunchedEffect(ref.id) {
val bytes = model.imageBytes(ref)
val bytes = model.attachmentBytes(ref)
if (bytes == null) { failed = true; return@LaunchedEffect }
bitmap = android.graphics.BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
if (bitmap == null) failed = true
@@ -644,6 +653,75 @@ fun ColumnScope.ChatScreen(model: MessengerViewModel, chat: MessengerViewModel.P
modifier = Modifier.testTag("send"),
) { Text("Send") }
}
// Voice notes sit under the composer rather than inside it: recording
// replaces the row with its own controls, and squeezing a fourth control
// into a row that already has three makes every one of them too small.
if (chat.peerId != null) VoiceComposer(model, chat)
}
/**
* Record and send a voice note.
*
* ⚠️ **Tap to start, tap to send** rather than hold-to-talk. Hold-to-talk reads
* well in a demo and badly on a real phone: the gesture competes with scrolling,
* and letting go early — which happens constantly — silently discards what was
* said. Two deliberate taps cannot be lost by accident.
*/
@Composable
private fun ColumnScope.VoiceComposer(
model: MessengerViewModel,
chat: MessengerViewModel.Phase.Chat,
) {
val context = LocalContext.current
var granted by remember {
mutableStateOf(
ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO)
== PackageManager.PERMISSION_GRANTED
)
}
// ⚠️ Requested when the user asks to record, not at startup. A microphone
// prompt on first launch, before anything has been said, is the kind a
// person denies on reflex — and a denied permission is far harder to
// recover than one that was never asked for.
val ask = rememberLauncherForActivityResult(
ActivityResultContracts.RequestPermission()
) { allowed ->
granted = allowed
if (allowed) model.startRecording()
}
val seconds = chat.recordingSeconds
if (seconds == null) {
TextButton(
onClick = {
if (granted) model.startRecording()
else ask.launch(Manifest.permission.RECORD_AUDIO)
},
modifier = Modifier.testTag("record-voice"),
) { Text("Record a voice note") }
} else {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
// A visible timer: a recorder with no feedback is one people talk
// into after it has already stopped.
Text(
"Recording ${"%.1f".format(seconds)}s",
modifier = Modifier.weight(1f).testTag("recording-timer"),
)
TextButton(
onClick = model::cancelRecording,
modifier = Modifier.testTag("cancel-voice"),
) { Text("Cancel") }
Button(
onClick = model::stopAndSendRecording,
modifier = Modifier.testTag("send-voice"),
) { Text("Send") }
}
}
}
@Composable
@@ -665,7 +743,14 @@ private fun Bubble(line: Session.Line, model: MessengerViewModel) {
)
.padding(horizontal = 12.dp, vertical = 8.dp),
) {
line.attachment?.let { AttachedImage(it, model) }
line.attachment?.let { ref ->
// One field, two very different things. Audio has no thumbnail
// to show and a duration that matters; an image has the
// reverse. Routing on the kind rather than trying to render
// both with one composable.
if (ref.kind == io.flamenet.e2e.Attachment.AUDIO) VoiceBubble(ref, model)
else AttachedImage(ref, model)
}
if (line.text.isNotEmpty()) {
Text(
line.text,
@@ -785,3 +870,58 @@ private fun ColumnScope.IdentityPanel(
}
}
}
/**
* A received voice note: play, stop, and how long it runs.
*
* ⚠️ Fetched and decrypted **on tap**, never on render. A thread of twenty
* notes would otherwise download and decrypt all twenty on scroll — bytes
* nobody asked for, and twenty decrypted files in the cache for the sake of a
* list that has not been listened to.
*/
@Composable
private fun VoiceBubble(ref: io.flamenet.e2e.Attachment.Ref, model: MessengerViewModel) {
val context = LocalContext.current
val scope = rememberCoroutineScope()
val player = remember { VoiceNote.Player(context.cacheDir) }
var playing by remember(ref.id) { mutableStateOf(false) }
var busy by remember(ref.id) { mutableStateOf(false) }
// ⚠️ A MediaPlayer that outlives the composable holds an audio focus and a
// file handle. Leaving the thread mid-playback must stop it, or the note
// keeps playing into a screen that is no longer there.
DisposableEffect(ref.id) { onDispose { player.stop() } }
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
modifier = Modifier.testTag("voice-${ref.id}"),
) {
TextButton(
enabled = !busy,
onClick = {
if (playing) {
player.stop()
playing = false
return@TextButton
}
busy = true
scope.launch {
val bytes = model.attachmentBytes(ref)
busy = false
if (bytes == null) return@launch
runCatching {
player.play(bytes) { playing = false }
playing = true
}
}
},
modifier = Modifier.testTag("play-voice"),
) { Text(if (playing) "Stop" else "Play") }
Text(
if (ref.duration > 0) "%.0fs".format(ref.duration) else "Voice note",
style = MaterialTheme.typography.bodySmall,
)
}
}
app/src/main/kotlin/io/flamenet/messenger/app/MessengerViewModel.kt
+106 -2
View File
@@ -87,6 +87,8 @@ class MessengerViewModel(app: Application) : AndroidViewModel(app) {
* for all of them. Showing a reassuring tick that covers some
* unknown subset of the members would be worse than showing none.
*/
/** Seconds elapsed while recording a voice note, or null when not. */
val recordingSeconds: Double? = null,
val trust: IdentityTrust = IdentityTrust.UNKNOWN,
/** The number both people read aloud. Null until something is pinned. */
val safetyNumber: String? = null,
@@ -675,8 +677,99 @@ class MessengerViewModel(app: Application) : AndroidViewModel(app) {
}.onFailure { failure -> update { it.copy(busy = false, error = describe(failure)) } }
}
/** Bytes for a received image, fetched and cached on first view. */
suspend fun imageBytes(ref: io.flamenet.e2e.Attachment.Ref): ByteArray? {
// -- Voice notes (§6.6) --------------------------------------------------
private var recording: VoiceNote.Recording? = null
private var recordingTicker: kotlinx.coroutines.Job? = null
/**
* Begin recording.
*
* ⚠️ The caller must already hold `RECORD_AUDIO`. Asking here would raise
* the dialog after the user has pressed record, so the first note of every
* install would be lost to a prompt.
*/
fun startRecording() {
if (recording != null) return
val started = runCatching { VoiceNote.start(getApplication<Application>().cacheDir) }
started.onFailure { failure ->
update { it.copy(error = "Could not start recording: ${failure.message}") }
return
}
recording = started.getOrNull()
// A visible timer, because a recorder with no feedback is one people
// talk into after it has already stopped.
recordingTicker = viewModelScope.launch {
while (recording != null) {
val seconds = recording?.elapsedSeconds
update { it.copy(openChat = it.openChat?.copy(recordingSeconds = seconds)) }
kotlinx.coroutines.delay(200)
}
}
}
/** Throw the recording away. Nothing is sent and no file is left behind. */
fun cancelRecording() {
recordingTicker?.cancel()
recordingTicker = null
recording?.cancel()
recording = null
update { it.copy(openChat = it.openChat?.copy(recordingSeconds = null)) }
}
/**
* Stop, and send what was captured.
*
* A recording too short to carry speech is dropped rather than delivered:
* a note that plays silence looks like a delivery bug, and the user who
* fumbled the button already knows they did.
*/
fun stopAndSendRecording() = viewModelScope.launch {
recordingTicker?.cancel()
recordingTicker = null
val finished = withContext(Dispatchers.IO) { recording?.stop() }
recording = null
update { it.copy(openChat = it.openChat?.copy(recordingSeconds = null)) }
val current = session ?: return@launch
val chat = signedIn()?.openChat ?: return@launch
val peerId = chat.peerId
if (finished == null) {
update { it.copy(error = "Too short to send — hold to record.") }
return@launch
}
if (peerId == null) {
update { it.copy(error = "Voice notes are one-to-one for now.") }
return@launch
}
update { it.copy(busy = true, error = null) }
val done = withContext(Dispatchers.IO) {
runCatching {
current.sendVoice(peerId, finished.bytes, finished.mime, finished.seconds)
}
}
done.onSuccess { sent ->
update {
it.copy(
busy = false,
openChat = it.openChat?.copy(lines = current.history(chat.threadId)),
error = if (sent.failed.isEmpty()) null
else "Delivered to ${sent.delivered.size} of " +
"${sent.delivered.size + sent.failed.size} devices.",
)
}
}.onFailure { failure -> update { it.copy(busy = false, error = describe(failure)) } }
}
/**
* Bytes for a received attachment, fetched and cached on first use.
*
* Named for attachments rather than images: photos and voice notes take
* the same path, and a method called `imageBytes` that audio also calls is
* the kind of name that quietly stops being true.
*/
suspend fun attachmentBytes(ref: io.flamenet.e2e.Attachment.Ref): ByteArray? {
val current = session ?: return null
return withContext(Dispatchers.IO) {
runCatching { current.attachmentBytes(ref, getApplication<Application>().cacheDir) }
@@ -809,4 +902,15 @@ class MessengerViewModel(app: Application) : AndroidViewModel(app) {
is Vault.Corrupt -> "This phone's stored keys could not be read."
else -> failure.message ?: "Something went wrong reaching that server."
}
/**
* ⚠️ A recorder outlives the view model unless something stops it, and a
* live MediaRecorder holds the microphone against the whole device.
*/
override fun onCleared() {
recordingTicker?.cancel()
recording?.cancel()
recording = null
super.onCleared()
}
}
app/src/main/kotlin/io/flamenet/messenger/app/Session.kt
+17
View File
@@ -86,6 +86,23 @@ class Session(
return sent
}
/**
* Send a voice note.
*
* The duration is measured while recording rather than read back from the
* container: the sender knows how long they held the button, and parsing
* metadata to rediscover it is a second source of truth that can disagree.
*/
fun sendVoice(peerUserId: Long, bytes: ByteArray, mime: String, seconds: Double):
io.flamenet.client.Conversations.Sent {
val sent = conversations.sendAttachment(
peerUserId, bytes, mime, Attachment.AUDIO, duration = seconds)
if (sent.reachedSomeone) {
record(Vault.threadForUser(peerUserId), "", mine = true, attachment = sent.attachment)
}
return sent
}
/** Bytes for an attachment, cached on disk after the first fetch. */
fun attachmentBytes(ref: Attachment.Ref, cacheDir: java.io.File): ByteArray {
val cached = java.io.File(cacheDir, "att-" + ref.id.replace('/', '_'))
app/src/main/kotlin/io/flamenet/messenger/app/VoiceNote.kt
+162
View File
@@ -0,0 +1,162 @@
package io.flamenet.messenger.app
import android.media.MediaPlayer
import android.media.MediaRecorder
import java.io.File
/**
* Recording and playing voice notes.
*
* Platform APIs only — `MediaRecorder` and `MediaPlayer` — because a voice note
* is an ordinary encrypted attachment once it is bytes, and everything below
* this file already handles that. Nothing here knows about the relay.
*
* AAC in an MP4 container (`audio/mp4`): playable by the other clients without
* a codec conversation, and small enough that a minute of speech is tens of
* kilobytes rather than megabytes — which matters when every byte is uploaded
* encrypted and stored by somebody paying for it.
*/
object VoiceNote {
/** What a finished recording produced. */
data class Recorded(val bytes: ByteArray, val mime: String, val seconds: Double) {
override fun equals(other: Any?): Boolean =
other is Recorded && bytes.contentEquals(other.bytes) &&
mime == other.mime && seconds == other.seconds
override fun hashCode(): Int = bytes.contentHashCode()
}
const val MIME = "audio/mp4"
/**
* A recording in progress.
*
* ⚠️ **Holds a file the caller must dispose of.** Both [stop] and [cancel]
* delete it; a path that does neither leaves plaintext audio in the cache
* directory, which is exactly the thing this product is not supposed to do.
*/
class Recording internal constructor(
private val recorder: MediaRecorder,
private val file: File,
private val startedAtMillis: Long,
) {
/**
* Stop, and return the bytes.
*
* ⚠️ **`MediaRecorder.stop()` throws if it never captured a frame** —
* a tap rather than a hold, or a permission that was granted but is
* being denied by a policy. The exception is `RuntimeException` with no
* useful message, and it reads as "recording is broken" when the truth
* is "there was nothing to record". Treated as an empty recording,
* because that is what it is.
*/
fun stop(): Recorded? {
val elapsed = (System.currentTimeMillis() - startedAtMillis) / 1000.0
val captured = try {
recorder.stop()
true
} catch (_: RuntimeException) {
false
} finally {
recorder.release()
}
val bytes = if (captured && file.exists()) file.readBytes() else ByteArray(0)
file.delete()
// Below this, the container holds a header and nothing worth
// sending. Better to drop it than to deliver a note that plays
// silence and looks like a delivery bug.
if (!captured || bytes.size < MIN_USEFUL_BYTES || elapsed < MIN_SECONDS) return null
return Recorded(bytes, MIME, elapsed)
}
/** Abandon it, and leave nothing behind. */
fun cancel() {
try {
recorder.stop()
} catch (_: RuntimeException) {
// Never started, or stopped with no frames. Nothing to salvage
// and nothing to report — this path is the user changing their
// mind, not a failure.
} finally {
recorder.release()
file.delete()
}
}
val elapsedSeconds: Double
get() = (System.currentTimeMillis() - startedAtMillis) / 1000.0
}
private const val MIN_SECONDS = 0.4
private const val MIN_USEFUL_BYTES = 512
/**
* Start recording into the cache directory.
*
* @throws IllegalStateException when the microphone is unavailable — most
* often because `RECORD_AUDIO` has not been granted. The caller asks for
* the permission first; this does not, because a library that raises
* dialogs is a library that raises them at the wrong moment.
*/
fun start(cacheDir: File): Recording {
val file = File.createTempFile("voice-", ".m4a", cacheDir)
val recorder = MediaRecorder().apply {
setAudioSource(MediaRecorder.AudioSource.MIC)
setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
// Speech, not music. Mono at 32 kbit/s is clearly intelligible and
// roughly a tenth the size of a stereo default nobody asked for.
setAudioChannels(1)
setAudioSamplingRate(44_100)
setAudioEncodingBitRate(32_000)
setOutputFile(file.absolutePath)
}
try {
recorder.prepare()
recorder.start()
} catch (e: Exception) {
recorder.release()
file.delete()
throw IllegalStateException("could not start recording: ${e.message}", e)
}
return Recording(recorder, file, System.currentTimeMillis())
}
/**
* Play decrypted bytes.
*
* ⚠️ Writes them to a file first, because `MediaPlayer` cannot be handed a
* byte array. The file lives in the cache and is deleted when playback
* ends or is stopped — decrypted audio sitting on disk after the user has
* finished with it is a leak, however briefly.
*/
class Player(private val cacheDir: File) {
private var player: MediaPlayer? = null
private var file: File? = null
fun play(bytes: ByteArray, onFinished: () -> Unit) {
stop()
val target = File.createTempFile("play-", ".m4a", cacheDir).also { it.writeBytes(bytes) }
file = target
player = MediaPlayer().apply {
setDataSource(target.absolutePath)
setOnCompletionListener {
stop()
onFinished()
}
prepare()
start()
}
}
fun stop() {
player?.let { runCatching { it.stop() }; it.release() }
player = null
file?.delete()
file = null
}
val isPlaying: Boolean get() = player?.isPlaying == true
}
}