Merge main: pick up the regenerated doc pages
This commit is contained in:
Paul Hitt
+7
-7
@@ -1,12 +1,6 @@
|
||||
# Copy to .env and fill in. Generate the three secrets with:
|
||||
#
|
||||
# openssl rand -hex 32 # # Host port the relay is published on, on 127.0.0.1 only. Change this rather
|
||||
# than editing docker-compose.yml — 8080 is frequently already taken on a
|
||||
# machine that self-hosts other things, and editing the compose file makes the
|
||||
# next `git pull` conflict.
|
||||
FN_HOST_PORT=8080
|
||||
|
||||
POSTGRES_PASSWORD
|
||||
# openssl rand -hex 32 # POSTGRES_PASSWORD
|
||||
# openssl rand -base64 32 # FN_AT_REST_SECRET
|
||||
#
|
||||
# POSTGRES_PASSWORD is hex, not base64, and that is not a style choice: it is
|
||||
@@ -22,6 +16,12 @@ POSTGRES_PASSWORD
|
||||
# Keep issuer.pem. Losing it invalidates every token you have issued; leaking it
|
||||
# lets anyone mint tokens this relay will accept.
|
||||
|
||||
# Host port the relay is published on, on 127.0.0.1 only. Change this rather
|
||||
# than editing docker-compose.yml — 8080 is frequently already taken on a
|
||||
# machine that self-hosts other things, and editing the compose file makes the
|
||||
# next `git pull` conflict.
|
||||
FN_HOST_PORT=8080
|
||||
|
||||
POSTGRES_PASSWORD=
|
||||
FN_AT_REST_SECRET=
|
||||
FN_ISSUER_PRIVATE_KEY=
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
<!-- GENERATED by Scripts/build-doc-pages.sh from flamenet-e2e/SELF-HOSTING.md.
|
||||
Edits made HERE are deleted the next time that script runs.
|
||||
Change the markdown source and re-run it.
|
||||
source-sha256: a4ed8fd6048d30cd -->
|
||||
source-sha256: 0f36788aea97abd7 -->
|
||||
<article class="band doc doc-generated">
|
||||
<div class="wrap wrap-narrow">
|
||||
<h1 id="self-hosting">Self-hosting</h1>
|
||||
@@ -52,12 +52,24 @@ told.</p>
|
||||
cd flamenet-relay
|
||||
cp .env.example .env</code></pre>
|
||||
<p>Generate the three secrets <code>.env</code> asks for:</p>
|
||||
<pre class="bash"><code>openssl rand -base64 32 #("#") POSTGRES_PASSWORD
|
||||
<pre class="bash"><code>openssl rand -hex 32 #("#") POSTGRES_PASSWORD
|
||||
openssl rand -base64 32 #("#") FN_AT_REST_SECRET
|
||||
|
||||
openssl genpkey -algorithm ed25519 -out issuer.pem
|
||||
openssl pkey -in issuer.pem -outform DER | tail -c 32 | base64 #("#") FN_ISSUER_PRIVATE_KEY
|
||||
openssl pkey -in issuer.pem -pubout -outform DER | tail -c 32 | base64 #("#") FN_ISSUER_PUBLIC_KEY</code></pre>
|
||||
<p>🔴 <strong><code>-hex</code> for the database password, not
|
||||
<code>-base64</code>, and this one is not cosmetic.</strong> It is
|
||||
interpolated into
|
||||
<code>postgres://flamenet:<password>@db:5432/flamenet</code>,
|
||||
where base64's <code>/</code>, <code>+</code> and <code>=</code> are all
|
||||
meaningful. A password containing <code>/</code> makes that URL invalid
|
||||
and the relay dies at start-up with a bare <code>Code=-1000</code> and
|
||||
the connection string echoed back at you — which reads like the database
|
||||
is unreachable rather than like the password needs escaping.
|
||||
<code>openssl rand -base64 32</code> produces one of those roughly
|
||||
<strong>three times in four</strong>, so following the old instructions
|
||||
usually failed.</p>
|
||||
<p>Then create the proxy network the compose file attaches to, and
|
||||
start:</p>
|
||||
<pre class="bash"><code>docker network create web
|
||||
|
||||
+39
-18
@@ -3,7 +3,7 @@
|
||||
<!-- GENERATED by Scripts/build-doc-pages.sh from flamenet-e2e/docs/SPEC.md.
|
||||
Edits made HERE are deleted the next time that script runs.
|
||||
Change the markdown source and re-run it.
|
||||
source-sha256: 41002a1ef1cc4ff5 -->
|
||||
source-sha256: 50c7c8d5420e65d1 -->
|
||||
<article class="band doc doc-generated">
|
||||
<div class="wrap wrap-narrow">
|
||||
<h1 id="flamenet-messenger--end-to-end-encryption-spec-v1">Flamenet
|
||||
@@ -1956,7 +1956,7 @@ status for §13 and §14</h2>
|
||||
<td>n/a — no server change needed</td>
|
||||
<td>✅</td>
|
||||
<td>✅ verified cross-engine</td>
|
||||
<td>⚠️ built, not yet cross-engine</td>
|
||||
<td>✅ verified cross-engine</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>§14 sealed sender</td>
|
||||
@@ -1964,7 +1964,7 @@ status for §13 and §14</h2>
|
||||
<td>✅ sealed mode + delivery keys</td>
|
||||
<td>✅ wired end to end</td>
|
||||
<td>✅ wired end to end</td>
|
||||
<td>⚠️ wired, run not recorded</td>
|
||||
<td>⚠️ engine verified, not wired end to end</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
@@ -1973,21 +1973,42 @@ understating, which is the opposite of the failure §10 warns about but
|
||||
is corrected the same way: by establishing what has been run, not by
|
||||
flipping a symbol because the code looks present.</p>
|
||||
<ul>
|
||||
<li><strong>§13 on Android</strong> is implemented as of
|
||||
<code>98342dd</code>:
|
||||
<code>PQs</code>/<code>PQr</code>/<code>PQpending</code> on the session,
|
||||
<code>KDF_RK_v3</code>, both header fields, and the
|
||||
decapsulate-before-adopt ordering. Sixteen tests, including one pinning
|
||||
the canonical header as bytes so a reordered field cannot pass. ⚠️ It is
|
||||
<strong>not</strong> ✅ because no v3 session has been run between the
|
||||
Android and iOS engines. That run is what the row needs, and §12.10's
|
||||
conformance list is the shape it should take.</li>
|
||||
<li><strong>§14 on Android</strong> was never ❌ at all:
|
||||
<code>SealedSender.kt</code> is in the engine and delivery keys are
|
||||
wired through <code>Conversations</code>, <code>RelayClient</code> and
|
||||
<code>Session</code>. It is ⚠️ rather than ✅ because this table records
|
||||
<em>runs</em>, and no Android sealed sender run is written down here.
|
||||
The code being present is not the claim.</li>
|
||||
<li><p><strong>§13 on Android</strong> is ✅ as of <code>cc51854</code>.
|
||||
Thirteen assertions in <code>test/interop-kotlin.mjs</code> drive a v3
|
||||
session between the Kotlin and web engines in both directions. The
|
||||
load-bearing one is the web engine opening Kotlin's reply: sending it
|
||||
required decapsulating with the prekey seed, adopting the peer's
|
||||
<code>pqdh</code>, and encapsulating a fresh key, and doing those three
|
||||
in the wrong order (§13.2) derives a different root key with no error
|
||||
anywhere. Also pinned: a distinct ML-KEM key per step, so a reused one —
|
||||
which still decrypts, and silently gives up the healing §13.1 exists for
|
||||
— fails.</p>
|
||||
<p>🔴 <strong>That run is also what found the shipped code was
|
||||
wrong.</strong> The engine was attaching
|
||||
<code>pqdh</code>/<code>pqct</code> to messages for any peer that
|
||||
published a PQ prekey, which every v2 device does. A build with that
|
||||
would have broken every conversation with the released 0.6 client —
|
||||
different root key AND different AAD, so the first message never opens.
|
||||
Sixteen Kotlin unit tests did not catch it and could not: both sides of
|
||||
a self-test share the author's reading. Recorded because it is the
|
||||
argument for this table's whole discipline.</p>
|
||||
<p>⚠️ Cross-engine here means Kotlin against the <strong>web</strong>
|
||||
engine, which is normative. Kotlin against Swift directly has still not
|
||||
been run.</p></li>
|
||||
<li><p><strong>§14 on Android</strong> is ⚠️, and the reason moved
|
||||
rather than went away. The engine half is now verified cross-engine —
|
||||
six assertions seal in one engine and open in the other, both
|
||||
directions, including a seal addressed elsewhere that must not open.
|
||||
What is unproven is the column's actual claim, <em>wired end to
|
||||
end</em>: no recorded run shows an Android client sealing and unsealing
|
||||
against a relay.</p>
|
||||
<p>⚠️ <strong><code>LiveSealedSenderTest</code> is not that evidence,
|
||||
and reads as though it is.</strong> It returns early when
|
||||
<code>FN_LIVE_FLEET</code> is unset, and JUnit records the early return
|
||||
as <strong>passed</strong> —
|
||||
<code>tests="2" skipped="0" failures="0"</code> on a run that did
|
||||
nothing. A test that no-ops into green is worse than an absent
|
||||
one.</p></li>
|
||||
</ul>
|
||||
<p><strong>§13</strong> is complete and verified end to end on the web:
|
||||
32 tests in <code>test/ratchet.mjs</code> covering lockstep advance with
|
||||
|
||||
Executable
+145
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run the published self-hosting quick start, exactly as SELF-HOSTING.md gives it.
|
||||
#
|
||||
# 🔴 **The quick start broke, silently, and stayed broken for a day.** Three
|
||||
# separate faults at once — and the one that survived longest was a missing
|
||||
# `docker network create web`, which fails only at `up` time, after the images
|
||||
# have been pulled. Nothing in this repository ran those instructions, so
|
||||
# nothing said so. The people it failed for were self-hosters, at the exact
|
||||
# moment they were deciding whether this project works.
|
||||
#
|
||||
# ⚠️ **It has to actually BOOT.** `docker compose config` renders that compose
|
||||
# file happily with no `web` network in existence: the external-network check
|
||||
# happens at `up`. A job that only validated the YAML would have passed
|
||||
# throughout the outage it exists to prevent.
|
||||
#
|
||||
# Scripts/selfhost-smoke.sh # published image, the documented path
|
||||
# Scripts/selfhost-smoke.sh --build # build from source, the ARM path
|
||||
#
|
||||
# 🔴 **Not wired into CI, and not for want of trying.** This project's runner is
|
||||
# a docker executor with `privileged = false` and no socket mounted, so a job
|
||||
# here gets an alpine container with no bash, no docker CLI and no way to reach
|
||||
# a daemon — `env: can't execute bash` is as far as it got. Running it in CI
|
||||
# needs a runner change (privileged, a socket mount, or a shell executor), which
|
||||
# is a decision about a machine that also serves production.
|
||||
#
|
||||
# ⚠️ Until then this is a RELEASE-TIME check somebody has to run, on any machine
|
||||
# with docker:
|
||||
#
|
||||
# ssh <box> "cd /srv/prod/messenger/src && ./Scripts/selfhost-smoke.sh"
|
||||
#
|
||||
# A check that only runs when remembered is weaker than one that cannot be
|
||||
# forgotten. It is still worth having: on its first two runs it found the
|
||||
# missing `docker network create web` AND a documented password that broke the
|
||||
# relay three times in four.
|
||||
#
|
||||
# Leaves nothing behind: a scratch directory, its own compose project name, and
|
||||
# a trap that tears down volumes as well as containers.
|
||||
set -euo pipefail
|
||||
|
||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
mode="${1:-}"
|
||||
project="selfhost-smoke-$$"
|
||||
# ⚠️ Its own network name, not `web`. A shared name would collide with a real
|
||||
# proxy network on a machine that has one — and this script deletes it at the
|
||||
# end, which on the production box would be a very bad afternoon.
|
||||
network="${project}-web"
|
||||
work="$(mktemp -d)"
|
||||
|
||||
cleanup() {
|
||||
docker compose -p "$project" --project-directory "$work" \
|
||||
-f "$work/docker-compose.yml" -f "$work/docker-compose.override.yml" \
|
||||
down -v --remove-orphans >/dev/null 2>&1 || true
|
||||
docker network rm "$network" >/dev/null 2>&1 || true
|
||||
rm -rf "$work"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
echo "SELFHOST: laying out a fresh checkout the way a self-hoster would"
|
||||
cp "$here/docker-compose.yml" "$work/docker-compose.yml"
|
||||
cp "$here/docker-compose.build.yml" "$work/docker-compose.build.yml" 2>/dev/null || true
|
||||
cp "$here/.env.example" "$work/.env"
|
||||
# The build path needs the sources the Dockerfile copies.
|
||||
if [ "$mode" = "--build" ]; then
|
||||
cp -R "$here/Sources" "$here/Tests" "$here/Resources" "$here/Public" \
|
||||
"$here/Package.swift" "$here/Package.resolved" "$here/Dockerfile" "$work/" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
echo "SELFHOST: generating the secrets .env asks for"
|
||||
# The same four commands the documentation prints. Generated rather than
|
||||
# hardcoded, so a change to what the compose file demands shows up here.
|
||||
pem="$work/issuer.pem"
|
||||
openssl genpkey -algorithm ed25519 -out "$pem" 2>/dev/null
|
||||
set_env() {
|
||||
# ⚠️ In-place edit that tolerates an empty value on the right of `=`, which is
|
||||
# what .env.example ships. Anchored to the line start so a variable whose name
|
||||
# is a suffix of another one cannot be hit.
|
||||
python3 - "$work/.env" "$1" "$2" <<'PY'
|
||||
import sys, re
|
||||
path, key, val = sys.argv[1], sys.argv[2], sys.argv[3]
|
||||
text = open(path, encoding='utf-8').read()
|
||||
new, n = re.subn(rf'^{re.escape(key)}=.*$', f'{key}={val}', text, count=1, flags=re.M)
|
||||
if n != 1:
|
||||
sys.exit(f'{key} is not in .env.example — the documented steps have drifted')
|
||||
open(path, 'w', encoding='utf-8').write(new)
|
||||
PY
|
||||
}
|
||||
# 🔴 **hex, not base64, and this is the documented command for a reason.**
|
||||
# `openssl rand -base64 32` emits `/` and `+` about 73% of the time, and the
|
||||
# password goes into `postgres://user:PASSWORD@db/...` unencoded — a `/` ends
|
||||
# the authority and the relay dies with `NSURLErrorDomain Code=-1000`, which
|
||||
# names the URL and says nothing about passwords. This script generates what
|
||||
# SELF-HOSTING.md tells people to generate; if that drifts back to base64, this
|
||||
# fails again and says why.
|
||||
set_env POSTGRES_PASSWORD "$(openssl rand -hex 32)"
|
||||
set_env FN_AT_REST_SECRET "$(openssl rand -base64 32 | tr -d '\n')"
|
||||
set_env FN_ISSUER_PRIVATE_KEY "$(openssl pkey -in "$pem" -outform DER 2>/dev/null | tail -c 32 | base64 | tr -d '\n')"
|
||||
set_env FN_ISSUER_PUBLIC_KEY "$(openssl pkey -in "$pem" -pubout -outform DER 2>/dev/null | tail -c 32 | base64 | tr -d '\n')"
|
||||
|
||||
# A port nobody else is on, so this cannot collide with a relay already running
|
||||
# on the machine — which on the deployment box is the production one.
|
||||
port=18973
|
||||
set_env FN_HOST_PORT "$port"
|
||||
|
||||
echo "SELFHOST: docker network create (the step that was missing)"
|
||||
docker network create "$network" >/dev/null
|
||||
# The compose file names the external network `web`; point it at ours for the
|
||||
# duration without editing the file a self-hoster would actually use.
|
||||
cat > "$work/docker-compose.override.yml" <<YAML
|
||||
networks:
|
||||
web:
|
||||
name: ${network}
|
||||
external: true
|
||||
YAML
|
||||
|
||||
echo "SELFHOST: docker compose up -d"
|
||||
# ⚠️ Absolute paths. `-f` resolves against the CWD, NOT against
|
||||
# `--project-directory` — a relative name here is looked for wherever the script
|
||||
# happens to be run from, which is how this first failed: "open
|
||||
# /srv/prod/messenger/src/docker-compose.override.yml: no such file".
|
||||
files=(-f "$work/docker-compose.yml" -f "$work/docker-compose.override.yml")
|
||||
[ "$mode" = "--build" ] && files=(-f "$work/docker-compose.yml" -f "$work/docker-compose.build.yml" -f "$work/docker-compose.override.yml")
|
||||
build_flag=()
|
||||
[ "$mode" = "--build" ] && build_flag=(--build)
|
||||
docker compose -p "$project" --project-directory "$work" "${files[@]}" up -d "${build_flag[@]}"
|
||||
|
||||
echo "SELFHOST: waiting for the relay to answer"
|
||||
up=""
|
||||
for _ in $(seq 1 60); do
|
||||
if [ "$(curl -sf -m 2 "http://127.0.0.1:${port}/health" || true)" = "ok" ]; then up=1; break; fi
|
||||
sleep 2
|
||||
done
|
||||
if [ -z "$up" ]; then
|
||||
echo "SELFHOST FAILED: no /health after 120s — the documented quick start does not work"
|
||||
docker compose -p "$project" --project-directory "$work" "${files[@]}" logs --tail 40 relay || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 🔴 The landing page, not just /health. "The relay serves its own landing page,
|
||||
# so a working install explains itself rather than answering with a 404" is a
|
||||
# promise the compose file's own header makes, and a missing Resources copy in
|
||||
# the image would leave /health green and the page a 404.
|
||||
code=$(curl -s -m 5 -o /dev/null -w '%{http_code}' "http://127.0.0.1:${port}/")
|
||||
[ "$code" = "200" ] || { echo "SELFHOST FAILED: landing page answered $code"; exit 1; }
|
||||
|
||||
echo "SELFHOST OK: the documented quick start boots and serves its landing page"
|
||||
Reference in New Issue
Block a user