Resources/Views/configuration.leaf

@@ -3,7 +3,7 @@
 <!-- GENERATED by Scripts/build-doc-pages.sh from docs/CONFIGURATION.md.
      Edits made HERE are deleted the next time that script runs.
      Change the markdown source and re-run it.
-     source-sha256: 00f2d337316548ad -->
+     source-sha256: 4bbbe5461d19a20a -->
 <article class="band doc doc-generated">
   <div class="wrap wrap-narrow">
 <h1 id="configuring-a-relay">Configuring a relay</h1>
@@ -431,6 +431,51 @@ looking wrong. With this set, the publishable key stays the
 </tr>
 </tbody>
 </table>
+<h2 id="watching-the-key-log-relay32">Watching the key log
+(relay#("#")32)</h2>
+<table>
+<thead>
+<tr>
+<th>Variable</th>
+<th>Default</th>
+<th>What it does</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><code>FN_KEY_LOG_SUBJECT_QUERIES</code></td>
+<td>on</td>
+<td>Set to <code>false</code>, <code>0</code>, <code>no</code> or
+<code>off</code> to stop listing a single account's entries. The log
+head and the inclusion proofs still work — only per-subject enumeration
+goes away, and the route then refuses with
+<code>subject_queries_disabled</code> rather than a 404, so a monitor
+can tell "this relay does not answer that" from "there is no log
+here".</td>
+</tr>
+</tbody>
+</table>
+<p><code>GET /e2e/log/subject?subject=&lt;fnid&gt;</code> lists every
+entry published under one account — publications and retirements, with
+the device each names. It is unauthenticated, like the rest of §19: a
+log only its own users can read is not a transparency log, and a monitor
+is by definition somebody else.</p>
+<p>⚠️ <strong>That has a cost, and it is stated here rather than left to
+be discovered.</strong> Anyone holding somebody's fnid — the address
+they hand to contacts — can see how many devices they have published and
+roughly when. This is inherent to key transparency rather than an
+oversight: a substitution cannot be detected in a log nobody may read.
+What it is not is a directory; you need the identifier first.</p>
+<p><code>Scripts/watch-key-log.py</code> is a self-hostable monitor,
+forty lines of stdlib. 🔴 Run it somewhere you control — a hosted
+service everybody is enrolled in would learn which accounts are worth
+watching, which is a list nobody should be assembling.</p>
+<p>⚠️ It reports what the relay <strong>serves</strong>. A relay showing
+one log to you and another to your contact is the fork §19.4's head
+gossip exists to catch, and no single watcher sees it; witness
+co-signing (relay#("#")23) is the answer to that and does not exist yet. And
+a relay that stops answering is not an all-clear — the script says
+"could not reach", never "no changes".</p>
 
   </div>
 </article>

Scripts/watch-key-log.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+"""Watch a relay's key log for devices published under one account (relay#32).
+
+    watch-key-log.py --relay https://relay.example --subject <fnid> [--state file]
+
+Prints a line whenever a device is published or retired under that subject, and
+exits non-zero if anything changed since the last run — so cron plus a mail alias
+is a complete monitor.
+
+🔴 **Run this somewhere you control.** A hosted service everybody is enrolled in
+would learn which accounts are worth watching, which is a list nobody should be
+assembling. That is why this is forty lines of stdlib and no dependencies: the
+point is that you can read it and run it yourself.
+
+⚠️ **What it can and cannot tell you.** It reports what the relay SERVES. A relay
+that shows one log to you and another to your contact is the fork §19.4's head
+gossip exists to catch, and this script cannot see it alone — relay#23's
+co-signing witnesses are the answer to that, and they do not exist yet. What this
+does catch is the ordinary case: a device appearing under your account that you
+did not add.
+
+⚠️ A relay that simply stops answering is not proof of anything, and this says
+"could not reach" rather than "no changes". Silence is not an all-clear.
+"""
+import argparse
+import json
+import os
+import sys
+import urllib.parse
+import urllib.request
+
+
+def fetch(relay, subject):
+    url = (relay.rstrip("/") + "/e2e/log/subject?subject="
+           + urllib.parse.quote(subject, safe=""))
+    with urllib.request.urlopen(url, timeout=30) as r:
+        return json.load(r)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--relay", required=True)
+    ap.add_argument("--subject", required=True,
+                    help="the account's fnid, or local:<audience>/<user id>")
+    ap.add_argument("--state", default="key-log-watch.json")
+    args = ap.parse_args()
+
+    try:
+        answer = fetch(args.relay, args.subject)
+    except Exception as e:                      # noqa: BLE001 — any failure is the same answer
+        # ⚠️ Not "no changes". A relay that has stopped answering is worth
+        # noticing and is also just an outage; saying which would be inventing.
+        print(f"could not reach the log: {e}", file=sys.stderr)
+        return 2
+
+    entries = answer.get("entries", [])
+    seen = {f"{e['index']}:{e['kind']}:{e['device_id']}" for e in entries}
+
+    known = set()
+    if os.path.exists(args.state):
+        with open(args.state) as f:
+            known = set(json.load(f).get("seen", []))
+
+    fresh = sorted(seen - known, key=lambda k: int(k.split(":", 1)[0]))
+    for key in fresh:
+        index, kind, device = key.split(":", 2)
+        at = next((e.get("at") for e in entries if e["index"] == int(index)), None)
+        print(f"{kind:<8} {device}  index {index}  {at or 'no timestamp'}")
+
+    with open(args.state, "w") as f:
+        json.dump({"subject": args.subject, "seen": sorted(seen)}, f, indent=1)
+
+    if not known:
+        # ⚠️ The first run is a baseline, not a report. Treating everything as
+        # new the first time would train whoever runs this to ignore it.
+        print(f"baseline: {len(seen)} entries recorded, nothing to compare against yet",
+              file=sys.stderr)
+        return 0
+    return 1 if fresh else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

Sources/FlamenetRelay/Config.swift

@@ -117,6 +117,20 @@ struct Config: Sendable {
     /// per-address limit — for an operator who would rather admit an abuser
     /// than risk refusing a campus.
     let signupMaxPerNetworkPerHour: Int
+
+    /// Whether `GET /e2e/log/subject` will list one account's log entries.
+    ///
+    /// **On by default.** A substitution cannot be detected in a log nobody may
+    /// read, and a monitor is by definition somebody else — so an unauthenticated
+    /// read is the feature, not a leak in it.
+    ///
+    /// ⚠️ It does have a cost, and an operator is allowed to weigh it
+    /// differently: anyone holding an fnid can see how many devices that account
+    /// has published and roughly when. That is inherent to key transparency
+    /// rather than an oversight, and it is not a directory — you need the
+    /// identifier first. Off leaves the head and the proofs, which is the part
+    /// clients depend on; only per-subject enumeration goes away.
+    let keyLogSubjectQueries: Bool
     let attachmentBytesPerHour: Int
     /// The rolling 30-day per-account upload budget (C3). The hourly limits
     /// above stop bursts; this stops a steady drip from quietly owning the
@@ -242,6 +256,12 @@ struct Config: Sendable {
             signupMaxPerHour: int("FN_SIGNUP_MAX_PER_HOUR", AccountRoutes.signupMaxPerHourDefault),
             signupMaxPerNetworkPerHour:
                 int("FN_SIGNUP_MAX_PER_NETWORK", AccountRoutes.signupMaxPerNetworkDefault),
+
+            keyLogSubjectQueries: {
+                // On unless explicitly switched off.
+                let raw = (env["FN_KEY_LOG_SUBJECT_QUERIES"] ?? "").lowercased()
+                return !(raw == "false" || raw == "0" || raw == "no" || raw == "off")
+            }(),
             attachmentBytesPerHour: int("FN_ATTACHMENT_BYTES_RATE", 200 * 1024 * 1024),
             attachmentBytesPerMonth: int("FN_ATTACHMENT_BYTES_PER_MONTH", 500 * 1024 * 1024),
             attachmentRetentionDays: int("FN_ATTACHMENT_RETENTION_DAYS", 30),

Sources/FlamenetRelay/Entrypoint.swift

@@ -901,6 +901,10 @@ func register(routes r: Routes, on app: Application) throws {
         e2e.get("log", "head") { req in try await r.logHead(req) }
         e2e.get("log", "inclusion") { req in try await r.logInclusion(req) }
         e2e.get("log", "consistency") { req in try await r.logConsistency(req) }
+        // relay#32 — every entry under one subject, so a monitor can watch an
+        // account for devices its owner did not add. A query parameter, not a
+        // path segment: a subject may contain a slash.
+        e2e.get("log", "subject") { req in try await r.logSubjectEntries(req) }
         e2e.get("log", "device", ":device_id") { req in
             guard let id = req.parameters.get("device_id") else {
                 throw WireError.badRequest("Malformed device id.")

Sources/FlamenetRelay/RoutesKeyLog.swift

@@ -127,4 +127,70 @@ extension Routes {
                              at: row.createdAt.map { ISO8601DateFormatter().string(from: $0) })
             }))
     }
+
+    /// §19 — every entry published under one subject (relay#32).
+    ///
+    /// 🔴 **Without this a monitor cannot monitor.** Watching an account means
+    /// noticing a device published under it that its owner did not add — and
+    /// the only lookup that existed was by device id, which requires already
+    /// knowing the device the monitor is trying to discover. The `subject`
+    /// column was there; nothing exposed it.
+    ///
+    /// ⚠️ **A query parameter, not a path segment.** `KeyLog.logSubject` falls
+    /// back to `local:<audience>/<userID>` for an account with no fnid, and
+    /// that contains a slash — a path route would silently 404 for exactly the
+    /// accounts least able to notice.
+    ///
+    /// ⚠️ **Unauthenticated, like the rest of §19, and that has a cost worth
+    /// naming.** Anyone holding somebody's fnid — which is the address they
+    /// hand to contacts — can see how many devices they have published and
+    /// when. That is inherent to key transparency rather than a slip: a
+    /// substitution cannot be detected in a log nobody may read. It is stated
+    /// in the docs rather than left to be discovered.
+    func logSubjectEntries(_ request: Request) async throws -> Response {
+        // ⚠️ A code, not a 404: a relay that has enumeration switched off still
+        // serves the log head and the proofs, and a monitor should say "this
+        // relay does not answer that" rather than "there is no log here".
+        guard config.keyLogSubjectQueries else {
+            throw WireError("subject_queries_disabled",
+                "This server does not list a single account's log entries.", 501)
+        }
+        guard let subject = request.query[String.self, at: "subject"],
+              !subject.isEmpty, subject.count <= 256
+        else { throw WireError.badRequest("Missing or malformed subject.") }
+
+        // Same ordering rule as `logDevice`: the Merkle index is a POSITION in
+        // the tree, not a row id.
+        let ordered = try await LogEntry.query(on: request.db)
+            .sort(\.$id, .ascending).all()
+        var indexByRow: [Int: Int] = [:]
+        for (position, row) in ordered.enumerated() {
+            if let id = row.id { indexByRow[id] = position }
+        }
+        let rows = ordered.filter { $0.subject == subject }
+        return try .json(LogSubjectResponse(
+            subject: subject,
+            size: ordered.count,
+            entries: rows.compactMap { row in
+                guard let id = row.id, let index = indexByRow[id] else { return nil }
+                return .init(index: index, kind: row.kind, device_id: row.deviceID,
+                             leaf: row.leafHash,
+                             at: row.createdAt.map { ISO8601DateFormatter().string(from: $0) })
+            }))
+    }
+}
+
+struct LogSubjectResponse: Content {
+    let subject: String
+    let size: Int
+    let entries: [Entry]
+    struct Entry: Content {
+        let index: Int
+        let kind: String
+        /// Named here, unlike the by-device route, because a monitor's whole
+        /// job is learning which devices exist.
+        let device_id: String
+        let leaf: String
+        let at: String?
+    }
 }

Tests/FlamenetRelayTests/LogSubjectTests.swift

@@ -0,0 +1,170 @@
+import Fluent
+import Foundation
+import Testing
+import Vapor
+
+@testable import FlamenetRelay
+
+/// Watching the key log for one account (relay#32).
+///
+/// 🔴 **Without a by-subject lookup a monitor cannot monitor.** Watching an
+/// account means noticing a device published under it that its owner did not
+/// add — and the only lookup that existed was by device id, which requires
+/// already knowing the device the monitor is trying to discover.
+@Suite("key log by subject")
+struct LogSubjectTests {
+
+    private func entry(_ db: any Database, subject: String, device: String,
+                       kind: String, user: Int = 1) async throws {
+        let row = LogEntry()
+        row.leafHash = Data("\(subject)|\(device)|\(kind)".utf8).base64EncodedString()
+        row.userID = user
+        row.subject = subject
+        row.deviceID = device
+        row.kind = kind
+        try await row.create(on: db)
+    }
+
+    private func fetch(_ h: RelayHarness, subject: String) async throws -> String {
+        let escaped = subject.addingPercentEncoding(
+            withAllowedCharacters: .alphanumerics) ?? subject
+        let req = h.anonymousRequest(
+            .GET, "/e2e/log/subject?subject=" + escaped, body: Data())
+        let response = try await h.routes.logSubjectEntries(req)
+        return response.body.string ?? ""
+    }
+
+    @Test("every entry under one subject comes back")
+    func entriesForASubject() async throws {
+        try await withRelay { h in
+            try await entry(h.db, subject: "fnidA", device: "d1", kind: "publish")
+            try await entry(h.db, subject: "fnidA", device: "d2", kind: "publish")
+            try await entry(h.db, subject: "fnidB", device: "d3", kind: "publish")
+            let body = try await fetch(h, subject: "fnidA")
+            #expect(body.contains("d1"))
+            #expect(body.contains("d2"))
+            #expect(!body.contains("d3"))
+        }
+    }
+
+    /// 🔴 The device id is named, unlike the by-device route — a monitor's
+    /// whole job is learning which devices exist.
+    @Test("the answer names the devices")
+    func namesDevices() async throws {
+        try await withRelay { h in
+            try await entry(h.db, subject: "fnidA", device: "d1", kind: "publish")
+            #expect(try await fetch(h, subject: "fnidA").contains("device_id"))
+        }
+    }
+
+    /// A revocation is exactly what a monitor is watching for.
+    @Test("revocations appear alongside publications")
+    func revocationsAppear() async throws {
+        try await withRelay { h in
+            try await entry(h.db, subject: "fnidA", device: "d1", kind: "publish")
+            try await entry(h.db, subject: "fnidA", device: "d1", kind: "revoke")
+            let body = try await fetch(h, subject: "fnidA")
+            #expect(body.contains("publish"))
+            #expect(body.contains("revoke"))
+        }
+    }
+
+    /// ⚠️ **The Merkle index is a POSITION in the tree**, not a row id — the
+    /// same rule `logDevice` documents. A monitor asking for an inclusion proof
+    /// at the wrong index gets a leaf that does not verify, which reads as
+    /// tampering when it is arithmetic.
+    @Test("indices are positions in the whole log")
+    func indicesArePositions() async throws {
+        try await withRelay { h in
+            try await entry(h.db, subject: "other", device: "d0", kind: "publish")
+            try await entry(h.db, subject: "fnidA", device: "d1", kind: "publish")
+            let body = try await fetch(h, subject: "fnidA")
+            // The entry is second in the whole log, so its index is 1 — not 0,
+            // which is what a per-subject numbering would have produced.
+            #expect(body.contains("\"index\":1"))
+            #expect(body.contains("\"size\":2"))
+        }
+    }
+
+    /// A subject nobody has published under is an empty answer, not an error —
+    /// a monitor starting up must not read "no such account" as an alarm.
+    @Test("an unknown subject answers empty")
+    func unknownSubjectIsEmpty() async throws {
+        try await withRelay { h in
+            try await entry(h.db, subject: "fnidA", device: "d1", kind: "publish")
+            let body = try await fetch(h, subject: "nobody")
+            #expect(body.contains("\"entries\":[]"))
+        }
+    }
+
+    @Test("a missing subject is refused")
+    func missingSubjectRefused() async throws {
+        try await withRelay { h in
+            let req = h.anonymousRequest(.GET, "/e2e/log/subject", body: Data())
+            await #expect(throws: WireError.self) {
+                _ = try await h.routes.logSubjectEntries(req)
+            }
+        }
+    }
+
+    /// 🔴 Unauthenticated, like the rest of §19. A log only its own users can
+    /// read is not a transparency log, and a monitor is by definition somebody
+    /// else.
+    @Test("a stranger may read the log")
+    func strangersMayRead() async throws {
+        try await withRelay { h in
+            try await entry(h.db, subject: "fnidA", device: "d1", kind: "publish", user: 7)
+            #expect(try await fetch(h, subject: "fnidA").contains("d1"))
+        }
+    }
+
+    // MARK: - The switch
+
+    /// On with nothing configured: a substitution cannot be detected in a log
+    /// nobody may read, so the default has to be readable.
+    @Test("subject queries answer with nothing configured")
+    func onByDefault() async throws {
+        try await withRelay { h in
+            let req = h.anonymousRequest(.GET, "/e2e/log/subject?subject=local:test.relay/1")
+            _ = try await h.routes.logSubjectEntries(req)
+        }
+    }
+
+    /// An operator may weigh the enumeration cost differently, and off says so
+    /// with a code rather than pretending the log is not there — the head and
+    /// the proofs still work, and a monitor should be able to tell.
+    @Test("switched off, enumeration is refused but the log still exists")
+    func offRefusesWithACode() async throws {
+        try await withRelay(env: ["FN_KEY_LOG_SUBJECT_QUERIES": "false"]) { h in
+            let req = h.anonymousRequest(.GET, "/e2e/log/subject?subject=local:test.relay/1")
+            do {
+                _ = try await h.routes.logSubjectEntries(req)
+                Issue.record("a relay with enumeration off answered a subject query")
+            } catch let error as WireError {
+                #expect(error.code == "subject_queries_disabled")
+                #expect(error.data.status == 501)
+            }
+        }
+    }
+
+    /// Only an explicit no turns it off — a typo must not quietly stop a relay
+    /// being monitorable.
+    @Test("only an explicit no disables enumeration")
+    func onlyExplicitNoDisables() throws {
+        func flag(_ v: String?) throws -> Bool {
+            var env = [
+                "FN_ISSUER_PUBLIC_KEY": Curve25519.Signing.PrivateKey()
+                    .publicKey.rawRepresentation.base64EncodedString(),
+                "FN_RELAY_AUDIENCE": "test.relay",
+            ]
+            if let v { env["FN_KEY_LOG_SUBJECT_QUERIES"] = v }
+            return try Config.fromEnvironment(env).keyLogSubjectQueries
+        }
+        for on in [nil, "", "true", "yes", "1", "flase"] {
+            #expect(try flag(on) == true, "\(on ?? "unset") should leave it on")
+        }
+        for off in ["false", "FALSE", "0", "no", "off"] {
+            #expect(try flag(off) == false, "\(off) should switch it off")
+        }
+    }
+}

docs/CONFIGURATION.md

@@ -153,3 +153,32 @@ All optional. Each is absent rather than half-enabled when unconfigured.
 | `FN_DONATE_PROVIDER`, `FN_DONATE_SECRET` | Off unless set. A relay somebody else runs has no business soliciting money for us. |
 | `FN_DONATE_PUBLISHABLE_KEY` | Required alongside the two above: the payment fields are Stripe's own, mounted on the page, and they cannot mount without it. Set the secret and not this and the donate page shows no form at all. |
 | `FN_DONATE_ACCOUNT` | The connected account to charge, if `FN_DONATE_SECRET` belongs to a Stripe *platform* rather than to the account meant to receive the money. Sent as `Stripe-Account`, which makes the payment a direct charge there. Blank charges whatever account the key itself belongs to — the right answer when you are self-hosting with your own key, and the wrong one with a platform key, where Stripe answers 200 and the money lands on the platform with nothing on the page looking wrong. With this set, the publishable key stays the **platform's**. |
+
+
+## Watching the key log (relay#32)
+
+| Variable | Default | What it does |
+|---|---|---|
+| `FN_KEY_LOG_SUBJECT_QUERIES` | on | Set to `false`, `0`, `no` or `off` to stop listing a single account's entries. The log head and the inclusion proofs still work — only per-subject enumeration goes away, and the route then refuses with `subject_queries_disabled` rather than a 404, so a monitor can tell "this relay does not answer that" from "there is no log here". |
+
+`GET /e2e/log/subject?subject=<fnid>` lists every entry published under one
+account — publications and retirements, with the device each names. It is
+unauthenticated, like the rest of §19: a log only its own users can read is not a
+transparency log, and a monitor is by definition somebody else.
+
+⚠️ **That has a cost, and it is stated here rather than left to be discovered.**
+Anyone holding somebody's fnid — the address they hand to contacts — can see how
+many devices they have published and roughly when. This is inherent to key
+transparency rather than an oversight: a substitution cannot be detected in a log
+nobody may read. What it is not is a directory; you need the identifier first.
+
+`Scripts/watch-key-log.py` is a self-hostable monitor, forty lines of stdlib.
+🔴 Run it somewhere you control — a hosted service everybody is enrolled in would
+learn which accounts are worth watching, which is a list nobody should be
+assembling.
+
+⚠️ It reports what the relay **serves**. A relay showing one log to you and
+another to your contact is the fork §19.4's head gossip exists to catch, and no
+single watcher sees it; witness co-signing (relay#23) is the answer to that and
+does not exist yet. And a relay that stops answering is not an all-clear — the
+script says "could not reach", never "no changes".