Absorb spec §4.1/§4.2 canonicalization and §9 key-management simplification #10

## Problem Statement The posta wire-protocol spec (`code.bas.es/posta/spec`) changed twice in May 2026 and `posta-server` has not absorbed either change: - **§4.1 Canonical URL** + **§4.2 Display form** (spec commit `4a5cf80`) replace the old "lowercase scheme/host + drop default port" rule with a deterministic, total canonicalization algorithm. The wire fields `sender`, `recipient`, and actor-doc `url` MUST now be in canonical form, and §5.2.1 / §7.2 comparisons are byte-equality on the canonical string. 25 normative conformance vectors ship under `testdata/vectors/url-canonical/`. - **§9 Key management** (spec commit `7dbbf57`) collapses key rotation and revocation into editing the actor doc. The 30-day "retain window" concept and the `retiringSince` state are gone. Verifier-side actor-doc cache is now capped at the timestamp window (5 min, §5.3) — that 5 min is the propagation bound for both replay protection and key removal. Today the server: - Uses the spec library's loose `posta.NormalizeURL` (which does not implement §4.1) at five call sites — peer URLs into `contacts`, `messages`, the per-sender rate limiter, the daemon's host derivation, and the inbox seed. - Stores `peer_url` rows that the new §4.1 would either normalize differently or reject outright. Two contacts with `https://x/inbox` and `https://x/inbox/` are currently two rows; under §4.1 they are one identity. - Has its own parallel host normalizer in `internal/daemon/manifest.go IdentityHost` that diverges from §4.1 (no IDN handling, no trailing-slash strip, no userinfo/IP-literal/query/fragment rejection). - Still models the dropped retain-window in `internal/keys/`: a `RetiringSince` field on the `Key` struct, a `Current()` that skips "retiring" keys, and package/doc comments that describe a workflow the spec no longer defines. - Bypasses the §5.3-conformant actor-doc cache in one place (the contacts handler's bespoke `fetchActorName` does its own `http.Get` and `DecodeActorDoc`). The server is not currently conformant with the spec it claims to implement, and operators have no way to type a participant address in display form (`alice.example/inbox`) — every input surface demands a full `https://…` URL that's then run through a normalizer that won't even reject obviously-invalid inputs. ## Solution Absorb both spec changes end-to-end: - Every URL crossing into the server (CLI, REST inputs, manifest entries) is canonicalized once at the edge using the spec library's `Canonicalize()` (precondition issue in `posta/spec`). Internal storage and the wire fields hold canonical form only. The CLI and the contact-list UI accept and display §4.2 display form. - The daemon's per-identity host dispatch derives its lookup key from the canonical URL, not from a parallel normalizer. - Existing `peer_url` rows are re-canonicalized in a schema migration that handles the collision cases the strict form introduces. - The keys file format and the inbox's actor-doc construction shed every reference to the retain-window model. A key is in the file or it isn't; removing it stops new envelopes from being signed with it, and verifiers stop accepting the removed `keyId` within the 5-min cache cap. - Contacts and any other code that reads peer actor docs go through the inbox's shared `ActorCache`, inheriting §5.3 freshness, singleflight, and the URL-match check. ## User Stories 1. As an operator, I want to type `forge identity add --url alice.example/inbox` (display form), so that I do not have to remember whether the protocol wants `https://` or the trailing slash. 2. As an operator, I want my manifest URL `https://alice.example:443/inbox/` to be accepted and canonicalized to `https://alice.example/inbox`, so that small typing differences do not surface as configuration errors. 3. As an operator, I want the daemon to refuse to start with a clear error if a manifest URL contains userinfo, an IP literal, a query, or a fragment, so that I find out at boot instead of when a peer's first envelope is rejected by §7.2. 4. As a user adding a contact via the API, I want my submitted peer URL to be normalized to canonical form before it lands in the contacts table, so that I do not end up with two rows pointing at the same participant. 5. As a user, I want existing contacts that differed only by trailing slash or path-case-from-percent-encoding to be merged on upgrade, with the older row's history retained, so that I do not lose conversation threads. 6. As a user sending a message, I want the peer URL on `POST /api/v1/messages` to be canonicalized server-side, so that the `recipient` field on the wire is canonical regardless of how the client typed it. 7. As a user, I want the contact list to render display form (`alice.example`, `posta.no/u/arne`), so that the address book reads as identities rather than as URLs. 8. As a user, I want IDN hosts (`café.example`) to be accepted as input and stored as their A-label canonical (`xn--caf-dma.example`), so that two clients typing the U-label and the A-label end up at the same identity. 9. As a receiver, I want envelopes whose `recipient` is non-canonical to fail with `421 wrong-recipient` rather than be quietly accepted, so that misconfigured senders are caught at the protocol seam. 10. As a verifier, I want an actor doc whose `url` field is non-canonical to be rejected so that the cascade to `bad-signature` happens at §5.2.1 rather than being masked. 11. As an operator rotating a key, I want to add a new entry to `keys.json`, restart, and have the new key advertised in the actor doc, so that I do not need to know any retain-window or retiring-state vocabulary. 12. As an operator revoking a compromised key, I want to remove the entry from `keys.json` and trust that verifiers stop accepting it within five minutes, so that I have a real revocation path bounded by the timestamp window. 13. As a maintainer reading `internal/keys/`, I want the data model to match the current spec, so that I am not misled by `RetiringSince` and "retiring" comments that describe a workflow the spec deleted. 14. As a user adding a contact, I want the actor-doc fetch that resolves the contact's name to share the inbox's actor cache, so that repeated lookups within five minutes do not re-hit the network and a deleted key from a known contact stops being honored on the same schedule as the inbox. 15. As an operator, I want a single source of URL canonicalization, so that the daemon's host dispatch, the inbox's recipient check, the rate limiter's per-sender key, and the contacts table all agree on identity. 16. As a maintainer, I want the server's tests to exercise the spec's conformance vectors at the seams where URLs enter the system, so that any regression is caught locally rather than at protocol level. 17. As a user with a contact whose URL contains percent-encoded unreserved characters (e.g., `%7Ealice`), I want it normalized to `~alice` so that two clients exchanging that contact via different paths end up at one row. 18. As an operator running the migration, I want any merge of `peer_url` rows to retain the older row's id, fold message counts and read-watermarks deterministically, and log every merge so that I can audit what the upgrade did. 19. As a user, I want `POST /api/v1/contacts` to return `400` with a typed rejection category (`non-https-scheme`, `userinfo-present`, `ip-literal-host`, `malformed-host`, `malformed-port`, `query-present`, `fragment-present`, `malformed-path`) when my URL is non-canonical and unsalvageable, so that the client can show a precise error. 20. As a maintainer, I want the daemon's `IdentityHost` to be removed (or trivially derived from `Canonicalize`), so that there is no parallel set of URL rules in the codebase. ## Implementation Decisions **Precondition: spec library lands `Canonicalize()`** This PRD is gated on `posta/spec` shipping a `Canonicalize(input string) (canonical string, err error)` API in `pkg/posta`, with rejection categories surfaced as typed errors and a vectors-driven test suite over `testdata/vectors/url-canonical/`. The same library work also tightens `DecodeActorDoc` to reject actor docs whose `url` field is not canonical. That issue lives in the spec repo and is not in scope here, but every server module below assumes it. **Module A — `internal/keys` simplification** Drop the `RetiringSince` field from the `Key` struct. The `keys.json` schema becomes a flat list of `{id, publicKey, privateKey, createdAt}`. Strip the package-level "rotation" comment and the `Current() skips retiring` semantics; the remaining notion of "current" is "the signing key the runner is configured to use" — pick the most-recently-created entry, no special state. Fix the error string in `daemon/runner.go` that mentions "non-retiring." Update the inbox's `PublishedKeys` doc comment to drop "retiring keys still inside their retain window." No data migration on disk: nothing was writing the `retiringSince` field in production keys files. JSON unmarshalling tolerates unknown fields, so older files keep working; newer ones omit it. **Module B — daemon host derivation** Replace `daemon.IdentityHost` with: canonicalize the manifest URL via `posta.Canonicalize`, then derive `host[:port]` from the canonical string. The dispatch table in `daemon.Daemon` is keyed on the canonical-derived host. Manifest validation rejects entries whose URL is rejected by §4.1, with the spec's rejection category in the error message. **Module C — URL-input adapter (deep, new)** A new module (likely at `internal/posta` or as helpers on the API package, name TBD in implementation) that takes whatever the user typed and returns canonical URL or a typed rejection. Responsibilities: - Accept inputs with or without `https://` scheme; prepend if absent before calling `posta.Canonicalize`. - Map spec-library rejection categories to the strings listed in user story 19, and to HTTP 400 for REST inputs. - Provide a `DisplayForm(canonical) string` derived from §4.2 for response payloads and CLI output. This module is the single seam every input surface uses: `POST /api/v1/contacts`, `POST /api/v1/messages` (peer field), the search endpoint, the `identity add --url` CLI, and any future ingest path. The five existing `posta.NormalizeURL(...)` call sites and the rate limiter's normalization key all migrate to this module. **Module D — `peer_url` rewrite migration (new)** A new schema migration in `internal/store` that walks `messages` and `contacts` (and any other `peer_url`-bearing table) and re-canonicalizes each row. Collision policy: - If two rows now share the same canonical `peer_url`, keep the row with the older `id` (or older `created_at`, depending on table). - For `messages`: nothing to merge — both rows are independent envelopes, just with the new canonical form on the duplicate. - For `contacts`: fold the surviving row's `display_name` (keep the older row's), `pinned` (logical OR), `last_message_at` (max), and any read-watermark fields (max). Drop the loser row. - Log every merge at `slog.Info` with both original URLs, the merged canonical, and the kept/dropped ids, so the operator has an audit trail. - Migration runs in the existing transactional migration framework alongside the other versioned migrations in `internal/store`. **Module E — contacts via inbox.Cache** Delete `fetchActorName` in `internal/api/contacts.go`. Replace its single caller with a call through the inbox's shared `ActorCache` (already exposed via `Inbox.Cache()`). The cache's `Resolve(...)` API takes `(senderURL, keyID)`; for the "I just want the name" use case, add a thin helper (`FetchDoc(senderURL)` or similar) on the inbox/cache surface that returns the validated `*posta.ActorDoc`. This eliminates the bespoke HTTP path and inherits §5.3 freshness, singleflight collapsing, and the actor-doc-URL-match check. **Module F — inbox/actor-doc wiring** After the spec library tightens `DecodeActorDoc`, audit the inbox's actor-doc construction (`dynamicActor()` in `internal/inbox/inbox.go`) to ensure `opts.URL` is canonical at construction time. The natural place to do this is in the per-identity runner builder: canonicalize the manifest URL once, propagate the canonical string into both the inbox `URL` and any other consumer. **Out-of-scope cross-references** The PR will not introduce its own `Canonicalize` function in the server. If the spec library ships later than the server work needs to land, gate this PRD behind that work; do not shim. ## Testing Decisions **What makes a good test here.** Tests exercise external behavior at module seams, not internal SQL or struct layout. Vectors-driven tests are preferred for anything URL-shaped: the spec ships 25 conformance vectors and the server's URL-input adapter is the natural place to consume them. For the migration, the test contract is "what does the table look like after migrating from a representative pre-canonical fixture DB" — not the specific SQL. **Modules under test:** - **Module A — keys.** Lock in that the simplified `Key` struct unmarshals legacy `keys.json` files (with stray `retiringSince` fields) without error and ignores the field, that `Current()` returns the most-recently-created entry, and that `PublishedEntries()` returns every key in the file. Prior art: there is currently minimal coverage on `internal/keys/`; this is the moment to add a small focused test file. - **Module C — URL-input adapter.** Vectors-driven: every accept vector under `testdata/vectors/url-canonical/` produces the listed canonical output, every reject vector produces the listed rejection category, and display-form round-trips for accept vectors are bijective. Also test the "no scheme → prepend `https://`" path and a few HTTP-shaped tests that REST inputs return 400 + the rejection category in the response body. Prior art: `internal/daemon/manifest_test.go` already builds table-driven cases over URL inputs (`https://Arne.Posta.NO:8443` → `arne.posta.no:8443` style) and is the closest existing pattern. - **Module D — peer_url migration.** Build a representative pre-migration SQLite database in `:memory:` with rows that collide under §4.1 (trailing slash, percent-encoded unreserved, IDN U-label, case-only-distinct hosts that survive lowering). Run the migration. Assert: rows merged on the older id, counts/watermarks folded correctly, log lines emitted. Prior art: `internal/store/sqlite_test.go` already constructs `:memory:` stores and asserts post-state; same harness applies. - **Module E — contacts via inbox.Cache.** Behaviour test: two contact creations within the cache's positive TTL for the same peer URL trigger one upstream actor-doc GET (verified through a fake `ActorFetcher` counting calls). Prior art: `internal/api/api_test.go newRig` already wires a real `inbox.Cache` and intercepts SSE; the same fixture pattern applies, swapping the cache's fetcher for a counting fake. **Modules not getting their own tests (B, F):** - **B (daemon host derivation)** is exercised through Module C's vectors and the existing `manifest_test.go` host-extraction cases, which will be updated to the canonical-form output. - **F (inbox actor-doc wiring)** is exercised indirectly: if `opts.URL` is non-canonical at runner construction, the spec library's tightened `DecodeActorDoc` will reject the seeded doc — the existing inbox tests will fail loudly. No new test needed. ## Out of Scope - The spec-library work: implementing `Canonicalize()`, the rejection-category error type, and tightening `DecodeActorDoc`. That work belongs in `posta/spec` and is the precondition for this PRD. - Renaming `keys.json` or changing its on-disk path. The schema simplification stays backward-compatible by virtue of JSON unmarshalling tolerating unknown fields; no rewrite of existing files. - Multi-device key fleets (one key per device). Spec §9 now allows them; tooling to manage them (e.g., a `key add` CLI separate from `identity add`) is a follow-up. - Localized display rendering of U-labels in the CLI/web. The §4.2 display form ships as A-labels; rendering U-labels with anti-spoofing policy is a UI concern, not a protocol one, and is deferred. - Migrating the wire format on existing stored envelopes (the raw signed bytes in `messages` are immutable by construction — they were already canonical under the rules in force when signed, or they would not have verified). - Cache-Control / ETag emission on the actor-doc GET response. The current `Cache-Control: max-age=300` is correct against §5.3 and stays as-is. ## Further Notes - The conformance vectors at `posta/spec/testdata/vectors/url-canonical/` are normative for the spec library, but consuming them in the server's URL-input adapter test gives the server a regression net at zero ongoing cost — every spec-library update that adds vectors strengthens the server's coverage automatically. - Coordination with `posta/spec`: open a parallel issue there for `Canonicalize()` + tightened `DecodeActorDoc`. Land that first; the server PR can then go in without shims. - The `keys.json` simplification (Module A) is the only piece that is fully independent of the spec library; it can land on its own ahead of the canonicalization wave if useful for unblocking iOS/TUI work that touches the keys format. - The migration in Module D is the only operationally irreversible step. Worth a `--dry-run` mode on the migration that prints the merge plan without applying it.