API password auth: a non-latin-1 password can never authenticate

[Ro-28]

Problem

A password containing characters outside latin-1 can never authenticate over HTTP. The result is a permanent 401 with no way to succeed and no error explaining why. Pure-ASCII passwords are unaffected, which is why this has gone unnoticed.

ASGI carries raw header bytes and Starlette decodes header values as latin-1, so the credential string reaching the middleware is the latin-1 decoding of whatever bytes the client sent. api/auth.py:71-73 then re-encodes that string as UTF-8 before comparing it against the UTF-8 bytes of the configured password:

if not secrets.compare_digest(
    credentials.encode("utf-8"), self.password.encode("utf-8")
):

For pässwörd-中文 a UTF-8 client sends bytes that decode to mojibake, and no value the client can send will match.

Fix

api/auth.py:71 — recover the wire bytes instead of re-encoding the mojibake:

if not secrets.compare_digest(
    credentials.encode("latin-1"), self.password.encode("utf-8")
):

Stay in bytes rather than round-tripping back to str: a .decode("utf-8") on attacker-supplied garbage would raise and turn a 401 into a 500. A string decoded from latin-1 is always latin-1-encodable, so no try/except is needed.

Behaviour change worth a CHANGELOG line: today a client sending the latin-1 bytes of a latin-1-representable password (pässwörd) authenticates, because Starlette decodes it back to the original string. That path stops working, in exchange for the UTF-8 path that modern clients actually use. Hard to hit in practice — httpx (and therefore TestClient) rejects non-ASCII header values, and curl/browsers emit UTF-8.

Tests

There is no coverage for api/auth.py today. Add tests/test_auth.py with TestPasswordAuthMiddleware, TestNoPasswordConfigured and TestNonAsciiPassword. The non-ASCII case needs a raw-ASGI helper that drives the app directly, because TestClient cannot send non-ASCII header bytes.

Acceptance criteria

• A deployment with OPEN_NOTEBOOK_PASSWORD=pässwörd-中文 authenticates when the client sends the password UTF-8-encoded.
• ASCII passwords keep working unchanged.
• Invalid UTF-8 in the Authorization header returns 401, not 500.
• tests/test_auth.py covers the three classes above.
• CHANGELOG entry under Unreleased → Fixed, noting the latin-1 client behaviour change.

Already fixed — not in scope

The other two findings from the original report were already resolved before this issue was filed:

• Constant-time comparison landed in fix: use constant-time comparison for the API password #1003 (secrets.compare_digest, which is the same function as hmac.compare_digest).
• The middleware docstring was corrected in chore: remove dead auth helper and fix stale default-password docs #1026, which also removed check_api_password as dead code — it no longer exists, so nothing should be changed there.

PR #1291 attempted all three against a stale base and was closed by its author without a rebase.