Skip to content

feat: brute-force telemetry (attempt_status, /attempts) and security hardening - #8

Open
ethicnology wants to merge 31 commits into
mainfrom
feat/bruteforce-telemetry
Open

feat: brute-force telemetry (attempt_status, /attempts) and security hardening#8
ethicnology wants to merge 31 commits into
mainfrom
feat/bruteforce-telemetry

Conversation

@ethicnology

@ethicnology ethicnology commented Aug 4, 2026

Copy link
Copy Markdown
Member

A Recoverbull user has no way to know that their backup is being attacked.
An attacker holding a copy of a Backup File can probe the Key Server
(3 password guesses per cooldown window), or grief the victim by keeping
their identifier permanently rate-limited — and the legitimate user only
finds out when their recovery fails. Failed attempts were previously
invisible to everyone except the server operator.

This PR adds advisory brute-force telemetry to the Key Server and hardens
the service against the findings of three audit rounds. Telemetry is
advisory by design: the server cannot distinguish an attacker from the
user or another of the user's devices, and a compromised server can
fabricate or suppress counters. Clients must warn, never act automatically.

What this PR changes

Telemetry contract

attempt_status on successful /fetch and /trash — exact attempt
counters for the identifier's current cooldown window:

{
  "attempt_status": {
    "total_attempts": 3,
    "failed_attempts": 1,
    "remaining_attempts": 0,
    "window_started_at": "2026-08-05T12:17:41Z",
    "previous_attempt_at": "2026-08-05T14:37:22Z",
    "resets_at": "2026-08-06T15:04:13Z"
  }
}
  • total_attempts counts all lookups, including database hits: a hit
    does not prove ownership, because a public /store caller can plant a
    matching row. A client warns when the total exceeds the user's own
    operations.
  • A successful lookup never resets the counters (an earlier version of
    this PR reset them on success; that was reversed — a planted row would
    otherwise erase the security signal).
    GET /attempts — public telemetry snapshot of the identifiers
    currently rate-limited, rebuilt at most once per minute and served as
    immutable gzip bytes with a strong ETag (304 on If-None-Match,
    Cache-Control: public, max-age=):
{
  "version": 1,
  "collection_started_at": "2026-08-05T09:00:00Z",
  "entries": [
    {
      "id_hash": "7a06e6b2…",
      "total_attempts": 3,
      "failed_attempts": 1,
      "window_started_at": "2026-08-05T12:00:00Z",
      "last_attempt_at": "2026-08-05T14:00:00Z"
    }
  ]
}
  • id_hash is SHA-256 over the raw identifier bytes (not the hex
    string): a client recognizes its own identifier by hashing it locally;
    nobody can recover a raw identifier from the list (pre-image resistance),
    which keeps the list useless for griefing or targeted lockout.
  • Entries live in the same in-memory map as the rate-limiter and expire
    with it (cooldown or reboot): nothing is persisted, in line with the
    whitepaper's daily-wipe privacy model.
  • Snapshot timestamps are hour-truncated; direct responses carry exact
    timestamps. Precision follows the knowledge gradient (documented in the
    README, "Timestamp precision by response").
    GET /info gains attempts_collection_started_at (cheap wipe check
    during the existing connection check) and max_attempt_identifiers (map
    capacity, so clients can warn when the service is under pressure). It never
    exposes a live identifier count.
    Hardening (three audit rounds)
    Two earlier external audit rounds (13 findings) are absorbed in this branch,
    each fix proven by a characterization test that was verified to fail without
    it: idempotent /store closing the authentication_key oracle, atomic
    check-and-increment, cooldown expiry + sweeper, spawn_blocking for diesel,
    global token buckets on unauthenticated writes and lookups, structured
    logging without identifiers, CORS removal, hex canonicalization, request
    timeout, dependency advisories (bytes/time/idna/tracing-subscriber), CI with
    cargo test + cargo audit.
    Third round (this push):
  • e76992f fix: origin-aware warrant canary live reload and startup
    capacity bounds — /info re-reads the canary from the dotenv file on
    each request: edits are served immediately, a removed CANARY line
    serves an empty canary (the compromise signal, never masked), an
    unreadable file falls back to the startup value (no false alarm), an
    environment-provided canary stays authoritative. Also bounds
    RATE_LIMIT_MAX_IDENTIFIERS to 1, 10M and DATABASE_MAX_CONCURRENCY
    to 1, 1024 at startup.
  • 317b19a fix: harden request handling — /store checks length
    before base64 decode; /attempts honors weak If-None-Match validators
    per RFC 9110.
  • 7e74691 chore: trim tokio features — fs, process, signal and
    parking_lot no longer reach the release binary (Cargo.lock −59 lines).
  • 79d63ff ci: re-audit the pinned Cargo.lock daily — new RustSec
    advisories no longer wait for a push.
  • 73805e3 docs — canary signaling procedure, timestamp precision by
    response, secret_id hashing (hex strings, unlike id_hash over raw
    bytes), nginx proxy_cache vs If-None-Match caveat, torrc PoW defense.
    What this PR deliberately does not change
  • No raw identifiers are ever exposed (a test asserts the raw value never
    appears in the /attempts body).
  • No new persistence, no schema change, no new stored data.
  • Rate-limit parameters, authentication flow, response formats (additive
    only).
  • The recovery-lockout design tension (an attacker holding the Backup File
    can keep the victim's identifier locked out) is a protocol property, not
    a code bug: it is documented in the README with the detection path this
    PR ships and the real mitigations (redundancy, exported Backup Key,
    social recovery; backoff / proof-of-work / multi-server for a v2).
    Status
  • Full suite: 51/51 (cargo test -- --test-threads=1), including
    characterization tests for every audit finding.
  • cargo audit: no vulnerability (one non-applicable dev-only unsound
    warning, rand via axum-test).
  • cargo clippy --all-targets: clean.
  • Client integration: recoverbull-client-dart and bull-3 PRs prepared
    separately.
    Suggested reading order
  1. 17a52ca attempt_status on successful fetch/trash (counter semantics).
  2. e6c7326 the /attempts snapshot (public contract, caching, gzip).
  3. a6326af /info metadata.
  4. The hardening commits in message order; each fix ships with the test
    that proves it.

@ethicnology ethicnology self-assigned this Aug 4, 2026
@ethicnology
ethicnology requested review from BullishNode and i5hi August 4, 2026 00:25
The warrant canary workflow described in the whitepaper requires the
operator to update or remove the canary without restarting the server,
but env::var never sees file edits: dotenvy loads the file only at
startup. /info now re-reads the canary from the dotenv file on each
request with origin-aware semantics:

- file-provided canary (the common case): edits are served immediately,
  a removed CANARY line serves an empty canary (the compromise signal,
  never masked by a fallback), a missing/unreadable file falls back to
  the startup value (an ops error must not raise a false alarm);
- environment-provided canary: authoritative, signaling requires a
  restart with a changed value. The dotenv path returned by dotenv() is
  kept as the live source, so a file in a parent directory works too.

Also bound RATE_LIMIT_MAX_IDENTIFIERS to [1, 10000000] and
DATABASE_MAX_CONCURRENCY to [1, 1024] at startup: a zero or absurdly
large value silently disabled the memory/concurrency protections.
/store: check the encrypted_secret length before the base64 decode, so
oversized input is rejected by the cheap check without paying for a
full decode of a body that is rejected anyway. Both rejections stay
400 and reveal nothing about other users' data.

/attempts: RFC 9110 evaluates If-None-Match with the weak comparison
function, so a weak validator W/"…" must match our strong ETag.
Conditional requests remain an optimization, not a security boundary.

Each change ships with a test that pins the new behavior.
tokio "full" compiled in process spawning, signal handling, filesystem
APIs and parking_lot, none of which the server calls. The feature set
is now the minimal one the code uses (macros, rt-multi-thread, net,
sync, time, io-util for tests): fs, process, signal and parking_lot no
longer reach the release binary, and Cargo.lock drops 59 lines.

Also apply clippy's manual_is_multiple_of suggestion in is_base64.
New RustSec advisories are published independently of our pushes, so an
advisory against an unchanged dependency tree used to go unnoticed
until the next push. The audit job now also runs on a daily schedule.
- Warrant canary: the live-reload semantics and the exact signaling
  procedure (edit, remove, or environment-provided), plus chmod 600
  for the dotenv file.
- Timestamp precision by response: exact for identifier-holders
  (attempt_status, 429 requested_at), hour-truncated for everyone
  (public snapshot) — the precision follows the knowledge gradient.
- secret_id is SHA-256 over the concatenated lowercase hex strings,
  unlike the /attempts id_hash over raw bytes: client implementers
  must not mix the two.
- nginx: proxy_cache answers conditional requests with the cached 200
  body, so the bodyless 304 only benefits direct clients — an egress
  tradeoff, bounded by limit_conn x limit_rate.
- torrc: enable HiddenServicePoWDefensesEnabled (Tor 0.4.8+).
- Startup bounds for RATE_LIMIT_MAX_IDENTIFIERS and
  DATABASE_MAX_CONCURRENCY.
@ethicnology ethicnology changed the title feat: monitor brute-force attempts feat: brute-force telemetry (attempt_status, /attempts) and security hardening Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant