Skip to content

multipart-staging-retire-pending #717

Description

@eduralph

Child slice of #692, split during Plan.

  • Slug: multipart-staging-retire-pending
  • Defect / goal: the staging and retirement records — the two whose identity lives
    partly in the key — do not exist, and PendingEntry cannot yet carry ownership.
    This child lands OwnedEntry/StagedPlacement (sidx:, disjoint-staging rule
    0016:475-491), PartNumberSet and RetirePayload (retire:*, token grammar
    0016:437-453), the two key-taking decode APIs (decode_owned_entry(key, bytes),
    decode_retire_obligation(key, bytes) — a decode that cannot see the key cannot
    validate against it, which is exactly how v2's shape failed its review), their
    encode_record/decode_record arms, and the ONE metadata.rs allowance: PendingEntry
    (metadata.rs:1528) gains owner: Option<multipart::UploadId> and
    staged: Option<multipart::StagedPlacement> (#[serde(default, skip_serializing_if = "Option::is_none")]), drops Copy (forced — UploadId is a String newtype), and
    gains the torn-shape rejection in a manual Deserialize. Salvage the corresponding
    types and the metadata.rs hunk from results/issue_692/iteration-v2/patch.diff,
    fixing the recorded defects.
  • Success criterion: every type this child lands round-trips, and each hand-authored
    torn value below is rejected with a typed error (ADR-0045) — one named negation per leg
    in build-notes.md:
    (1b) decode_owned_entry takes the sidx: key and rejects a payload whose owner
    differs from the key's upload id (v2 review, multipart.rs:1555);
    (1d) decode_retire_obligation takes the retire: key and rejects a payload whose
    mode or generation identity disagrees with the key's token — generation-scoped payload
    under a session token and vice versa are both errors (v2 review,
    multipart.rs:1789/1800; the archived test at multipart_records.rs:1108 affirmed
    this case — it must now reject);
    (1h) the retire session-token arm honours the token's optional :<part>:<attempt>
    suffix (0016:437-453): a whole-session Session/Parts obligation under a per-part
    token, and a per-part Chunks obligation under a session-wide token, are both
    rejected (batch-review multipart.rs:2024);
    (1i, EcScheme half) StagedPlacement's EcScheme is rejected unless
    erasure::supported(k, m) (erasure.rs:120 — the Validate erasure coding schemes at lower-layer read and reconstruct boundaries #285 precedent: untrusted stored
    geometry like ReedSolomon { k: 0, m: 1 } is a typed error, not a panic);
    (1e) a PendingEntry with exactly one of owner/staged is torn and rejected at
    decode under both a pending: and a sidx: reading (v2 review,
    metadata.rs:1537/1541) — both-absent (legacy) and both-present (owned) are the only
    valid shapes;
    (leg 2) decode→encode is the identity on a legacy value: a legacy pending: value
    with neither new field re-encodes byte-identically — the identity every
    require(key, encode(prior)) CAS in metadata.rs depends on (metadata.rs:1368-1391,
    ADR-0047:38-50; 0016:512-524);
    (leg 3 corollary, binding) a sidx: value whose staged placement length does not
    match its scheme's fragment count decodes — placement length is the standing
    contextual-check example (AGENTS.md:146-149, ADR-0045, 0016:513-527), quarantined
    by GC, not rejected; leg 1i validates the scheme's geometry, never the placement's
    length;
    (leg 4, docs currency — SETTLED YES) PendingEntry gains two persisted fields, so
    this child adds the short paragraph to
    docs/design/architecture/05-building-block-view.md § "The metadata model", mirroring
    the ADR-0047 optional-inode-fields paragraph at :186-192 (AGENTS.md:154-158:
    a merge requirement, not a follow-up).
  • Falsifiability: RED is criterion-ABSENCE — born-at-tier. C4-verify classifies
    ADDED_TEST crates/core/tests/multipart_staging_retire.rs + CRATEs
    core/custodian/dst/metadata-redb/server; the GREEN leg is cargo test -p wyrd-core --test multipart_staging_retire; the RED leg reverts production and the test fails to
    compileUNVERIFIABLE (exit 77), EXPECTED and PRE-DECLARED as a §6 item.
    Demonstrated red Do MUST capture instead (binding): six named negations, one per
    binding rejection leg (1b, 1d, 1h-session-under-part, 1h-part-under-session, 1i-EcScheme,
    1e) — drop that single check, run the test, paste the failing output into
    build-notes.md, revert. Plus one for leg 2: remove ONE skip_serializing_if attribute
    and show the byte-identity leg fail. The leg-3 corollary is negated the other way: make
    the length-mismatched placement reject and show the assert-it-decodes leg fail. A leg
    green under its own negation is not load-bearing and must be rewritten.
  • Invariant to restore: C-1 (docs/principles.md:109, §6 row at :137;
    0016:2802-2813), over this child's category: a stored record's fields may not
    disagree with each other OR WITH THE KEY THAT NAMES THEM, and the disagreement must
    surface as an error, never as a value
    (ADR-0045). An owned entry attributed to the
    wrong session is staged data renewed or reclaimed under the wrong identity. A retirement
    payload under the wrong token — or the wrong scope of token — reclaims one generation's
    data while clearing another's obligation. Untrusted EcScheme geometry that decodes is
    the Validate erasure coding schemes at lower-layer read and reconstruct boundaries #285 panic class made durable. A torn PendingEntry turns a structural invariant
    into a convention. And a PendingEntry that does not re-encode byte-identically turns
    every existing pending: lease renewal into a permanent Conflict.
  • Repo + branch target: getwyrd/wyrd @ main
  • Surfaces: data
  • Reproduction: n/a — new functionality; builds on child-2's merged result. The
    one live-path touch is PendingEntry, and leg 2 proves the live path (every existing
    pending: CAS) is byte-identical — the extension is inert until core: UploadPart staging — slots, sidx/psum, drain fence (636.4) #657 writes the first
    sidx: record.
  • Scope (one logical fix) / out of scope: extend crates/core/src/multipart.rs with
    the staging/retirement types, both key-taking decoders, and their dispatch arms; the
    ONE PendingEntry hunk in crates/core/src/metadata.rs described under Goal — nothing
    else in that file changes; the explicitly ALLOWED mechanical ripple in the 8 files
    that construct or copy PendingEntry (crates/core/src/write.rs,
    crates/core/tests/mutation_regressions.rs,
    crates/custodian/tests/{gc,restore_reconcile,segmented_map_consumers}.rs,
    crates/dst/tests/custodian.rs, crates/metadata-redb/tests/conformance.rs,
    crates/server/tests/custodian_gc.rs) — owner: None, staged: None initializer lines
    / clone-instead-of-copy fixes only, ≤ 8 changed lines per file, no logic change, no
    new function
    ; a ninth ripple file or a non-mechanical hunk means the seam is wrong:
    STOP and hand back. Plus the ONE docs paragraph of leg 4. Keep every hunk in the shared
    files as small as briefed — a wider hunk is a needless rebase surface for ceiling-refused-placement-writes-do-not-certify #710/repoint-chunk-segmented-placement-moves #711
    whichever folds first. Budget ≈ 900 added lines / 12 files. / out of scope: custodian
    source code (crates/custodian/src/ untouched — only its tests' initializers);
    every docs/design/ file except the one 05-building-block-view.md paragraph — ADRs,
    proposals and specs untouched (INTEGRATION §2 immutability); the outcome enums, answer
    table, digests, sha2 (Brief — multipart verb × state answer table + digests (654 split 3/3) #693 — no Cargo.toml/Cargo.lock change); knob values
    (core: multipart knob constants + derivations (636.2) #655); store round trips (core: multipart admission + Create/Abort (636.3) #656core,custodian: retirement routing + drain + terminal delete with production dispatch (636.6) #659); reaper/windows (s3 multipart: reap discontinued uploads (abandoned-session detection + staged-part reclamation) #625).
  • Budget: ≤ 900 added semantic lines (module extension ≈ 400, metadata.rs ≈ 45,
    test ≈ 400, ripple ≈ 40 mechanical, docs ≈ 15) across exactly 12 files — 4
    substantive (multipart.rs, metadata.rs, the new test, the one docs paragraph) and 8
    mechanical, all named above.
  • External dependencies: typos, docs-renderer, cargo-deny, cargo-machete, cargo-mutants — all registered doctor ids; docs-renderer is load-bearing HERE (leg 4 edits a rendered architecture doc), the rest warn-skip locally while CI enforces them (INTEGRATION §3). Nothing else beyond the base Rust toolchain: pure functions, no runtime, no Docker, no new crate.
  • Test file: crates/core/tests/multipart_staging_retire.rs — a NEW file, not
    optional (C4-verify's added-*/tests/*.rs discriminator). The key-taking legs, the
    byte-identity leg and the docs-adjacent legs live here; co-located unit tests may ship in
    addition.
  • Verification posture: declared — born-at-tier (posture (a)), as its siblings: the
    UNVERIFIABLE RED is PRE-DECLARED so C2/C4 land as a known sign-off item rather than a
    surprise NEEDS-HUMAN. Everything built is exercised at Check under the named test +
    gating C4-ci, and the eight negation demonstrations in build-notes.md replace the
    flippable red.
  • Production reach: the ONE live-path touch in this whole 3-child chain. PendingEntry
    is on the existing pending: write/renew path, so (a) what honours the seam now is the
    legacy shape itself — both new fields absent — and leg 2 proves that path re-encodes
    byte-identically, i.e. the extension is genuinely inert; (b) the production wiring
    that writes a sidx: record with owner/staged set lands in core: UploadPart staging — slots, sidx/psum, drain fence (636.4) #657, which needs the
    store round trips (core: multipart admission + Create/Abort (636.3) #656core,custodian: retirement routing + drain + terminal delete with production dispatch (636.6) #659) first; (c) the torn-shape rejection (leg 1e) is exercised
    load-bearingly by hand-authored values in the named test, not by dead scaffolding.
  • Citations expected: cite path:line on the merged base for every change. Sources Do
    MUST open: 0016:475-491 (the sidx: disjoint-staging rule), 0016:437-453 (the
    retirement-token grammar leg 1h enforces), 0016:512-527 (the skip_serializing_if
    identity argument and the placement-length contextual boundary), ADR-0045, ADR-0047:38-50.
    Peer callsites Do MAY open: crates/core/src/metadata.rs:1368-1391 (the
    identity-preserving optional-field precedent the PendingEntry hunk must mirror);
    crates/core/src/erasure.rs:120 (supported(k, m), leg 1i's predicate and the Validate erasure coding schemes at lower-layer read and reconstruct boundaries #285
    precedent); docs/design/architecture/05-building-block-view.md:186-192 (the ADR-0047
    paragraph leg 4 mirrors — match its voice and length, do not restate the proposal).
    Salvage: results/issue_692/iteration-v2/patch.diff — take the staging/retirement
    types and its metadata.rs hunk, then FIX the recorded defects (the reviews found the
    decoders unable to SEE the key at multipart.rs:1555/1789/1800, the token suffix ignored
    at :2024, and StagedPlacement deriving unchecked EcScheme at :1657) rather than
    re-shipping the reviewed shape.
  • Prior-art check (triage cycles): verified at Plan against 9dbcd72: no
    OwnedEntry, StagedPlacement, PartNumberSet or RetirePayload exists on
    origin/main; PendingEntry (metadata.rs:1528) carries only lease_expiry_millis;
    git -C ../wyrd log origin/main -- crates/core/src/metadata.rs shows no multipart-related
    commit. Open PRs: none touching these paths today, but ceiling-refused-placement-writes-do-not-certify #710 and repoint-chunk-segmented-placement-moves #711 are in flight over
    core/src/metadata.rs
    — hence the conflict declaration below. Closed/rejected: core: multipart record family + state machine as pure functions (636.1) #654's
    two archived attempts and Brief — multipart record family + validating decoders (654 split 2/3) #692's own two — the batch review's token-scope blocker
    (multipart.rs:2024) and the reviewer's C5/T2 findings are this child's binding legs
    1h/1i, not suggestions.
  • Difficulty: medium
  • Depends on: child-2
  • Ordering note: Wave 2 — terminal. Depends on: child-2 is a genuine build-on
    (this child's StagedPlacement decode reuses the validated-nested-type pattern child-2
    establishes for ChunkRef, and its records dispatch through child-1's envelope), and it
    also wave-serialises the shared multipart.rs. This child alone carries the chain's
    external conflicts.
    core,custodian: repoint_chunk — ceiling-safe placement moves in a segmented record (635.4c) #682 was SPLIT on 2026-08-08 into ceiling-refused-placement-writes-do-not-certify #710 (shares
    core/src/metadata.rs — its MAX_VALUE_BYTES enforcement vs this child's PendingEntry
    region) and repoint-chunk-segmented-placement-moves #711 (shares BOTH core/src/metadata.rs, its repoint_chunk primitive,
    AND dst/tests/custodian.rs, its substantive edits vs this child's mechanical
    initializer lines). Because the proposal's ordering fields may only name sibling labels,
    Conflicts with: 710, 711 is added to THIS child's materialised brief.md at split
    acceptance — it must never share a wave with either. Brief — multipart verb × state answer table + digests (654 split 3/3) #693 (Depends on: 692) and core: multipart knob constants + derivations (636.2) #655
    (blocked by Brief — multipart verb × state answer table + digests (654 split 3/3) #693) are repointed at THIS child at the same moment. Cite by symbol, not
    by line number
    , in metadata.rs: the base will have advanced under ceiling-refused-placement-writes-do-not-certify #710/repoint-chunk-segmented-placement-moves #711.
  • Disposition hint: new-feature

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions