Skip to content

fix: sealing pipeline commit batch robustness and allocation validation #13631

Description

@marco-storswift

Summary

Several bugs in the Lotus sealing pipeline cause sectors to get stuck in perpetual retry loops, block callers indefinitely, and perform non-deterministic allocation validation. This issue tracks fixes across commit_batch.go, input.go, and precommit_policy.go.

Issues Fixed

1. Sectors stuck in infinite retry loop (commit_batch.go)

Sectors experiencing non-retryable failures (e.g., missing precommit info, collateral calculation errors) are retried indefinitely with no eviction mechanism. This causes resources to be wasted on sectors that will never succeed.

Fix: Track consecutive non-retryable failures per sector and evict after 5 consecutive failures. Evicted sectors receive a failure result delivered to all waiters, and their state is cleaned up.

2. AddCommit callers blocked forever when sector is silently dropped (commit_batch.go)

If a sector is queued for batch processing but processBatchV2 drops it without producing a result (e.g., when infos is empty and res.Sectors is empty), the caller in AddCommit blocks indefinitely waiting for a result that never arrives.

Fix: Added a defensive fallback after processBatchV2 that iterates over all sectors in the batch and ensures every sector receives a result. Sectors missing from the response get an explicit failure result. Also fixed processBatchV2 to return res when infos is empty but res.Sectors has failures.

3. Non-deterministic allocation TermMin/TermMax check (commit_batch.go)

The allocationCheck method uses live ts.Height() as the reference point for TermMin/TermMax validation. Since chain height advances between retries, an allocation can flip from valid to invalid (or vice versa) non-deterministically, causing inconsistent behavior.

Fix: Changed to use PreCommitEpoch as a fixed reference point, making validation deterministic.

4. Missing allocation term validation (input.go)

getClaimTerms did not validate that an allocation's TermMax is sufficient for the sector's minimum lifetime requirement. This could allow sectors to be created with allocations that expire before the sector's required minimum expiration.

Fix: Added minAllocationTermForSealProof which computes the minimum allocation term based on the seal proof's max prove commit duration, precommit randomness lookback, and minimum sector expiration. getClaimTerms now rejects allocations with TermMax below this minimum.

5. Missing allocation validation in precommit policy (precommit_policy.go)

BasicPreCommitPolicy.Expiration did not validate piece CID and size against the on-chain allocation, and did not clamp the deal end epoch to the allocation's TermMax. This could lead to sectors being precommitted with expiration beyond what the allocation allows.

Fix: Added allocation lookup and validation:

  • Verify piece CID and size match the on-chain allocation
  • Clamp endEpoch to TermMax if the deal end epoch exceeds it

Affected Files

  • storage/pipeline/commit_batch.go
  • storage/pipeline/input.go
  • storage/pipeline/precommit_policy.go

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    🔎 Awaiting Review

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions