Add resource limits for archive extensions and sparse maps - #118
Conversation
|
Please describe your motivation? |
|
Thanks for asking. I have added a Motivation section and opened #119 to separate the problem statement from the implementation. This originated in RustFS's Snowball ingestion path, which streams TAR archives supplied by untrusted network clients. RustFS can enforce logical member limits after The proposed limits are opt-in and remain unlimited by default. The downstream integration is rustfs/rustfs#6942, and the full motivation and compatibility goals are in #119. |
|
Hi @zanieb , could you please review this PR when you have a chance? If everything looks good, I'd appreciate it if you could merge it. Thanks! |
Motivation
Server-side consumers may parse TAR archives supplied by untrusted clients. They can bound logical entries after
Archive::entries()yields them, but tokio-tar consumes GNU long-name and long-link payloads, PAX extensions, and GNU sparse continuation data before the corresponding logical entry is exposed. At that point an application-level check is too late to prevent extension buffering, sparse-map growth, or excessive continuation reads. Counting logical entries also does not bound the physical metadata records consumed by the parser.This was encountered while hardening RustFS's MinIO-compatible Snowball ingestion path in rustfs/rustfs#6942. The limits belong at the parser boundary so a caller can reject an archive before the associated allocation or out-of-budget read. They remain opt-in and default to unlimited, preserving behavior for existing callers. The cancellation-safety and stream-fusing changes ensure that an error or cancelled
next()future cannot leave extension parsing in a partially advanced state.The problem statement and compatibility goals are tracked in #119.
Summary
ArchiveBuilderPendingand cancellednext()futuresAll new limits default to unlimited, preserving existing behavior unless a caller opts in.
Testing
cargo fmt --all -- --checkUV_NO_CONFIG=1 uvx --from 'astral-dev-toolchain-cargo-hack>=0.6.45' cargo-hack hack check --all --ignore-private --each-feature --no-dev-depsUV_NO_CONFIG=1 uvx --from 'astral-dev-toolchain-cargo-shear>=1.13.4' cargo-shear shearcargo check --all --all-targets --all-featurescargo testcargo test --no-default-featurescargo clippy --workspace --all-targets --all-features --locked -- -D warningscargo +1.83.0 testCloses #119.