Problem
Applications that stream untrusted TAR archives can limit the size and number of logical entries after Archive::entries() yields them. Some resource use occurs before that boundary, however:
- GNU long-name and long-link payloads and local or global PAX extensions are consumed before the following logical entry is returned.
- GNU sparse maps and continuation blocks are parsed while constructing a logical sparse entry.
- Physical metadata entries do not have a one-to-one relationship with the logical entries visible to the caller.
Consequently, a caller cannot reliably prevent a declared extension payload from being buffered, a sparse map from growing, or a long continuation chain from being read by validating only the yielded entry. This is particularly relevant for compressed or streaming inputs, where the request size is not a useful bound on parser work or decoded metadata.
Motivation
RustFS accepts TAR streams from untrusted network clients through its MinIO-compatible Snowball extraction path. It needs to preserve compatibility with valid archives while applying explicit per-request memory and parser-work budgets. The downstream integration is tracked in rustfs/rustfs#6942.
The same boundary applies to other server-side consumers of tokio-tar: the parser is the earliest layer that can reject extension metadata before allocation or an out-of-budget continuation read.
Desired behavior
ArchiveBuilder should offer opt-in limits for:
- each extension payload and cumulative extension payload bytes;
- physical archive entry headers;
- GNU sparse map entries;
- GNU sparse continuation blocks.
Limits should be enforced before the corresponding buffer or sparse I/O map grows and before an out-of-budget continuation block is read. If asynchronous parsing returns Pending, is cancelled, or encounters a structural error, subsequent polling should not resume from partially lost parser state or continue beyond the invalid archive position.
Compatibility and non-goals
- Existing behavior should remain unchanged unless a caller enables a limit.
- This should not impose universal default limits or reject otherwise valid large archives.
- Callers remain responsible for application-specific limits on logical entry sizes and total extracted data.
Proposed implementation
#118
Problem
Applications that stream untrusted TAR archives can limit the size and number of logical entries after
Archive::entries()yields them. Some resource use occurs before that boundary, however:Consequently, a caller cannot reliably prevent a declared extension payload from being buffered, a sparse map from growing, or a long continuation chain from being read by validating only the yielded entry. This is particularly relevant for compressed or streaming inputs, where the request size is not a useful bound on parser work or decoded metadata.
Motivation
RustFS accepts TAR streams from untrusted network clients through its MinIO-compatible Snowball extraction path. It needs to preserve compatibility with valid archives while applying explicit per-request memory and parser-work budgets. The downstream integration is tracked in rustfs/rustfs#6942.
The same boundary applies to other server-side consumers of
tokio-tar: the parser is the earliest layer that can reject extension metadata before allocation or an out-of-budget continuation read.Desired behavior
ArchiveBuildershould offer opt-in limits for:Limits should be enforced before the corresponding buffer or sparse I/O map grows and before an out-of-budget continuation block is read. If asynchronous parsing returns
Pending, is cancelled, or encounters a structural error, subsequent polling should not resume from partially lost parser state or continue beyond the invalid archive position.Compatibility and non-goals
Proposed implementation
#118