Random entry access with the archive stream closed.

[vpenades]

Typically, the usual way of accessing the contents of an archive, both with System.IO.Compression and with SharpCompress is to open the archive stream, and use the stream to browse the contents of the archive.

This approach expect burst operations: Open -> heavy read/writes -> Close

Now I have the requirement of accessing specific files within many archives, randomly, over time.

Keeping the archives streams open is not practical because amongst other things, it requires resource management (disposing the streams)

So as a first approach, I built a framework on top of SharpCompress that does this:

• Open an archive
• read the TOC and gather all the entries in a new structure for each entry.
• Close the archive

Them when I need to access a file archive at random, I use the structure for the archive entry, which:

• opens the stream
• finds the entry
• opens the archive entry
• returns a stream that, when closed, it will also close the archive stream.

This works well except for one detail: every time I open an entry it has the overhead of reading the complete TOC of the archive or seek through the archive back and forth.

So my question is: Could it be possible to introduce a new reading API so file entries contain all the internal information so when opened, can go straight away to the location where the specific entry is located, and read it, without requiring to reprocess the whole archive?

I've attempted successfully to do it with ZIP archives and it works just fine, but I am aware that this would not be doable for all archive types

[adamhathcock]

There is ArchiveFactory.GetArchiveInformationAsync and overloads that give what you're looking for?

I'd have to know what format you're dealing with to get that behavior you're describing. I feel like you're mentioning Tar which is like that and I probably just need to remove TarArchive for compressed Tar files

[vpenades]

There is ArchiveFactory.GetArchiveInformationAsync and overloads that give what you're looking for?

I'd have to know what format you're dealing with to get that behavior you're describing. I feel like you're mentioning Tar which is like that and I probably just need to remove TarArchive for compressed Tar files

No, not TAR, mostly ZIP, RAR and 7Z

The idea would be to have something like this:

interface IEntryHandle : IEntry
{
    Stream OpenEntryStream();
}

Task<List<IEntryHandle>> ArchiveFactory.GetArchiveTableOfContentsAsync(string archivePath);

What this would do is to return a collection of IEntryHandle entries that could be used to open the specific entry within the archive.

The difference witjh the current APIs would be that GetArchiveTableOfContentsAsync would close the archive stream, and each IEntryHandle would reopen the entire archive stream when opening the OpenEntryStream.

[adamhathcock]

Doing that wouldn't solve the problem. 7Zip can contain a single stream of little files that will cause the behavior you mentioned. 7Zip Folders/Streams have to be decompressed inline as they contain multiple files.

The different formats do things in different ways to try to save space. Zip is the only format that is guaranteed to have a compressed stream single to each entry as well as a single dictionary that contains all entries.

Things aren't simple.

[vpenades]

Things aren't simple.

I'm aware, I know it's not feasible for all file formats, it's the same as with the formats that allow for random entry access vs the archives that require sequential access.

Certainly, this would be limited to archives that support random access.