"Resume" for remote pmtiles extract

[orblivion]

Hi,

I'm curious how feasible it would be to add a "resume" feature to remote extracts?

Supposing I'm making a large extract from a larger remote file. The connection cuts, I re-run the exact command again. It would be very nice in some cases to continue more or less where it left off. Perhaps this requires an extra "progress" file that keeps track of the last thing written, and makes sure that I'm running with the same polygon or bbox as before. Maybe it also makes sure the remote file hasn't changed between calls (if there are hashes or something). Or maybe such "progress" data is already inherent in the incomplete local extracted copy?

If it's not something you want to add into mainline, or not something you want to spend time on yourself but may consider merging, I'd be curious what you'd guess the onramp might be for a Go developer to add such a feature?

Thanks!

[bdon]

I don't think this is worth the complexity of implementation

Maybe it also makes sure the remote file hasn't changed between calls (if there are hashes or something).

This would require saving the ETag somewhere, like to a partial download sidecar file

The sidecar would also need to save off the input arguments (polygon, bbox, etc)

You'd need to know how much of the file is downloaded, the simplest way to do this is to use the number of bytes of the partially downloaded file, but the extract is by default multithreaded so we can't trust that the downloaded parts are written in sequential order