Does buildah support resumable pulling/pushing layer?

[justadogistaken]

Description
There is resumable pulling/pushing ability supported in registry api v2. With Range item added in header to achieve resumable pulling/pushing. I wonder if buildah has built-in support for pulling/pushing layers?

[flouthoc]

Hi @justadogistaken , Thanks for creating the issue.

I don't think buildah supports that as of now. But i also think there are some duplicate issues for this on c/image or podman. @vrothberg @mtrmac Could confirm this better since similar issue was discussed recently.

[justadogistaken]

@flouthoc I checked the source code again, and I found there is a function "GetBlobAt" that buildah depends on from "".
This function supports resumable pulling/pushing. But I'm not sure if buildah leverages this function.
func (f *zstdFetcher) GetBlobAt(chunks []chunked.ImageSourceChunk) (chan io.ReadCloser, chan error, error)

[mtrmac]

GetBlobAt is not for “resumable” pulling, that’s specifically for the zstd-chunked format (pulling only the changed files, essentially.).

Pulls are only retried at the image granularity (but any correctly-pulled layer should typically be committed to storage, so on retries it would not be pulled again). Some support for per-layer retries is discussed, but unlikely to make a lot of difference, because correctly-pulled layers should already be reused. Retries within a layer, using range requests and the like, are not implemented (and there is an issue specifically tracking a single kind of retry of that kind).

Pushes are also retried at the image granularity, with a similar situation about per-layer retries. Within-layer retries are not implemented, and not very likely to be, because the data is streamed to minimize on-disk and memory usage, and once it is sent to the registry, it is “gone” and the client can’t go back and re-send it (the compression implementations have no way to roll back and re-create data from a past stream state).

[justadogistaken]

@mtrmac Thanks for your response. As for my knowledge, none of the "cris" have "resumable operation" implemented. Could you share me some info if there it is?

[justadogistaken]

@mtrmac I found that containerD may implement the "resumable operation".

[mtrmac]

@justadogistaken I mean… what problem are you actually trying to solve, if any? If you want to see how the implementation works just for a survey or something, the code is available.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[flouthoc]

Converting this to discussion since this looks more like a question/discussion rather than feature request at this point.