Skip to content

Add resumable downloads to get_file - #2128

Closed
nickita-khylkouski wants to merge 1 commit into
fsspec:masterfrom
nickita-khylkouski:feat/1617-resumable-downloads
Closed

Add resumable downloads to get_file#2128
nickita-khylkouski wants to merge 1 commit into
fsspec:masterfrom
nickita-khylkouski:feat/1617-resumable-downloads

Conversation

@nickita-khylkouski

@nickita-khylkouski nickita-khylkouski commented Sep 10, 2026

Copy link
Copy Markdown

Closes #1617.

Adds resume=True to get_file and get for local path destinations. The generic path opens an existing destination in append mode and seeks the remote file to the same offset. HTTP resumes with an internal range request, restarts safely if the server ignores it, and checks the remote size before accepting a completed local file. There is no public byte-range API.

Tests:

  • 8 focused resume/get-file tests passed
  • full suite: 1,917 passed, 70 skipped, 2 xfailed
  • 52 existing GNU stat -c comparison cases fail on macOS with BSD stat
  • Ruff check/format passed on all changed files

@martindurant

Copy link
Copy Markdown
Member

This is a very big PR and hard to review.

I don't mind the idea of resumable downloads (open local file with "ab" and seek in remote file) not byte ranges, although the two conflict with each other. I think the implementation should be significantly simpler than this!

@nickita-khylkouski nickita-khylkouski changed the title Add byte ranges and resumable downloads to get_file Add resumable downloads to get_file Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suggestion: continue download

2 participants