Skip to content

Add resumable downloads and exponential-backoff retries#1009

Open
chuckyutan wants to merge 2 commits into
nathom:devfrom
chuckyutan:resumable-downloads
Open

Add resumable downloads and exponential-backoff retries#1009
chuckyutan wants to merge 2 commits into
nathom:devfrom
chuckyutan:resumable-downloads

Conversation

@chuckyutan

Copy link
Copy Markdown

Problem

Large tracks fail permanently on a single connection drop. A requests-level IncompleteRead during a long download bubbles up, and the current logic only retries once before giving up — and that retry restarts the download from byte 0. On flaky connections or large hi-res files, a single mid-stream drop is enough to lose the whole track.

Example failure:

ERROR  Error downloading track '...', retrying:
       ('Connection broken: IncompleteRead(30225799 bytes read, 34705938 more expected)', ...)
ERROR  Persistent error downloading track '...', skipping: (...)

Changes

fast_async_download — resumable downloads

  • If a partial file already exists on disk, request the remainder with a Range: bytes=N- header.
  • Append (ab) only when the server honors the range with 206 Partial Content. If it responds 200 OK (range ignored), fall back to a fresh overwrite (wb) so existing bytes aren't duplicated/corrupted.
  • Treat 416 Range Not Satisfiable as already-complete and return.
  • Covers all sources that stream through fast_async_download: Qobuz, Tidal, SoundCloud (FLAC), and unencrypted Deezer.

track.py — more retries with backoff

  • Retry up to 3 times (4 attempts total) instead of one, with exponential backoff (2s / 4s / 8s).
  • Applies to all sources. Combined with resume, retries continue from where the download left off instead of restarting.

Notes

  • No config/API changes; behavior is transparent.
  • Encrypted Deezer and SoundCloud-mp3 (m3u8) use separate download paths and don't gain resume, but still benefit from the increased retries.

chuckyutan and others added 2 commits June 26, 2026 17:32
Large tracks would fail permanently on a single connection drop
(IncompleteRead) because downloads were not resumable and only one
retry was attempted.

- fast_async_download: resume interrupted downloads via the Range
  header, appending only when the server honors it with 206 (falls
  back to a fresh download on 200 to avoid duplicating bytes; treats
  416 as already-complete). Covers Qobuz, Tidal, SoundCloud-FLAC and
  unencrypted Deezer.
- track.py: retry up to 3 times (4 attempts total) with exponential
  backoff (2s/4s/8s) instead of giving up after one retry. Applies to
  all sources.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Spin up a local HTTP server that interrupts the first response
mid-stream (reproducing IncompleteRead) and verify that a retry resumes
via the Range header and produces a byte-for-byte correct file. Also
covers the 200-fallback case where the server ignores Range, ensuring
the partial file is overwritten rather than appended to.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant