fix: smoother progress for xet downloads

[tobocop2]

Fixes #4058.

Problem

Xet downloads barely report progress. For a 2.5GB file the bar updates about 9 times total -- 0% → 1.7% → 5.6% → 11% → 25% → 84% → 100%. That 25% to 84% gap is ~1.5GB of nothing. Looks completely stuck.

The cause: xet_get() and download_bucket_files() use a 1-arg callback. xet-core supports a 2-arg signature that gives fine-grained network-level updates instead.

Fix

make_xet_progress_callback(progress_bar, file_size) returns a 2-arg callback so xet-core gives us useful progress. Transfer bytes are scaled to file size so the bar tracks 0-100% correctly (xet dedup can make transfer size differ from file size).

Wired into both xet_get() and download_bucket_files(). No public API changes.

Demo

Qwen3 4B (2.5 GB, xet-stored) with a custom tqdm_class callback. Builds on #4056 which fixes custom tqdm_class in non-TTY environments.

Before (bar appears stuck)

After (smooth progress)

Note

Both downloads finish in roughly the same time (~80s). The difference is purely in how progress is reported.

Multi-file: snapshot_download Qwen3-8B (5 xet shards, 16.4 GB)

Before (bar barely moves)

After (smooth per-file progress)

Related

• feat: Add download progress callback & fix tqdm_class silenced in non-TTY #4051 / fix: add progress_updater param for granular download progress #4043 (earlier PRs that identified the granularity problem)
• Fix custom tqdm_class silently broken in non-TTY environments #4056 (fixes custom tqdm_class handling, orthogonal to this)

---

Note

Low Risk
Low risk: changes are limited to progress callback wiring/scaling for Xet downloads and are covered by new unit tests; core download/auth flows are otherwise unchanged.

Overview
Xet downloads now report smooth, accurate tqdm progress. A new make_xet_progress_callback helper generates the 2-argument xet-core callback and scales network transfer bytes to the expected file size (including capping and handling unknown sizes).

This callback replaces the previous 1-arg updater in both xet_get() and HfApi.download_bucket_files(), and adds targeted tests to validate the callback signature and scaling behavior (including multi-file shared progress bars).

^{Reviewed by Cursor Bugbot for commit d29edc7. Bugbot is set up for automated code reviews on this repo. Configure here.}