fix: handle hf_hub_download crash when stderr lacks a real file descriptor#4065
Open
tobocop2 wants to merge 1 commit intohuggingface:mainfrom
Open
fix: handle hf_hub_download crash when stderr lacks a real file descriptor#4065tobocop2 wants to merge 1 commit intohuggingface:mainfrom
tobocop2 wants to merge 1 commit intohuggingface:mainfrom
Conversation
f08d2e6 to
6eff9fb
Compare
This was referenced Apr 7, 2026
_create_progress_bar crashes with ValueError when stderr.fileno() returns -1 (Textual, Jupyter, pytest) or raises OSError. The crash is in tqdm's multiprocessing lock init, not the download itself. Add _SafeTqdm fallback that uses a threading lock instead, and wrap both cls() calls in try/except (OSError, ValueError).
6f4b2a4 to
43399d3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
hf_hub_downloadcrashes withValueError: bad value(s) in fds_to_keepin any environment wheresys.stderris not backed by a real file descriptor.The download itself is fine. Only tqdm's progress bar initialization fails, but because
_create_progress_barhas no safety net, it takes down the entire download.Affected environments
Any runtime where
sys.stderris not backed by a real file descriptor:Root cause
_create_progress_barcallscls(disable=disable, name=name, **kwargs)with no exception handling. When stderr'sfileno()returns-1, tqdm's__init__tries to create a multiprocessing lock, which spawns a resource tracker subprocess viafork_exec(), which rejects the invalid fd.Crash Demo
Fix
Add a
_SafeTqdmsubclass that uses athreading.RLockinstead of a multiprocessing lock, and wrap bothcls()calls in_create_progress_barwithtry/except (OSError, ValueError)to fall back to_SafeTqdm.Fixes #4066
Note
Low Risk
Low risk: changes are limited to progress bar initialization/error handling and should not affect download correctness, only whether/which progress reporting is shown.
Overview
Prevents
hf_hub_download/snapshot progress reporting from crashing in environments wheresys.stderr.fileno()is invalid by wrapping_create_progress_barconstruction intry/except (OSError, ValueError)and falling back to a new_SafeTqdmthat uses athreading.RLock.When the HF
tqdmsubclass cannot be initialized, the code now emits a warning and continues the download without progress reporting. Adds a regression test that patchessys.stderrto return-1forfileno()and asserts the download succeeds.Reviewed by Cursor Bugbot for commit 43399d3. Bugbot is set up for automated code reviews on this repo. Configure here.