Skip to content

fix: handle hf_hub_download crash when stderr lacks a real file descriptor#4065

Open
tobocop2 wants to merge 1 commit intohuggingface:mainfrom
tobocop2:fix/tqdm-bad-fd-safety
Open

fix: handle hf_hub_download crash when stderr lacks a real file descriptor#4065
tobocop2 wants to merge 1 commit intohuggingface:mainfrom
tobocop2:fix/tqdm-bad-fd-safety

Conversation

@tobocop2
Copy link
Copy Markdown

@tobocop2 tobocop2 commented Apr 7, 2026

Problem

hf_hub_download crashes with ValueError: bad value(s) in fds_to_keep in any environment where sys.stderr is not backed by a real file descriptor.

The download itself is fine. Only tqdm's progress bar initialization fails, but because _create_progress_bar has no safety net, it takes down the entire download.

Affected environments

Any runtime where sys.stderr is not backed by a real file descriptor:

  • Textual TUI apps
  • Jupyter notebooks
  • pytest capture
  • WSGI/ASGI web servers (gunicorn, uvicorn)
  • Sandboxed/containerized runtimes
  • IDE embedded terminals

Root cause

_create_progress_bar calls cls(disable=disable, name=name, **kwargs) with no exception handling. When stderr's fileno() returns -1, tqdm's __init__ tries to create a multiprocessing lock, which spawns a resource tracker subprocess via fork_exec(), which rejects the invalid fd.

Crash Demo

Crash demo

Fix

Add a _SafeTqdm subclass that uses a threading.RLock instead of a multiprocessing lock, and wrap both cls() calls in _create_progress_bar with try/except (OSError, ValueError) to fall back to _SafeTqdm.

Fixes #4066


Note

Low Risk
Low risk: changes are limited to progress bar initialization/error handling and should not affect download correctness, only whether/which progress reporting is shown.

Overview
Prevents hf_hub_download/snapshot progress reporting from crashing in environments where sys.stderr.fileno() is invalid by wrapping _create_progress_bar construction in try/except (OSError, ValueError) and falling back to a new _SafeTqdm that uses a threading.RLock.

When the HF tqdm subclass cannot be initialized, the code now emits a warning and continues the download without progress reporting. Adds a regression test that patches sys.stderr to return -1 for fileno() and asserts the download succeeds.

Reviewed by Cursor Bugbot for commit 43399d3. Bugbot is set up for automated code reviews on this repo. Configure here.

@tobocop2 tobocop2 force-pushed the fix/tqdm-bad-fd-safety branch from f08d2e6 to 6eff9fb Compare April 7, 2026 21:52
@tobocop2 tobocop2 changed the title fix: handle tqdm crash when stderr lacks a real file descriptor fix: handle hf_hub_download crash when stderr lacks a real file descriptor Apr 7, 2026
_create_progress_bar crashes with ValueError when stderr.fileno()
returns -1 (Textual, Jupyter, pytest) or raises OSError. The crash
is in tqdm's multiprocessing lock init, not the download itself.

Add _SafeTqdm fallback that uses a threading lock instead, and
wrap both cls() calls in try/except (OSError, ValueError).
@tobocop2 tobocop2 force-pushed the fix/tqdm-bad-fd-safety branch from 6f4b2a4 to 43399d3 Compare April 8, 2026 03:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: hf_hub_download crashes with ValueError when stderr lacks a real file descriptor

1 participant