Skip to content

Commit c566bbf

Browse files
authored
fix(setup): don't fail mount on empty bucket subfolder (#208)
## Problem Running an HF Job with a bucket volume pointing at an empty (or not-yet-created) subfolder fails the mount before the job starts, surfacing as a `MountFailure`. `validate_path_prefix()` (`src/hub_api.rs`) runs before mounting and treats an empty listing as "does not exist", then `src/setup.rs` panics the sidecar via `.unwrap_or_else(|e| panic!(...))`. For a bucket this is wrong: a subfolder that is empty and one that does not exist are indistinguishable (both list as `[]`), because bucket keys are flat and a prefix only "exists" if there are objects under it. This blocks exactly the case you'd want to support: mounting an empty/new subfolder in order to write into it. The validation also ran before `read_only` was computed, so it even blocked write mounts. - Direct mount of an empty bucket subfolder (`-v hf://buckets/org/b/sub/dir:/mnt`) hits the bug. - Bucket root (`-v hf://buckets/org/b:/mnt`) is fine (validation skipped for an empty prefix). - Local-dir sources are fine because `huggingface_hub` works around this by uploading a `.keep` placeholder. ## Fix Move the validation after `read_only` is computed and only run it for read-only mounts, warning instead of panicking: - **Write mounts**: skip the check entirely. The folder is created by writing into it. - **Read-only mounts**: a missing/empty prefix just yields an empty mount, so we `warn!` instead of panicking the sidecar. This keeps the fail-fast hint for typo'd subfolders without killing the job. This removes the failure at the source and makes the `.keep` placeholder in `huggingface_hub` (`sync_job_volume`) unnecessary for bucket sources.
1 parent 4cdeedc commit c566bbf

1 file changed

Lines changed: 15 additions & 9 deletions

File tree

src/setup.rs

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -387,15 +387,6 @@ pub fn build_with_runtime(
387387
.unwrap_or_else(|e| panic!("Failed to initialize Hub client: {e}"))
388388
});
389389

390-
// Validate that the subfolder exists on the remote.
391-
if !hub_client.path_prefix().is_empty() {
392-
runtime.block_on(async {
393-
hub_client.validate_path_prefix().await.unwrap_or_else(|e| {
394-
panic!("{e}");
395-
});
396-
});
397-
}
398-
399390
if options.overlay && options.read_only {
400391
panic!(
401392
"--overlay with --read-only is pointless: overlay enables local writes, --read-only disables them. Use --read-only alone instead."
@@ -407,6 +398,21 @@ pub fn build_with_runtime(
407398
info!("Repo mounts are always read-only");
408399
}
409400

401+
// Validate that the subfolder exists on the remote, but only for read-only
402+
// mounts. A bucket subfolder cannot be distinguished from a non-existent
403+
// one (both list as empty), so failing here would block the legitimate
404+
// case of mounting an empty/new subfolder to write into it. For write
405+
// mounts the folder is created by writing, so we skip the check entirely.
406+
// For read-only mounts a missing prefix just yields an empty mount, so we
407+
// warn instead of panicking the sidecar.
408+
if read_only && !hub_client.path_prefix().is_empty() {
409+
runtime.block_on(async {
410+
if let Err(e) = hub_client.validate_path_prefix().await {
411+
warn!("{e}");
412+
}
413+
});
414+
}
415+
410416
// Overlay: local writes allowed, but no remote write token/upload.
411417
let remote_read_only = read_only || options.overlay;
412418
let refresher = hub_client.token_refresher(remote_read_only);

0 commit comments

Comments
 (0)