Skip to content

Commit 4cdeedc

Browse files
authored
feat(fuse): implement link() as a server-side copy (#206)
## Problem `link()` returns ENOTSUP, so hardlink-first import pipelines fall back to a full copy through the mount. Copying a file whose bytes already live in CAS re-chunks and re-uploads everything, and dedup against the existing xorbs runs into xet-core's defrag prevention, which re-stores a large share of the bytes in short fragments. Measured on production buckets (multi-GB media files imported this way): ~2x stored bytes, +90-160% xorb overhead, reconstructions of 15-17k segments alternating between two xorb sets, CPR pinned at the defrag-prevention hysteresis equilibrium (~8 chunks/segment). Related: the metrics side of that investigation is huggingface/xet-core#886. ## Change Implement `link()` as a **server-side copy**: one batch `AddFile` pointing at the source's committed xet hash. No bytes move through CAS, the import is instant, and the original clean layout is preserved. The alias gets its own inode, so this intentionally diverges from POSIX same-inode semantics: writes to one path never affect the other, and `st_ino` differs between the two paths. Callers that hardlink for instant-copy semantics (the *arr import case) get exactly what they need. The previous ENOTSUP rationale (in-memory links never persisted to the Hub) no longer applies since the alias is a real Hub entry. Guards, mirroring the rename/create paths: - dirty source or no committed hash: ENOTSUP (callers keep their copy fallback, and we never alias a stale hash) - overlay mode: ENOTSUP (must not mutate the remote) - directory source: EPERM, missing parent: ENOENT, non-directory parent: ENOTDIR, existing target: EEXIST (checked against remote children too, and re-checked under the write lock) - read-only mount: EROFS, OS junk names: EACCES Structured like `rename()`: validate under read lock, commit the batch op, insert the alias under write lock (with negative-cache removal and queued-delete cancellation for the destination path). NFS backend unchanged: nfsserve 0.11 does not dispatch `NFSPROC3_LINK` (proc_unavail), so hardlinks over NFS fail at the protocol level before reaching us. Supporting it would need an upstream nfsserve change. ## Tests Four new tests in `virtual_fs/tests.rs`: happy path (exactly one AddFile with the source hash, no delete, alias resolvable with its own inode, source untouched), dirty source (ENOTSUP, no remote op), existing target (EEXIST, no remote op), directory source (EPERM). Full lib suite: 340 passed.
1 parent 578081d commit 4cdeedc

3 files changed

Lines changed: 336 additions & 8 deletions

File tree

src/fuse.rs

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -417,8 +417,16 @@ impl Filesystem for FuseAdapter {
417417
}
418418
}
419419

420-
fn link(&self, _req: &Request, _ino: INodeNo, _newparent: INodeNo, _newname: &OsStr, reply: ReplyEntry) {
421-
reply.error(Errno::from_i32(libc::ENOTSUP));
420+
/// Hard link, implemented as a server-side copy: a new Hub entry pointing at
421+
/// the source's xet hash (no data re-uploaded). The alias gets its own inode,
422+
/// so st_ino differs from the source; callers like the *arr import pipelines
423+
/// only need the instant copy semantics.
424+
fn link(&self, _req: &Request, ino: INodeNo, newparent: INodeNo, newname: &OsStr, reply: ReplyEntry) {
425+
let newname = os_to_str!(newname, reply);
426+
match self.runtime.block_on(self.virtual_fs.link(ino.0, newparent.0, newname)) {
427+
Ok(attr) => self.reply_entry_tracked(reply, &attr),
428+
Err(e) => reply.error(Errno::from_i32(e)),
429+
}
422430
}
423431

424432
/// Remove an empty directory.

src/virtual_fs/mod.rs

Lines changed: 124 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2928,6 +2928,130 @@ impl VirtualFs {
29282928
Ok(self.make_vfs_attr(inodes.get(ino).ok_or(libc::ENOENT)?))
29292929
}
29302930

2931+
/// Hard-link `ino` at `newparent/newname` as a server-side copy: a new Hub
2932+
/// file entry referencing the source's xet hash, so no bytes move through
2933+
/// CAS. The alias gets its own inode (writes to one path never affect the
2934+
/// other), which diverges from POSIX same-inode semantics but matches what
2935+
/// link() callers like the *arr import pipelines need: an instant,
2936+
/// storage-free copy. Re-uploading the content instead would shred its
2937+
/// layout (dedup against the existing xorbs runs into xet-core's defrag
2938+
/// prevention, re-storing a large share of the bytes in fragments).
2939+
///
2940+
/// Sources without a committed hash (dirty, or created and never flushed)
2941+
/// return ENOTSUP so callers fall back to a regular copy.
2942+
pub async fn link(&self, ino: u64, newparent: u64, newname: &str) -> VirtualFsResult<VirtualFsAttr> {
2943+
if self.read_only {
2944+
return Err(libc::EROFS);
2945+
}
2946+
// Overlay keeps writes local; committing an alias to the Hub would
2947+
// mutate the remote behind the overlay's back.
2948+
if self.overlay() {
2949+
return Err(libc::ENOTSUP);
2950+
}
2951+
if self.filter_os_files && is_os_junk(newname) {
2952+
debug!("link: rejecting OS junk name: {}", newname);
2953+
return Err(libc::EACCES);
2954+
}
2955+
2956+
debug!("link: ino={}, newparent={}, newname={}", ino, newparent, newname);
2957+
2958+
// Phase 1a: validate the source under a read lock, before loading the
2959+
// destination's remote children. A rejected source (dirty or hashless,
2960+
// the routine *arr link-then-copy fallback) must not pay for a
2961+
// list_tree it would throw away.
2962+
let (source_path, xet_hash, size, mode, uid, gid) = {
2963+
let inodes = self.inode_table.read().expect("inodes poisoned");
2964+
let source = inodes.get(ino).ok_or(libc::ENOENT)?;
2965+
if source.kind != InodeKind::File {
2966+
return Err(libc::EPERM);
2967+
}
2968+
if source.is_dirty() {
2969+
return Err(libc::ENOTSUP);
2970+
}
2971+
let Some(hash) = source.xet_hash.as_ref().filter(|hash| !hash.is_empty()).cloned() else {
2972+
return Err(libc::ENOTSUP);
2973+
};
2974+
(
2975+
source.full_path.clone(),
2976+
hash,
2977+
source.size,
2978+
source.mode,
2979+
source.uid,
2980+
source.gid,
2981+
)
2982+
};
2983+
2984+
// Load remote children so the EEXIST check below also sees files that
2985+
// exist remotely but were never looked up (also validates newparent).
2986+
self.ensure_children_loaded(newparent).await?;
2987+
2988+
// Phase 1b: validate the destination under a read lock.
2989+
let new_full_path = {
2990+
let inodes = self.inode_table.read().expect("inodes poisoned");
2991+
let parent_entry = match inodes.get(newparent) {
2992+
Some(e) if e.kind == InodeKind::Directory => e,
2993+
Some(_) => return Err(libc::ENOTDIR),
2994+
None => return Err(libc::ENOENT),
2995+
};
2996+
let new_full_path = inode::child_path(&parent_entry.full_path, newname);
2997+
if inodes.lookup_child(newparent, newname).is_some() {
2998+
return Err(libc::EEXIST);
2999+
}
3000+
new_full_path
3001+
};
3002+
3003+
// Phase 2: commit the alias to the Hub.
3004+
let now = SystemTime::now();
3005+
let mtime_ms = now.duration_since(UNIX_EPOCH).unwrap_or_default().as_millis() as u64;
3006+
let ops = [BatchOp::AddFile {
3007+
path: new_full_path.clone(),
3008+
xet_hash: xet_hash.clone(),
3009+
mtime: mtime_ms,
3010+
content_type: None,
3011+
}];
3012+
if let Err(e) = self.hub_client.batch_operations(&ops).await {
3013+
error!(
3014+
"link: failed to commit {} as alias of {}: {}",
3015+
new_full_path, source_path, e
3016+
);
3017+
return Err(libc::EIO);
3018+
}
3019+
3020+
self.negative_cache_remove(&new_full_path);
3021+
// Cancel any queued remote delete for the destination path (rm b && ln a b).
3022+
if let Some(fm) = &self.flush_manager {
3023+
fm.cancel_delete(&new_full_path);
3024+
}
3025+
3026+
// Phase 3: insert the alias into the inode table under the write lock.
3027+
let mut inodes = self.inode_table.write().expect("inodes poisoned");
3028+
if inodes.get(newparent).is_none() {
3029+
return Err(libc::ENOENT);
3030+
}
3031+
// A concurrent create may have won the race since phase 1; the remote
3032+
// add already landed, but locally the newer entry owns the name.
3033+
if inodes.lookup_child(newparent, newname).is_some() {
3034+
return Err(libc::EEXIST);
3035+
}
3036+
info!("Linked {} -> {} (server-side copy)", source_path, new_full_path);
3037+
let new_ino = inodes.insert(
3038+
newparent,
3039+
newname.to_string(),
3040+
new_full_path,
3041+
InodeKind::File,
3042+
size,
3043+
now,
3044+
Some(xet_hash),
3045+
mode,
3046+
uid,
3047+
gid,
3048+
);
3049+
inodes.touch_parent(newparent, now);
3050+
inodes.touch(new_ino);
3051+
3052+
Ok(self.make_vfs_attr(inodes.get(new_ino).ok_or(libc::EIO)?))
3053+
}
3054+
29313055
pub async fn unlink(&self, parent: u64, name: &str) -> VirtualFsResult<()> {
29323056
if self.read_only {
29333057
return Err(libc::EROFS);
@@ -3091,12 +3215,6 @@ impl VirtualFs {
30913215
}
30923216
}
30933217

3094-
pub async fn link(&self, _ino: u64, _new_parent: u64, _new_name: &str) -> VirtualFsResult<VirtualFsAttr> {
3095-
// Hard links are not supported — they are ephemeral (in-memory only) and never
3096-
// persisted to the hub, which makes them a source of subtle bugs with no benefit.
3097-
Err(libc::ENOTSUP)
3098-
}
3099-
31003218
pub async fn rmdir(&self, parent: u64, name: &str) -> VirtualFsResult<()> {
31013219
if self.read_only {
31023220
return Err(libc::EROFS);

src/virtual_fs/tests.rs

Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4458,6 +4458,208 @@ fn rename_clean_file_remote_and_local() {
44584458
});
44594459
}
44604460

4461+
// ── link(): server-side copy ─────────────────────────────────────────────
4462+
4463+
/// link() on a clean remote file commits one AddFile with the source's xet
4464+
/// hash (no data upload) and inserts a separate alias inode locally.
4465+
#[test]
4466+
fn link_clean_file_creates_server_side_copy() {
4467+
let hub = MockHub::new();
4468+
hub.add_file("src.txt", 100, Some("hash1"), None);
4469+
let xet = MockXet::new();
4470+
let (rt, vfs) = vfs_simple(&hub, &xet);
4471+
4472+
rt.block_on(async {
4473+
let src_attr = vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4474+
4475+
let alias_attr = vfs.link(src_attr.ino, ROOT_INODE, "alias.txt").await.unwrap();
4476+
assert_ne!(alias_attr.ino, src_attr.ino, "alias gets its own inode");
4477+
assert_eq!(alias_attr.size, 100);
4478+
4479+
let logs = hub.take_batch_log();
4480+
assert_eq!(logs.len(), 1, "exactly one batch commit");
4481+
assert_eq!(logs[0].len(), 1, "a single AddFile, no delete");
4482+
match &logs[0][0] {
4483+
BatchOp::AddFile { path, xet_hash, .. } => {
4484+
assert_eq!(path, "alias.txt");
4485+
assert_eq!(xet_hash, "hash1");
4486+
}
4487+
other => panic!("expected AddFile, got {other:?}"),
4488+
}
4489+
4490+
// The alias is a real entry: resolvable, right hash, source untouched.
4491+
let inodes = vfs.inode_table.read().unwrap();
4492+
let alias = inodes.get(alias_attr.ino).unwrap();
4493+
assert_eq!(alias.full_path.as_ref(), "alias.txt");
4494+
assert_eq!(alias.xet_hash.as_deref(), Some("hash1"));
4495+
assert!(!alias.is_dirty());
4496+
let src = inodes.get(src_attr.ino).unwrap();
4497+
assert_eq!(src.full_path.as_ref(), "src.txt");
4498+
});
4499+
}
4500+
4501+
/// link() on a dirty source must not alias a stale hash: ENOTSUP so the
4502+
/// caller falls back to a regular copy.
4503+
#[test]
4504+
fn link_dirty_source_not_supported() {
4505+
let hub = MockHub::new();
4506+
hub.add_file("src.txt", 100, Some("hash1"), None);
4507+
let xet = MockXet::new();
4508+
let (rt, vfs) = vfs_simple(&hub, &xet);
4509+
4510+
rt.block_on(async {
4511+
let src_attr = vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4512+
{
4513+
let mut inodes = vfs.inode_table.write().unwrap();
4514+
inodes.get_mut(src_attr.ino).unwrap().set_dirty();
4515+
}
4516+
4517+
let err = vfs.link(src_attr.ino, ROOT_INODE, "alias.txt").await.unwrap_err();
4518+
assert_eq!(err, libc::ENOTSUP);
4519+
assert!(hub.take_batch_log().is_empty(), "no remote op for a rejected link");
4520+
});
4521+
}
4522+
4523+
/// link() onto an existing name is EEXIST and sends no remote op.
4524+
#[test]
4525+
fn link_existing_target_eexist() {
4526+
let hub = MockHub::new();
4527+
hub.add_file("src.txt", 100, Some("hash1"), None);
4528+
hub.add_file("dst.txt", 50, Some("hash2"), None);
4529+
let xet = MockXet::new();
4530+
let (rt, vfs) = vfs_simple(&hub, &xet);
4531+
4532+
rt.block_on(async {
4533+
let src_attr = vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4534+
let err = vfs.link(src_attr.ino, ROOT_INODE, "dst.txt").await.unwrap_err();
4535+
assert_eq!(err, libc::EEXIST);
4536+
assert!(hub.take_batch_log().is_empty());
4537+
});
4538+
}
4539+
4540+
/// link() on a directory is EPERM (POSIX).
4541+
#[test]
4542+
fn link_directory_eperm() {
4543+
let hub = MockHub::new();
4544+
hub.add_file("d/x.txt", 10, Some("hash1"), None);
4545+
let xet = MockXet::new();
4546+
let (rt, vfs) = vfs_simple(&hub, &xet);
4547+
4548+
rt.block_on(async {
4549+
let dir_attr = vfs.lookup(ROOT_INODE, "d").await.unwrap();
4550+
let err = vfs.link(dir_attr.ino, ROOT_INODE, "d2").await.unwrap_err();
4551+
assert_eq!(err, libc::EPERM);
4552+
assert!(hub.take_batch_log().is_empty());
4553+
});
4554+
}
4555+
4556+
/// link() on a clean file whose committed hash is missing/empty returns
4557+
/// ENOTSUP (we never alias a hashless entry), with no remote op.
4558+
#[test]
4559+
fn link_no_committed_hash_not_supported() {
4560+
let hub = MockHub::new();
4561+
hub.add_file("src.txt", 100, Some("hash1"), None);
4562+
let xet = MockXet::new();
4563+
let (rt, vfs) = vfs_simple(&hub, &xet);
4564+
4565+
rt.block_on(async {
4566+
let src_attr = vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4567+
{
4568+
let mut inodes = vfs.inode_table.write().unwrap();
4569+
inodes.get_mut(src_attr.ino).unwrap().xet_hash = None;
4570+
}
4571+
4572+
let err = vfs.link(src_attr.ino, ROOT_INODE, "alias.txt").await.unwrap_err();
4573+
assert_eq!(err, libc::ENOTSUP);
4574+
assert!(hub.take_batch_log().is_empty(), "no remote op for a rejected link");
4575+
});
4576+
}
4577+
4578+
/// link() into a missing parent directory is ENOENT, with no remote op.
4579+
#[test]
4580+
fn link_missing_parent_enoent() {
4581+
let hub = MockHub::new();
4582+
hub.add_file("src.txt", 100, Some("hash1"), None);
4583+
let xet = MockXet::new();
4584+
let (rt, vfs) = vfs_simple(&hub, &xet);
4585+
4586+
rt.block_on(async {
4587+
let src_attr = vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4588+
let err = vfs.link(src_attr.ino, 999_999, "alias.txt").await.unwrap_err();
4589+
assert_eq!(err, libc::ENOENT);
4590+
assert!(hub.take_batch_log().is_empty());
4591+
});
4592+
}
4593+
4594+
/// link() into a non-directory parent is ENOTDIR, with no remote op.
4595+
#[test]
4596+
fn link_nondir_parent_enotdir() {
4597+
let hub = MockHub::new();
4598+
hub.add_file("src.txt", 100, Some("hash1"), None);
4599+
hub.add_file("file.txt", 50, Some("hash2"), None);
4600+
let xet = MockXet::new();
4601+
let (rt, vfs) = vfs_simple(&hub, &xet);
4602+
4603+
rt.block_on(async {
4604+
let src_attr = vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4605+
let file_attr = vfs.lookup(ROOT_INODE, "file.txt").await.unwrap();
4606+
let err = vfs.link(src_attr.ino, file_attr.ino, "alias.txt").await.unwrap_err();
4607+
assert_eq!(err, libc::ENOTDIR);
4608+
assert!(hub.take_batch_log().is_empty());
4609+
});
4610+
}
4611+
4612+
/// link() to an OS junk name is rejected with EACCES, with no remote op.
4613+
#[test]
4614+
fn link_os_junk_name_eacces() {
4615+
let hub = MockHub::new();
4616+
hub.add_file("src.txt", 100, Some("hash1"), None);
4617+
let xet = MockXet::new();
4618+
let (rt, vfs) = vfs_simple(&hub, &xet);
4619+
4620+
rt.block_on(async {
4621+
let src_attr = vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4622+
let err = vfs.link(src_attr.ino, ROOT_INODE, ".DS_Store").await.unwrap_err();
4623+
assert_eq!(err, libc::EACCES);
4624+
assert!(hub.take_batch_log().is_empty());
4625+
});
4626+
}
4627+
4628+
/// link() on a read-only mount is EROFS, with no remote op.
4629+
#[test]
4630+
fn link_read_only_erofs() {
4631+
let hub = MockHub::new();
4632+
hub.add_file("src.txt", 100, Some("hash1"), None);
4633+
let xet = MockXet::new();
4634+
let (rt, vfs) = vfs_readonly(&hub, &xet);
4635+
4636+
rt.block_on(async {
4637+
let src_attr = vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4638+
let err = vfs.link(src_attr.ino, ROOT_INODE, "alias.txt").await.unwrap_err();
4639+
assert_eq!(err, libc::EROFS);
4640+
assert!(hub.take_batch_log().is_empty());
4641+
});
4642+
}
4643+
4644+
/// link() in overlay mode is ENOTSUP (committing an alias would mutate the
4645+
/// remote behind the overlay's back), with no remote op.
4646+
#[test]
4647+
fn link_overlay_not_supported() {
4648+
let hub = MockHub::new();
4649+
hub.add_file("src.txt", 100, Some("hash1"), None);
4650+
let xet = MockXet::new();
4651+
4652+
let overlay_root = fresh_overlay_dir("link");
4653+
let t = make_overlay_test_vfs_with_root(hub.clone(), xet, overlay_root);
4654+
4655+
t.runtime.block_on(async {
4656+
let src_attr = t.vfs.lookup(ROOT_INODE, "src.txt").await.unwrap();
4657+
let err = t.vfs.link(src_attr.ino, ROOT_INODE, "alias.txt").await.unwrap_err();
4658+
assert_eq!(err, libc::ENOTSUP);
4659+
assert!(hub.take_batch_log().is_empty());
4660+
});
4661+
}
4662+
44614663
// ── LRU eviction: end-to-end memory bound ───────────────────────────────
44624664

44634665
/// End-to-end test of the forget-driven eviction path on a hierarchical

0 commit comments

Comments
 (0)