Commit 4cdeedc
authored
feat(fuse): implement link() as a server-side copy (#206)
## Problem
`link()` returns ENOTSUP, so hardlink-first import pipelines fall back
to a full copy through the mount. Copying a file whose bytes already
live in CAS re-chunks and re-uploads everything, and dedup against the
existing xorbs runs into xet-core's defrag prevention, which re-stores a
large share of the bytes in short fragments.
Measured on production buckets (multi-GB media files imported this way):
~2x stored bytes, +90-160% xorb overhead, reconstructions of 15-17k
segments alternating between two xorb sets, CPR pinned at the
defrag-prevention hysteresis equilibrium (~8 chunks/segment). Related:
the metrics side of that investigation is huggingface/xet-core#886.
## Change
Implement `link()` as a **server-side copy**: one batch `AddFile`
pointing at the source's committed xet hash. No bytes move through CAS,
the import is instant, and the original clean layout is preserved.
The alias gets its own inode, so this intentionally diverges from POSIX
same-inode semantics: writes to one path never affect the other, and
`st_ino` differs between the two paths. Callers that hardlink for
instant-copy semantics (the *arr import case) get exactly what they
need. The previous ENOTSUP rationale (in-memory links never persisted to
the Hub) no longer applies since the alias is a real Hub entry.
Guards, mirroring the rename/create paths:
- dirty source or no committed hash: ENOTSUP (callers keep their copy
fallback, and we never alias a stale hash)
- overlay mode: ENOTSUP (must not mutate the remote)
- directory source: EPERM, missing parent: ENOENT, non-directory parent:
ENOTDIR, existing target: EEXIST (checked against remote children too,
and re-checked under the write lock)
- read-only mount: EROFS, OS junk names: EACCES
Structured like `rename()`: validate under read lock, commit the batch
op, insert the alias under write lock (with negative-cache removal and
queued-delete cancellation for the destination path).
NFS backend unchanged: nfsserve 0.11 does not dispatch `NFSPROC3_LINK`
(proc_unavail), so hardlinks over NFS fail at the protocol level before
reaching us. Supporting it would need an upstream nfsserve change.
## Tests
Four new tests in `virtual_fs/tests.rs`: happy path (exactly one AddFile
with the source hash, no delete, alias resolvable with its own inode,
source untouched), dirty source (ENOTSUP, no remote op), existing target
(EEXIST, no remote op), directory source (EPERM). Full lib suite: 340
passed.1 parent 578081d commit 4cdeedc
3 files changed
Lines changed: 336 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
417 | 417 | | |
418 | 418 | | |
419 | 419 | | |
420 | | - | |
421 | | - | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
422 | 430 | | |
423 | 431 | | |
424 | 432 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2928 | 2928 | | |
2929 | 2929 | | |
2930 | 2930 | | |
| 2931 | + | |
| 2932 | + | |
| 2933 | + | |
| 2934 | + | |
| 2935 | + | |
| 2936 | + | |
| 2937 | + | |
| 2938 | + | |
| 2939 | + | |
| 2940 | + | |
| 2941 | + | |
| 2942 | + | |
| 2943 | + | |
| 2944 | + | |
| 2945 | + | |
| 2946 | + | |
| 2947 | + | |
| 2948 | + | |
| 2949 | + | |
| 2950 | + | |
| 2951 | + | |
| 2952 | + | |
| 2953 | + | |
| 2954 | + | |
| 2955 | + | |
| 2956 | + | |
| 2957 | + | |
| 2958 | + | |
| 2959 | + | |
| 2960 | + | |
| 2961 | + | |
| 2962 | + | |
| 2963 | + | |
| 2964 | + | |
| 2965 | + | |
| 2966 | + | |
| 2967 | + | |
| 2968 | + | |
| 2969 | + | |
| 2970 | + | |
| 2971 | + | |
| 2972 | + | |
| 2973 | + | |
| 2974 | + | |
| 2975 | + | |
| 2976 | + | |
| 2977 | + | |
| 2978 | + | |
| 2979 | + | |
| 2980 | + | |
| 2981 | + | |
| 2982 | + | |
| 2983 | + | |
| 2984 | + | |
| 2985 | + | |
| 2986 | + | |
| 2987 | + | |
| 2988 | + | |
| 2989 | + | |
| 2990 | + | |
| 2991 | + | |
| 2992 | + | |
| 2993 | + | |
| 2994 | + | |
| 2995 | + | |
| 2996 | + | |
| 2997 | + | |
| 2998 | + | |
| 2999 | + | |
| 3000 | + | |
| 3001 | + | |
| 3002 | + | |
| 3003 | + | |
| 3004 | + | |
| 3005 | + | |
| 3006 | + | |
| 3007 | + | |
| 3008 | + | |
| 3009 | + | |
| 3010 | + | |
| 3011 | + | |
| 3012 | + | |
| 3013 | + | |
| 3014 | + | |
| 3015 | + | |
| 3016 | + | |
| 3017 | + | |
| 3018 | + | |
| 3019 | + | |
| 3020 | + | |
| 3021 | + | |
| 3022 | + | |
| 3023 | + | |
| 3024 | + | |
| 3025 | + | |
| 3026 | + | |
| 3027 | + | |
| 3028 | + | |
| 3029 | + | |
| 3030 | + | |
| 3031 | + | |
| 3032 | + | |
| 3033 | + | |
| 3034 | + | |
| 3035 | + | |
| 3036 | + | |
| 3037 | + | |
| 3038 | + | |
| 3039 | + | |
| 3040 | + | |
| 3041 | + | |
| 3042 | + | |
| 3043 | + | |
| 3044 | + | |
| 3045 | + | |
| 3046 | + | |
| 3047 | + | |
| 3048 | + | |
| 3049 | + | |
| 3050 | + | |
| 3051 | + | |
| 3052 | + | |
| 3053 | + | |
| 3054 | + | |
2931 | 3055 | | |
2932 | 3056 | | |
2933 | 3057 | | |
| |||
3091 | 3215 | | |
3092 | 3216 | | |
3093 | 3217 | | |
3094 | | - | |
3095 | | - | |
3096 | | - | |
3097 | | - | |
3098 | | - | |
3099 | | - | |
3100 | 3218 | | |
3101 | 3219 | | |
3102 | 3220 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4458 | 4458 | | |
4459 | 4459 | | |
4460 | 4460 | | |
| 4461 | + | |
| 4462 | + | |
| 4463 | + | |
| 4464 | + | |
| 4465 | + | |
| 4466 | + | |
| 4467 | + | |
| 4468 | + | |
| 4469 | + | |
| 4470 | + | |
| 4471 | + | |
| 4472 | + | |
| 4473 | + | |
| 4474 | + | |
| 4475 | + | |
| 4476 | + | |
| 4477 | + | |
| 4478 | + | |
| 4479 | + | |
| 4480 | + | |
| 4481 | + | |
| 4482 | + | |
| 4483 | + | |
| 4484 | + | |
| 4485 | + | |
| 4486 | + | |
| 4487 | + | |
| 4488 | + | |
| 4489 | + | |
| 4490 | + | |
| 4491 | + | |
| 4492 | + | |
| 4493 | + | |
| 4494 | + | |
| 4495 | + | |
| 4496 | + | |
| 4497 | + | |
| 4498 | + | |
| 4499 | + | |
| 4500 | + | |
| 4501 | + | |
| 4502 | + | |
| 4503 | + | |
| 4504 | + | |
| 4505 | + | |
| 4506 | + | |
| 4507 | + | |
| 4508 | + | |
| 4509 | + | |
| 4510 | + | |
| 4511 | + | |
| 4512 | + | |
| 4513 | + | |
| 4514 | + | |
| 4515 | + | |
| 4516 | + | |
| 4517 | + | |
| 4518 | + | |
| 4519 | + | |
| 4520 | + | |
| 4521 | + | |
| 4522 | + | |
| 4523 | + | |
| 4524 | + | |
| 4525 | + | |
| 4526 | + | |
| 4527 | + | |
| 4528 | + | |
| 4529 | + | |
| 4530 | + | |
| 4531 | + | |
| 4532 | + | |
| 4533 | + | |
| 4534 | + | |
| 4535 | + | |
| 4536 | + | |
| 4537 | + | |
| 4538 | + | |
| 4539 | + | |
| 4540 | + | |
| 4541 | + | |
| 4542 | + | |
| 4543 | + | |
| 4544 | + | |
| 4545 | + | |
| 4546 | + | |
| 4547 | + | |
| 4548 | + | |
| 4549 | + | |
| 4550 | + | |
| 4551 | + | |
| 4552 | + | |
| 4553 | + | |
| 4554 | + | |
| 4555 | + | |
| 4556 | + | |
| 4557 | + | |
| 4558 | + | |
| 4559 | + | |
| 4560 | + | |
| 4561 | + | |
| 4562 | + | |
| 4563 | + | |
| 4564 | + | |
| 4565 | + | |
| 4566 | + | |
| 4567 | + | |
| 4568 | + | |
| 4569 | + | |
| 4570 | + | |
| 4571 | + | |
| 4572 | + | |
| 4573 | + | |
| 4574 | + | |
| 4575 | + | |
| 4576 | + | |
| 4577 | + | |
| 4578 | + | |
| 4579 | + | |
| 4580 | + | |
| 4581 | + | |
| 4582 | + | |
| 4583 | + | |
| 4584 | + | |
| 4585 | + | |
| 4586 | + | |
| 4587 | + | |
| 4588 | + | |
| 4589 | + | |
| 4590 | + | |
| 4591 | + | |
| 4592 | + | |
| 4593 | + | |
| 4594 | + | |
| 4595 | + | |
| 4596 | + | |
| 4597 | + | |
| 4598 | + | |
| 4599 | + | |
| 4600 | + | |
| 4601 | + | |
| 4602 | + | |
| 4603 | + | |
| 4604 | + | |
| 4605 | + | |
| 4606 | + | |
| 4607 | + | |
| 4608 | + | |
| 4609 | + | |
| 4610 | + | |
| 4611 | + | |
| 4612 | + | |
| 4613 | + | |
| 4614 | + | |
| 4615 | + | |
| 4616 | + | |
| 4617 | + | |
| 4618 | + | |
| 4619 | + | |
| 4620 | + | |
| 4621 | + | |
| 4622 | + | |
| 4623 | + | |
| 4624 | + | |
| 4625 | + | |
| 4626 | + | |
| 4627 | + | |
| 4628 | + | |
| 4629 | + | |
| 4630 | + | |
| 4631 | + | |
| 4632 | + | |
| 4633 | + | |
| 4634 | + | |
| 4635 | + | |
| 4636 | + | |
| 4637 | + | |
| 4638 | + | |
| 4639 | + | |
| 4640 | + | |
| 4641 | + | |
| 4642 | + | |
| 4643 | + | |
| 4644 | + | |
| 4645 | + | |
| 4646 | + | |
| 4647 | + | |
| 4648 | + | |
| 4649 | + | |
| 4650 | + | |
| 4651 | + | |
| 4652 | + | |
| 4653 | + | |
| 4654 | + | |
| 4655 | + | |
| 4656 | + | |
| 4657 | + | |
| 4658 | + | |
| 4659 | + | |
| 4660 | + | |
| 4661 | + | |
| 4662 | + | |
4461 | 4663 | | |
4462 | 4664 | | |
4463 | 4665 | | |
| |||
0 commit comments