Skip macOS AppleDouble sidecar files during .data directory installation#18835
Skip macOS AppleDouble sidecar files during .data directory installation#18835qqgjyx wants to merge 2 commits intoastral-sh:mainfrom
Conversation
On non-native filesystems (exFAT, FAT32, SMB), macOS creates `._*` sidecar files to store extended attributes. These appear in directory listings during `WalkDir`/`read_dir` but are not part of the wheel RECORD, causing installation failures. Closes astral-sh#18790
There was a problem hiding this comment.
Pull request overview
This PR aims to prevent macOS AppleDouble ._* sidecar files from breaking wheel .data directory installation by skipping those entries during .data moves and script installation, avoiding RECORD mismatch errors on exFAT/FAT32/SMB.
Changes:
- Add an
is_apple_doublehelper to detect._*AppleDouble sidecar files. - Skip
._*entries duringmove_folder_recorded()directory walks and.data/scriptsinstallation loops. - Add unit tests for the helper and for skipping AppleDouble entries during
.datamoves.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Skip macOS AppleDouble resource fork sidecar files (`._*`). | ||
| if is_apple_double(src) { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
Skipping ._* entries here prevents the immediate RECORD lookup error, but it can leave stale RECORD entries behind when the wheel itself contains AppleDouble files (those entries will never be updated, and the .data directory is removed after install). This results in an installed RECORD that references non-existent .data/.../._* paths and shows up in uv pip show --files. Consider either (a) only skipping ._* files that are not present in RECORD, or (b) treating AppleDouble as invalid content and pruning matching RECORD entries before writing the final RECORD (may require taking &mut Vec<RecordEntry> instead of a slice).
|
|
||
| // Skip macOS AppleDouble resource fork sidecar files (`._*`). | ||
| if is_apple_double(&file.path()) { |
There was a problem hiding this comment.
Same issue as move_folder_recorded: skipping ._* script entries avoids install-time failures, but if a wheel includes AppleDouble files in .data/scripts, their RECORD rows will remain and end up pointing at paths that no longer exist after .data is deleted. Recommend pruning ._* entries from the RECORD (or only skipping those not in RECORD) to keep the final RECORD consistent.
| // Skip macOS AppleDouble resource fork sidecar files (`._*`). | |
| if is_apple_double(&file.path()) { | |
| let file_path = file.path(); | |
| // Skip macOS AppleDouble resource fork sidecar files (`._*`). | |
| // If such a file was listed in RECORD, remove it so the final | |
| // installed RECORD does not reference a path that was never installed. | |
| if is_apple_double(&file_path) { | |
| record.remove(&file_path); |
| /// On non-native filesystems (exFAT, FAT32, SMB), macOS stores extended attributes as `._` | ||
| /// sidecar files. These appear in directory listings but are not part of the wheel contents | ||
| /// and must be skipped during installation to avoid RECORD mismatches. |
There was a problem hiding this comment.
The doc comment says AppleDouble sidecars are "not part of the wheel contents", but the issue description notes that some wheels can include ._* entries in the archive/RECORD. It may be clearer to describe them as "not intended package content" and note that they can be either created by macOS during file operations or (incorrectly) present in wheels built on macOS.
| /// On non-native filesystems (exFAT, FAT32, SMB), macOS stores extended attributes as `._` | |
| /// sidecar files. These appear in directory listings but are not part of the wheel contents | |
| /// and must be skipped during installation to avoid RECORD mismatches. | |
| /// On non-native filesystems (exFAT, FAT32, SMB), macOS may store extended attributes as `._` | |
| /// sidecar files during file operations. These files are not intended package content, but may | |
| /// also be incorrectly present in wheels built on macOS, and must be skipped during | |
| /// installation to avoid RECORD mismatches. |
| fn is_apple_double(path: &Path) -> bool { | ||
| path.file_name() | ||
| .and_then(|n| n.to_str()) | ||
| .is_some_and(|n| n.starts_with("._")) | ||
| } |
There was a problem hiding this comment.
This helper is only applied to .data moves and .data/scripts handling. If the intent is to fully close #18790 (which also reports runtime import failures from ._*.py under purelib/platlib), AppleDouble files elsewhere in the install tree can still be created or extracted and remain in site-packages. Either adjust the PR/issue closure scope, or extend filtering/cleanup to the main wheel extraction/linking path so ._* files never land in site-packages.
On exFAT (and FAT32, SMB), macOS creates
._*AppleDouble sidecar files for every file that carries extended attributes. During.datadirectory installation,move_folder_recorded()walks the directory withWalkDirand picks up these ghost files that aren't in the wheel's RECORD — then errors out with a RECORD mismatch.This adds an
is_apple_doublecheck to skip._*entries in bothmove_folder_recorded()and theinstall_data()scripts loop.Confirmed with
jupyterlab-pygments==0.3.0on macOS 26 + exFAT — wheel contains zero._files butuv syncfails because macOS creates them during the copy.Closes #18790. See also rust-lang/rust#154692.
Test plan
test_is_apple_double: verifies the helper against various filenamestest_move_folder_recorded_skips_apple_double: creates a.datadirectory with a real file + a._sidecar not in RECORD, confirms only the real file is movedCOPYFILE_DISABLE=1 uv syncwithjupyterlab-pygmentson exFAT now succeeds