Resolve CSV file references with spaces against nested attachments#1208
Open
SixtyGrit wants to merge 1 commit into
Open
Resolve CSV file references with spaces against nested attachments#1208SixtyGrit wants to merge 1 commit into
SixtyGrit wants to merge 1 commit into
Conversation
CsvParser#file_paths transliterated every file reference's spaces to underscores before checking existence, while remove_spaces_from_filenames only renames the top level of files/ (nested entries are deliberately left as-is, per the unzip_spec added with the guided importer). A CSV referencing a nested attachment whose name contains spaces (files/subdir/photo name.jpg) could therefore never resolve: the lookup demanded underscores, the disk kept spaces. Try the reference as written first, then fall back to the underscored variant for top-level files that remove_spaces_from_filenames renamed.
Contributor
|
@SixtyGrit If this is something you need for Hyku, it also needs to be cherrypicked onto the 9-stable branch. Hyku isn't set up to use Bulkrax main yet (that includes Rob's changes that Pals brought up re: the export permissions) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CsvParser#file_pathstransliterates everyfilereference's spaces to underscores before checking existence (f.strip.tr(' ', '_')), while#remove_spaces_from_filenamesonly renames the top level offiles/— and the unzip_spec added with the guided importer (#1098) pins that one-level behavior deliberately, on the rationale that renaming nested entries would break CSV references.Those two halves contradict each other for nested attachments: a CSV referencing
files/subdir/photo name.jpgcan never resolve — the lookup demands the underscored variant, the disk keeps the spaces, and the import aborts withRuntimeError: File ... does not exist.This PR makes
file_pathstry the reference as written first, then fall back to the underscored variant (which still matches top-level files renamed byremove_spaces_from_filenames). No on-disk renaming behavior changes, so the #1098 spec's intent is preserved.Real-world reproduction
Hit on a Hyku 7.1.x tenant importing a per-work-folder zip (
files/<work-slug>/<photos>, the layout the CSV parser's parents/children conventions encourage) where four photos had spaces in their names:The file existed at
.../files/files/design-square-one/Square 1 Design.png. Nothing in the importer UI or validation surfaces the mismatch; the run simply dies on the first spaced reference. Operators have no reason to expect underscores: the CSV they wrote matches the files they zipped.Notes
A related observation from the same session, possibly worth a follow-up issue: the guided importer's upload validation passed this package, and the failure only surfaced later in
ImporterJob— the validator andfile_pathsevidently resolve attachment paths differently. Happy to file separately with details.Changes
file_paths: candidate list[as-written, underscored], first existing wins; error message names the as-written reference.🤖 Generated with Claude Code