feat: Do not filter tracked files even if they match gitignore ...

[PeterSchafer]

This PR changes the behaviour of the file filter utility to not plainly filter files if they match patterns in gitignore, .snyk and dcignore, but to also consider if a file is already tracked in git. The change will make files be not filtered if they are already tracked.

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues
✅	Code Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues
✅	Code Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[PeterSchafer]

/review

[PeterSchafer]

/review

[snyk-pr-review-bot]

PR Reviewer Guide 🔍

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Path Resolution Mismatch 🟠 [major] In GetFilteredFiles, filepath.Abs(f) resolves the input file path relative to the process's current working directory (CWD). However, in getGitTrackedFiles, the map keys are built using filepath.Abs(filepath.Join(repoRoot, entry.Name)). If the tool is executed from a subdirectory of the repository and provided with relative paths, filepath.Abs(f) will result in a different string than the one stored in the gitTrackedFiles map, causing the 'tracked file' check to fail and files to be incorrectly filtered. absPath, err := filepath.Abs(f) if err != nil { absPath = filepath.Clean(f) } // files tracked in git should not be filtered, even if they match gitignore patterns if gitTrackedFiles[absPath] { High Memory Overhead 🟡 [minor] The getGitTrackedFiles function constructs a map[string]bool containing the absolute paths of every single tracked file in the repository. In large monorepos with hundreds of thousands of files, this will cause significant memory consumption and a latency spike at the start of every filtering operation, as the entire git index must be iterated and string-concatenated into the map. func (fw *FileFilter) getGitTrackedFiles() map[string]bool { trackedFiles := make(map[string]bool) // open the git repository repo, err := git.PlainOpenWithOptions(fw.path, &git.PlainOpenOptions{ DetectDotGit: true, }) if err != nil { fw.logger.Debug().Msgf("failed to open git repository: %v", err) return trackedFiles } // get the worktree to find the root path and access the index worktree, err := repo.Worktree() if err != nil { fw.logger.Debug().Msgf("failed to get worktree: %v", err) return trackedFiles } repoRoot := worktree.Filesystem.Root() // get the index (staging area) - this contains all tracked files // A file is tracked in git once it's added to the index, even before commit idx, err := repo.Storer.Index() if err != nil { fw.logger.Debug().Msgf("failed to get git index: %v", err) return trackedFiles } // iterate through all entries in the index for _, entry := range idx.Entries { absolutePath := filepath.Join(repoRoot, entry.Name) // ensure the path is absolute and cleaned for consistent comparison absolutePath, err = filepath.Abs(absolutePath) if err != nil { absolutePath = filepath.Clean(filepath.Join(repoRoot, entry.Name)) } trackedFiles[absolutePath] = true }

📚 Repository Context Analyzed This review considered 9 relevant code sections from 9 files (average relevance: 0.79) pkg/utils/file_filter.go (lines 1-470) internal/api/fileupload/uploadrevision/errors.go (lines 1-174) internal/mocks/os.go (lines 1-93) pkg/utils/array.go (lines 1-207) pkg/apiclients/fileupload/batch.go (lines 1-133) pkg/apiclients/fileupload/client.go (lines 1-173) internal/utils/os.go (lines 1-36) ... and 2 more file(s)