Add user-facing documentation around fingerprinting snippet-scans (#1576)

* Add user-facing documentation around fingerprinting snippet-scans, add a proper `analyze --help` message around `x-snippet-scan`:
```
--x-snippet-scan
    Experimental flag to enable snippet scanning to identify open source code snippets using fingerprinting.
```

* Provied ficus sensible default endpoint

* Add default endpoint to pass to ficus

Author: james-fossa

Changelog.md

@@ -1,5 +1,9 @@
 # FOSSA CLI Changelog
 
+## 3.11.1
+
+- Better document `--x-snippet-scan`.
+
 ## 3.11.0
 
 - Add a dependency on Ficus, a new internal tool.

docs/references/subcommands/analyze.md

@@ -142,6 +142,80 @@ In addition to the [standard flags](#specifying-fossa-project-details), the anal
 | `--strict`                                                                        | Enforces strict analysis to ensure the most accurate results by rejecting fallbacks. When run with `--static-only-analysis`, the most optimal static strategy will be applied without fallbacks. |
 
 
+### Snippet Scanning
+
+Snippet scanning identifies potential open source code snippets within your first-party source code by comparing file fingerprints against FOSSA's knowledge base. This feature helps detect code that may have been copied from open source projects.
+
+#### Enabling Snippet Scanning
+
+| Name                | Description                                                                                                                                                                           |
+|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `--x-snippet-scan`  | Enable snippet scanning during analysis. This experimental feature fingerprints your source files and checks them against FOSSA's snippet database via ScanOSS integration.        |
+
+#### How Snippet Scanning Works
+
+When `--x-snippet-scan` is enabled, the CLI:
+
+1. **Hashes Files First**: Creates CRC64 hashes of all source files to identify which files need fingerprinting
+2. **Checks Necessity of Fingerprinting**: Checks with FOSSA servers to determine which file hashes are already known
+3. **Fingerprints New or Changed Files**: Uses the Ficus fingerprinting engine (written in Rust) to create cryptographic fingerprints only for files not previously seen
+4. **Filters Content**: By default, skips directories like `.git/`, and hidden directories. This includes, from `.fossa.yml`, `vendoredDependencies.licenseScanPathFilters.exclude`, documented further below.
+5. **Uploads Fingerprints**: Sends only the fingerprints to FOSSA's servers via ScanOSS integration
+6. **Receives Matches**: Gets back information about any matching open source components
+7. **Uploads Match Contents**: For files that have matches, uploads source code content temporarily to FOSSA servers.
+
+#### Data Sent to FOSSA
+
+**For Performance Optimization:**
+- CRC64 hashes of all files, to avoid re-fingerprinting unchanged files.
+
+**For Fingerprinting:**
+- ScanOSS-compatible fingerprints of source code to identify matches.
+
+**For Matched Files Only:**
+- The actual source code content of files that contain snippet matches.
+
+#### Data Retention
+
+- **File Fingerprints**: Stored permanently for caching and performance optimization
+- **Source Code Content**: Stored temporarily for 30 days and then automatically deleted
+- **CRC64 Hashes**: The likelihood of a collision with CRC64 (2^64 possible values) is extremely low.
+
+#### Directory Filtering
+
+By default, snippet scanning excludes common non-production directories and follows `.gitignore` patterns:
+
+- Hidden directories.
+- Globs as directed by `.gitignore` files.
+
+#### Custom Exclude Filtering
+
+You can customize which files and directories are excluded from snippet scanning by configuring exclude filters in your `.fossa.yml` file. Note that snippet scanning currently only supports exclude patterns, not `only` patterns.
+
+For example:
+```yaml
+version: 3
+vendoredDependencies:
+  licenseScanPathFilters:
+    exclude:
+      - "**/test/**"
+      - "**/tests/**"
+      - "**/spec/**"
+      - "**/node_modules/**"
+      - "**/dist/**"
+      - "**/build/**"
+      - "**/*.test.js"
+      - "**/*.spec.ts"
+```
+
+**Important Notes:**
+
+- Snippet scanning only uses the `exclude` filters from `licenseScanPathFilters` - `only` filters are ignored for this use-case.
+- Path filters use standard glob patterns (e.g., `**/*` for recursive matching, `*` for single-directory matching).
+- The configuration goes in the `vendoredDependencies.licenseScanPathFilters.exclude` section.
+- These exclude patterns are passed directly to the Ficus fingerprinting engine as `--exclude` arguments.
+- Default exclusions (hidden files, `.gitignore` patterns) are applied in addition to custom excludes.
+
 ### Experimental Options
 
 _Important: For support and other general information, refer to the [experimental options overview](../experimental/README.md) before using experimental options._

src/App/Fossa/Config/Analyze.hs

@@ -114,7 +114,6 @@ import Options.Applicative (
   InfoMod,
   Parser,
   eitherReader,
-  help,
   helpDoc,
   hidden,
   long,
@@ -345,7 +344,7 @@ cliParser =
     <*> flagOpt StaticOnlyTactics (applyFossaStyle <> long "static-only-analysis" <> stringToHelpDoc "Only analyze the project using static strategies.")
     <*> withoutDefaultFilterParser fossaAnalyzeDefaultFilterDocUrl
     <*> flagOpt StrictMode (applyFossaStyle <> long "strict" <> stringToHelpDoc "Enforces strict analysis to ensure the most accurate results by rejecting fallbacks.")
-    <*> switch (applyFossaStyle <> long "x-snippet-scan" <> help "Enable ficus snippet scanning")
+    <*> switch (applyFossaStyle <> long "x-snippet-scan" <> stringToHelpDoc "Experimental flag to enable snippet scanning to identify open source code snippets using fingerprinting.")
   where
     fossaDepsFileHelp :: Maybe (Doc AnsiStyle)
     fossaDepsFileHelp =

src/App/Fossa/Ficus/Analyze.hs

@@ -44,9 +44,9 @@ import Fossa.API.Types (ApiKey (..), ApiOpts (..))
 import Path (Abs, Dir, File, Path, toFilePath)
 import Srclib.Types (Locator (..), renderLocator)
 import Text.URI (render)
-import Text.URI.Builder ()
+import Text.URI.Builder (PathComponent (PathComponent), TrailingSlash (TrailingSlash), setPath)
 import Types (GlobFilter (..), LicenseScanPathFilters (..))
-import Prelude hiding (unwords)
+import Prelude
 
 newtype CustomLicensePath = CustomLicensePath {unCustomLicensePath :: Text}
   deriving (Eq, Ord, Show, Hashable)
@@ -171,8 +171,9 @@ ficusCommand :: Has Diagnostics sig m => FicusConfig -> BinaryPaths -> m Command
 ficusCommand ficusConfig bin = do
   endpoint <- case ficusConfigEndpoint ficusConfig of
     Just baseUri -> do
-      pure $ render baseUri
-    Nothing -> pure ""
+      proxyUri <- setPath [PathComponent "api", PathComponent "proxy", PathComponent "analysis"] (TrailingSlash False) baseUri
+      pure $ render proxyUri
+    Nothing -> pure "https://app.fossa.com/api/proxy/analysis"
   pure $
     Command
       { cmdName = toText $ toPath bin