As a PDS Node Operator, I want to validate that products listed as missing from the registry are truly absent from the PDS Search API

[jordanpadams]

Checked for duplicates

Yes - I've already checked

🧑‍🔬 User Persona(s)

PDS Node Operator, PDS Engineering Node

💪 Motivation

...so that I can confirm the accuracy of the missing products reports and identify any false positives where products are listed as missing in the legacy registry but are actually accessible via the PDS Search API.

Relates to #476 and #481.

📖 Additional Details

A spot-check validator script (validate_missing_in_api.py) reads the overall missing_bundles_in_registry.csv and missing_collections_in_registry.csv, randomly samples up to 500 LIDVIDs from the combined pool, queries the PDS Search API for each, and reports HTTP status codes. Products returning 200 are flagged as unexpectedly found in the API.

Acceptance Criteria:

• Script reads missing bundle and collection CSVs and queries PDS Search API endpoint for each LIDVID: /products/{lidvid}
• Random sampling (default 500) with optional fixed seed for reproducibility (--sample, --seed)
• Results written to docs/status/validation_results.csv with columns: node, lidvid, product_class, http_status, in_api
• Summary printed to stdout showing counts of confirmed missing (404), found (200), errors, and timeouts
• Configurable workers, timeout, sample size, and API base URL via CLI args
• --dry-run mode to preview URLs without querying the API
• --sample 0 runs against the full pool
• --found-only flag to write only products unexpectedly found in the API

For Internal Dev Team To Complete

Acceptance Criteria

⚙️ Engineering Details

🎉 I&T