As a data engineer, I want to generate a list of product LIDVIDs tracked in inventory files but missing from the registry

[jordanpadams]

Checked for duplicates

Yes - I've already checked

🧑‍🔬 User Persona(s)

Data Engineer, Node Operator

💪 Motivation

...so that I can identify products that were delivered in inventory files but never ingested into the registry, enabling targeted investigation and remediation of missing data.

📖 Additional Details

Multiple people are independently developing ad-hoc solutions to this problem. A standardized, reusable tool or report would eliminate duplicated effort and provide a consistent method for all nodes.

Known use case: approximately 500 miscellaneous Mars 2020 (M20) products are present in delivered inventory files but absent from the registry. The scale of M20 data (millions of products) makes manual discovery impractical.

Proposed output: a list (CSV or similar) of LIDVIDs that appear in inventory files but cannot be found in the registry index.

---

For Internal Dev Team To Complete

Acceptance Criteria

Given a set of delivered PDS4 inventory files
When I run the missing-products report tool
Then I expect a list of LIDVIDs that are referenced in those inventory files but do not exist in the registry

⚙️ Engineering Details

🎉 I&T

[stein-geo]

This functionality is crucial to maintaining trust in the registry. Such a test should not only be part of data registration but must be run as part of regular data integrity checks.