Merge to main for build17 release (registry v1.7.0) (#540)

* Bump actions/checkout from 5 to 6

Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump actions/cache from 4 to 5

Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@v4...v5)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump actions/upload-artifact from 5 to 6

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 5 to 6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v5...v6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump SonarSource/sonarqube-scan-action from 6 to 7

Bumps [SonarSource/sonarqube-scan-action](https://github.com/sonarsource/sonarqube-scan-action) from 6 to 7.
- [Release notes](https://github.com/sonarsource/sonarqube-scan-action/releases)
- [Commits](SonarSource/sonarqube-scan-action@v6...v7)

---
updated-dependencies:
- dependency-name: SonarSource/sonarqube-scan-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix end of file

* Update requirements

* Update changelog

* Commiting /github/workspace/src/pds/registry/VERSION.txt for stable release

* Setting next dev version to 1.7.0

* switch to dev version of components

* make integration tests work again

* downgrade sweeper because latest stables builds are broken and do not work with the latest version of api

* Update postman_collection.json (#470)

* update registry testing

First, modified .env and docker-compose to allow the user to move from a docker volume to a disk with more space.

Second, added 8 tests for "exists" capability in the registry-api query language

* Update postman_collection.json

Change the way wildcarding is expressed.

* Update postman_collection.json

Put * back in because of too happy to delete.

---------

Co-authored-by: Al Niessner <Al.Niessner@xxx.xxx>
Co-authored-by: thomas loubrieu <thomas.loubrieu@jpl.nasa.gov>

* fix wrong automated test

* Update missing products reports - 2026-01-28 10:08:52

* Update README.md

* Create requirements.txt

* Update README.md

* Update Integration test to account for the refactoring of the ancestry management (#474)

* update properties count test

* disable 2nd-order membership tests as this functionality is deprecated and must be reimplemented or removed at a later date

* update properties count test again- this does not appear to be consistent and may be picking up metadata keys

* use latest verion of sweeper after successful integration tests

* Add AOSS vs. Managed OpenSearch Trade Study

Refs #451

* Fix bug in requirements.txt

* Update missing products reports - 2026-03-03 15:55:00

* Update missing products reports - 2026-03-04 13:55:14

* Update queries to include up to 10000 records

Resolves #471

* Update missing products reports - 2026-03-04 13:57:11

* Bump actions/upload-artifact from 6 to 7

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 6 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v6...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update missing products reports - 2026-03-18 12:06:51

* Update missing products reports - 2026-03-18 12:07:22

* Update missing products reports - 2026-03-18 12:13:30

* Update missing products reports - 2026-03-18 13:06:49

* Split missing products reports by latest/superseded version and add burndown tracking

- generate_registry_status_reports.py: for missing bundles/collections, produce
  three CSVs per type (overall, *_latest_*, *_superseded_*) by grouping LIDVIDs
  by LID and comparing versions numerically so 3.9 < 3.13. Resolves
  pds-registry-client via sys.executable rather than shell PATH to avoid venv
  activation issues. Appends one row per run to docs/status/counts_history.csv
  for burndown chart tracking (append-only, never overwritten).
- backfill_history.py: new one-off script to populate counts_history.csv from
  git history of the status CSVs; idempotent (skips dates already present).
- docs/status/counts_history.csv: initial 5-snapshot backfill from git history
  (2025-11-21 through 2026-03-18).
- docs/status/README.md: updated metrics table to show Latest/Superseded/Total
  columns; added Historical Counts section documenting the burndown CSV format.
- .gitignore: ignore *.code-workspace files.
- CLAUDE.md: document the status reporting scripts and operational notes.

Closes #481. Relates to #476.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Add API validator script to spot-check missing products

validate_missing_in_api.py reads the overall missing bundle and collection
CSVs, randomly samples up to 500 LIDVIDs (configurable via --sample/--seed),
queries https://pds.nasa.gov/api/search/1/products/{lidvid} for each, and
reports HTTP status codes. Products returning 200 are flagged as unexpectedly
present in the API.

Features:
- Concurrent requests via ThreadPoolExecutor (--workers, default 10)
- Retry logic for transient errors (429, 5xx)
- Results written to docs/status/validation_results.csv
- --dry-run, --found-only, --sample 0 (full pool) modes
- Fixed seed support for reproducible sampling

Closes #484. Relates to #476.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update scripts/generate_registry_status_reports.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Initial plan

* Initial plan

* Initial plan

* Tighten type hints in split_by_version() in backfill_history.py

Co-authored-by: jordanpadams <33492486+jordanpadams@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NASA-PDS/registry/sessions/ab42f1c3-5d84-40a5-b8c0-c00a2938c827

* Initial plan

* Use --detach instead of -d in docker compose command in CLAUDE.md

Co-authored-by: jordanpadams <33492486+jordanpadams@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NASA-PDS/registry/sessions/3ad3b701-4ef9-4645-af27-3b698844b1e7

* Compute CSV counts once and reuse in both generate_metrics_from_csvs and append_history_row

Co-authored-by: jordanpadams <33492486+jordanpadams@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NASA-PDS/registry/sessions/00d36611-6202-4258-9b9c-e5fa636f42da

* Fix --volume to --volumes in CLAUDE.md docker compose command

Co-authored-by: jordanpadams <33492486+jordanpadams@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NASA-PDS/registry/sessions/42ce8b7b-f5ac-4e73-b3a6-b5f5fe6d9dc4

* Consolidate missing products CSVs into single file with superseded column

Instead of generating three separate CSVs per product type (overall, latest,
superseded), generate one CSV with a `superseded` column (true/false) indicating
whether each LIDVID is the latest version for its LID.  History counts and README
metrics table are unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update missing products reports - 2026-03-23 18:57:33

* Remove CSVs we are no longer generating

* Update missing products reports - 2026-03-24 11:40:39

* Update missing products reports - 2026-03-24 12:06:47

* Update missing products reports - 2026-03-24 12:45:49

* Update missing products reports - 2026-03-24 13:54:53

* Update missing products reports - 2026-03-24 14:41:16

* Fix superseded version detection for missing bundles and collections

Queries now fetch all products (not just found_in_registry=false) so that
version ordering is determined across the full set. Python filters to missing
rows after annotating superseded status, ensuring a missing LIDVID is correctly
marked superseded when a higher version exists in the registry.

Also adds CSV headers to output files and skips header rows in count aggregation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* skip broken tests

* skip broken tests

* down to a minor mismatch

* minimal skips

* Improve registry documentation for new users (#392, #391, #393, #394) (#502)

This commit addresses multiple documentation issues to better support new PDS users:

**Issue #391 - Add links to component documentation:**
- Added link to registry-client external docs from Registry Client section
- Added links from Registry Manager to update_status, delete_data, and create_reg pages
- Added link from Harvest section to load1.html user guide

**Issue #393 - Improve documentation consistency and ordering:**
- Reordered components: Harvest → Registry Manager → Registry Client → OpenSearch → Registry API
- Made Authentication/Authorization a subsection of OpenSearch
- Renamed "API" to "Registry API (PDS Search API)" for clarity and consistency
- Added note about archive status affecting API visibility
- Clarified archive status values with descriptions:
  * archived - publicly visible
  * certified - publicly visible
  * restricted - not publicly visible
  * staged - default, not publicly available
- Fixed inconsistent terminology: changed "uploaded" to "loaded" in load1.rst

**Issue #394 - Update registry-manager documentation:**
- Updated parameter documentation to reflect current CLI (2 required, 2 optional)
- Replaced deprecated -es with -registry parameter
- Updated examples to use new parameter names
- Fixed "registered" to "certified" in Registry API section

These changes improve discoverability and clarity for users new to the PDS Registry system.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* Add auto-generated human-readable Postman collection docs (#501)

* Add human-readable Postman collection docs with auto-generation workflow

- Add generate_collection_docs.py to convert postman_collection.json to
  Markdown with TOC, linked TestRail case IDs, and GitHub issue refs
- Add generated postman_collection.md alongside the JSON source
- Add GitHub Actions workflow to regenerate the doc on every push that
  touches postman_collection.json (resolves #497)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Exclude postman_collection.md from end-of-file-fixer hook

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Restrict postman docs workflow to feature branches only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Replace third-party commit action with git CLI commands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address Copilot review feedback on postman doc generator

- Fix TOC anchor mismatch: anchors now include HTTP method to match rendered headings
- Add anchor deduplication for repeated request names
- Remove unused extract_testrail_ids() function
- Add encoding="utf-8" to open() and write_text() for deterministic output
- Remove [skip ci] from workflow commit (no loop risk; trigger is .json not .md)
- Regenerate postman_collection.md with corrected TOC anchors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: thomas loubrieu <60993872+tloubrieu-jpl@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* update exists operator tests as NASA-PDS/registry-api#741

* update test for updated members/members and member-of/member-of support

* add missing host for elasticsearch service for docker compose to work with option --project-name

* simplify the api launch

* fix docker compose for github action support

* use --project-name value in the docker compose through a new env variable

* add missing numbering suffix

* add healthcheck on api and use it as a dependency for other services to start.

* simplify docker compose, adjust test or latest PR of registry-api #753

* change test expectation for GONE members/members end-point

* add check for good error message

* docs: regenerate postman_collection.md

* non existance has changed

* docs: regenerate postman_collection.md

* fixup last test failure

* docs: regenerate postman_collection.md

* Update missing products reports - 2026-04-27 14:15:27

* Bump SonarSource/sonarqube-scan-action from 7 to 8

Bumps [SonarSource/sonarqube-scan-action](https://github.com/sonarsource/sonarqube-scan-action) from 7 to 8.
- [Release notes](https://github.com/sonarsource/sonarqube-scan-action/releases)
- [Commits](SonarSource/sonarqube-scan-action@v7...v8)

---
updated-dependencies:
- dependency-name: SonarSource/sonarqube-scan-action
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Initial import of terraform for provisioned opensearch

* Correct committed versions of main and variable tf files.

* Initial commit of index template tf scripts.

* Fix opensearch provider.

* Add pipeline terraform.

* Make main provider.tf consistent.

* Make index_templates provider.tf consistent.

* Intermediate commit (to preserve content) of OSI pipeline tf scripts.

* Switched strategy for osi pipeline config body - went to an external yaml file.

* Fixed specification of policy file variable.

* Update missing products reports - 2026-05-11 13:35:12

* Revise pipeline terraform scripts.

* Add burnup charts, loaded product reports, and per-node tracking

- Add burnup_chart.html with interactive time-series visualization
- Add burnup_history.csv, burnup_by_node.csv and their *_latest variants
- Add loaded_bundles/collections CSVs and per-node OpenSearch DSL queries
- Update generate_registry_status_reports.py to produce all new reports
- Update docs/status/README.md to document new outputs

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

* Add workflow to publish burnup chart to GitHub Pages

- New publish-burnup-chart.yml triggers on push to main when
  docs/status/burnup_chart.html changes, copying it to gh-pages/status/
- Add docs/source/status.rst with link to the live chart
- Wire status.rst into the Sphinx toctree in index.rst

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

* Add burnup chart publishing to GitHub Pages

- Fix publish-burnup-chart workflow: save burnup_chart.html to /tmp before
  checking out gh-pages branch so the file survives the workspace replacement
- Add docs/source/status.rst linking to the live chart
- Wire status.rst into the Sphinx toctree
- Add docs/status/burnup_chart.html to main so the workflow can trigger

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

* Update publish-burnup-chart.yml

* Update CLAUDE.md and docs/status/README with loaded product and burnup chart documentation

- Expand Registry Status Reporting in CLAUDE.md: correct venv invocation,
  table of all six query config files with endpoints, data population
  distinction (legacy Solr vs new OpenSearch / PSA inflation), AOSS
  pagination caveat, full output file table, burnup chart details
- Add "How Numbers Are Calculated" section to docs/status/README.md
  explaining missing/loaded/staged/burnup derivation and the superseded
  flag algorithm
- Add Loaded Products and Burnup Charts report sections to README
- Update download script with new loaded_* and burnup_* files

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

* Update metrics

* Fix publish workflow to checkout triggering branch instead of hardcoded main

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

* Improve user documentation with procedural step-by-step instructions

Rewrites connection-setup, installation, and all user task docs to use
numbered steps throughout. Adds test_connection page, standardizes config
file naming to registry-{tool}-config-{node}-{venue} pattern under ~/.pds/,
and fixes autosectionlabel warnings in conf.py.

Resolves #515

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

* Switch over to full block for index template settings to provide more flexibility

* Fix typo for pipeline role arn.

* Add index alias support.

* Update missing products reports - 2026-06-09 11:21:52

* Expand registry loading status page with burnup chart and report links

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Scale x-axis tick density to selected date range on burnup chart

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix burnup chart x-axis to use calendar time scale independent of data

Switch from category scale to Chart.js time scale with chartjs-adapter-date-fns.
The axis now spans the full selected date range with evenly-spaced calendar
ticks (weekly for ≤6mo, monthly for ≤2yr, yearly for all time) regardless
of where data points happen to fall.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update missing products reports - 2026-06-11 08:05:34

* Update changelog

* Update reports

* Update changelog

* add test for security patch, update docker compose for latest sweeper docker upgrade

* docs: regenerate postman_collection.md

* update test after sweeper upgrade

* add explicit pull_policy for registry-api to make sure the local image is used when available

* Fix race condition: add depends_on between domain policy and domain

Without this, Terraform may attempt to create the domain access policy
before the OpenSearch domain exists, causing a ResourceNotFoundException.

* Bump actions/checkout from 6 to 7

Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump actions/cache from 5 to 6

Bumps [actions/cache](https://github.com/actions/cache) from 5 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@v5...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add script to detect and delete stale LDD_Info sentinel records

Adds scripts/detect_stale_ldd_sentinels.py which queries all node
registry -dd indexes for LDD_Info sentinels that have zero field
documents, reports them with ready-to-paste Dev Tools delete queries,
and optionally deletes them with explicit confirmation.

Key features:
- Paginates sentinel fetch with search_after so all records are found
- Checkpoints progress after each _count query; interrupted runs resume
- Retries transient connection errors with exponential backoff
- --delete flag required for any writes; prompts for 'yes' confirmation
- --report / --reset / --checkpoint flags for operational control

Fixes: NASA-PDS/registry-loader#89

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Re-enable secrets detection using standard detect-secrets

Replaces the disabled slim-detect-secrets GitHub Action and commented-out
pre-commit hook with the updated approach from cloud-tools:
- New `scripts/detect_secrets_baseline.sh` with scan/audit/check modes,
  `.detect-secrets-ignore` support, and Python-based comparison (no jq)
- Workflow delegates to the script instead of inlining logic
- `.detect-secrets-ignore` excludes docker/default-config, docs, test data
- `.secrets.baseline` regenerated with detect-secrets~=1.5.0; all 3 findings
  audited as false positives (Jenkins credential ref, test placeholder, md5 URL param)
- Pre-commit hook re-enabled pointing to the new script

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix --reset flag skipping sentinel fetch

load_checkpoint returned {"sentinels": [], "results": {}} as the default
when no file exists, so after --reset deleted the checkpoint the reload
saw "sentinels" was present and skipped the fetch entirely, printing
"Resuming from checkpoint — 0 sentinel(s) already fetched" / "Nothing to do."

Fix: return {"results": {}} so a fresh/reset checkpoint has no "sentinels"
key and the fetch is always triggered.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix status report showing false 'not yet checked' warning

print_status_report compared len(results) against len(sentinels) (raw hit
count), but results deduplicates by (index, namespace) key. With 420 raw
hits but only 164 unique pairs, the report falsely reported 256 unchecked.

Fix: compute total from unique (index, ns) keys in the sentinels list so
the denominator matches what results actually tracks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update secrets and dependencies

* Fix linter issues

* Update .gitignore

* Add terraform script for complete deployment of the Registry in the Dev Account (#475)

* initialize terraform to deploy simple regitry in dev, wip

* wip: terraform deployment of opensearch serverless collection

* make the depoyment work with external definition of IAM role in pds-cds-infra repository

* adjust the role/policy management without using SSM for now

* upgrade provider for compatibility purpose

* modularize terraform, to make external policies stand out

* add missing module directory

* clean role argument names

* rename opensearch api and dashboard access policy

* wip: untested credentials endpoint for registry-tools in terraform

* automate registry initialization with registry-mgr/harvest

* add registry-manager set-archive-status command to match reference test setup

* add usage documentation in readme.

* Update missing products reports - 2026-03-18 10:01:42

* Update missing products reports - 2026-03-18 12:04:37

* add missing api gateway and first version of automated initialization with test data on AWS

* add creation of the build directory

* validate terraform apply/destroy

* add build layer directory for plan phase.

* use pip3 instead of pip which might not be available locally.

* wip: registry deployed in dev with api and sweeper manually added from their own terraform, integration test still fail.

* remove automated lias creation as it does not work yet

* add comment for commented out code, specifically dedicated to Sean!

* update to initialize node registry without ref data.

* update to have log when the lambda layer is build as it does not work from terragrunt call

* try to clarify what path module is

* force to recreate the layer whenever the zip file has to be redeployed

* minor fixes, logs

* add sleep to make sure the harvest test products are properly indexed

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix COGNITO_ALLOWED_GROUPS substring matching by splitting into a set

* Strip whitespace from COGNITO_ALLOWED_GROUPS entries after splitting

* force delete registry indexes before deployment, make vpc endpoint service name unique to handle multiple deployment in the same account

* update sweeper call arguments per later update on sweeper docker image

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* update docker compose to make the test wait for api to be ready

* rename opensearch module as opensearch_serverless module to avoid conflicts

* make it compliant with python 3.12

* rename opensearch directory into opensearch_managed for consistency with opensearch_serverless

* fix renamed vpc endpoint service

* add the vpc endpoint with name depending on the opensearch collection name

* finally remove the collection service because it cannot work

* add missing relay variable

* Linter updates

* fix obsolete README instruction

---------

Co-authored-by: thomas loubrieu <thomas.loubrieu@jpl.nasa.gov>
Co-authored-by: Thomas Loubrieu <loubrieu@jpl.nasa.gov>
Co-authored-by: Jordan Padams <33492486+jordanpadams@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Jordan Padams <jordan.h.padams@jpl.nasa.gov>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* Bump python-jose from 3.3.0 to 3.4.0 in /terraform/lambda/src/layer

Bumps [python-jose](https://github.com/mpdavis/python-jose) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/mpdavis/python-jose/releases)
- [Changelog](https://github.com/mpdavis/python-jose/blob/master/CHANGELOG.md)
- [Commits](mpdavis/python-jose@3.3.0...3.4.0)

---
updated-dependencies:
- dependency-name: python-jose
  dependency-version: 3.4.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump requests from 2.32.3 to 2.33.0 in /terraform/lambda/src/layer

Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.33.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](psf/requests@v2.32.3...v2.33.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.33.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update dependabot.yml

* Update dependabot.yml

* Update unstable-cicd.yaml

* Update branch-cicd.yaml

* Update missing products reports - 2026-07-13 13:56:58

* Fix linter issues

* Bump actions/checkout in the github-actions group across 1 directory

Bumps the github-actions group with 1 update in the / directory: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 4 to 7
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v4...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update requirements

* Update changelog

* Update CLAUDE.md to require pre-commit install or tox before pushing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update requirements

* Update changelog

* update sweepers command to reflect ENTRYPOINT directive introduced in 2b83e392

* Update requirements

* Update changelog

* Add % completion by node charts and flip commit default to --no-commit

- Add "% Completion by Node" section to burnup_chart.html with four
  normalized line charts (all-versions + latest-only, bundles +
  collections). Each node is plotted as 0–100% of its own target so
  small nodes are directly comparable to large ones.
- Change generate_registry_status_reports.py default from auto-commit
  to --no-commit; --commit must now be passed explicitly to push.
- Update CLAUDE.md to reflect the new default.
- Regenerate docs/status/ CSVs and HTML with latest data.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update requirements

* Update changelog

* Add % completion by node charts and flip commit default to --no-commit

- Add "% Completion by Node" section to burnup_chart.html with four
  normalized line charts (all-versions + latest-only, bundles +
  collections). Each node is plotted as 0–100% of its own target so
  small nodes are directly comparable to large ones.
- Change generate_registry_status_reports.py default from auto-commit
  to --no-commit; --commit must now be passed explicitly to push.
- Update CLAUDE.md to reflect the new default.
- Regenerate docs/status/ CSVs and HTML with latest data.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update requirements

* Update changelog

* Update requirements

* Update changelog

* Remove useless script in docker compose setup (#539)

* don't use the local script to launch the API, this is useless and does not match the AWS deployment setup.

* trigger local precommit check

* migrate precommit configuration

* add other missing closing double quote

---------

Co-authored-by: Thomas Loubrieu <loubrieu@jpl.nasa.gov>

* Update requirements

* Update changelog

* Add user-facing documentation for registry status reports

Explains PDS Keyword Search as the source of truth baseline, the
comparison to the Registry API, and adds an FAQ covering node filtering,
superseded flag interpretation, and how to request a data release for
unreleased bundle/collection versions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update requirements

* Update changelog

* Update requirements

* Update changelog

* Split terraform scripts according to lifecycle of the components (#550)

* split terraform scripts according to lifecycle of the components, validated in dev, without terragrunt

* split terraform scripts according to lifecycle of the components, validated in dev, without terragrunt

---------

Co-authored-by: Thomas Loubrieu <loubrieu@jpl.nasa.gov>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: thomas loubrieu <thomas.loubrieu@jpl.nasa.gov>
Co-authored-by: PDSEN CI Bot <pdsen-ci@jpl.nasa.gov>
Co-authored-by: al-niessner <1130658+al-niessner@users.noreply.github.com>
Co-authored-by: Al Niessner <Al.Niessner@xxx.xxx>
Co-authored-by: Jordan Padams <jordan.h.padams@jpl.nasa.gov>
Co-authored-by: Jordan Padams <33492486+jordanpadams@users.noreply.github.com>
Co-authored-by: Alex Dunn <75815303+alexdunnjpl@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Thomas Loubrieu <loubrieu@jpl.nasa.gov>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: jimmie <jimmie.d.young@gmail.com>
Co-authored-by: Rishi Verma <riverma@apache.org>
Co-authored-by: edunn <alexander.e.dunn@jpl.nasa.gov>

Author: 15 people

.detect-secrets-ignore

@@ -0,0 +1,16 @@
+# Per-repo secret detection exclusions for registry.
+# One regex per line; lines starting with # are comments.
+# These supplement the global exclusions in scripts/detect_secrets_baseline.sh.
+
+# Test fixture directories that may contain sample/fake credentials
+.*/test/data/.*
+.*/tests/data/.*
+
+# Documentation files contain placeholder/example credentials only
+docs/.*\.rst
+docs/.*\.md
+README\.md
+CHANGELOG\.md
+
+# Docker default configs use well-known placeholder passwords (noted in CLAUDE.md)
+docker/default-config/.*

.github/dependabot.yml

@@ -9,12 +9,20 @@ updates:
     directory: "/" # Location of package manifests
     schedule:
       interval: "weekly"
+    groups:
+      github-actions:
+        patterns:
+          - "*"
+    target-branch: "develop"
+
   - package-ecosystem: "docker" # See documentation for possible values
     directory: "/docker/" # Location of package manifests
     schedule:
       interval: "weekly"
+    target-branch: "develop"
 
   - package-ecosystem: "pip" # See documentation for possible values
     directory: "/" # Location of package manifests
     schedule:
       interval: "weekly"
+    target-branch: "develop"

.github/workflows/automated-testing.yaml

@@ -14,7 +14,7 @@ jobs:
         steps:
             -
                 name: Checkout Repository
-                uses: actions/checkout@v5
+                uses: actions/checkout@v7
             -
                 name: ☕️ Set up OpenJDK
                 uses: actions/setup-java@v5

.github/workflows/branch-cicd.yaml

@@ -17,6 +17,7 @@ on:
         branches:
             -   '**'
             -   '!main'
+            -   '!develop'
 
 
 # What to Do
@@ -33,7 +34,7 @@ jobs:
         steps:
             -
                 name: 💳 Checkout
-                uses: actions/checkout@v5
+                uses: actions/checkout@v7
                 with:
                     lfs: true
                     fetch-depth: 0
@@ -45,7 +46,7 @@ jobs:
                     python-version: '3.13'
             -
                 name: 💵 Python Cache
-                uses: actions/cache@v4
+                uses: actions/cache@v6
                 with:
                     path: ~/.cache/pip
                     # The "key" used to indicate a set of cached files is the operating system runner

.github/workflows/codeql-analysis.yml

@@ -23,7 +23,7 @@ jobs:
 
     steps:
     - name: Checkout repository
-      uses: actions/checkout@v5
+      uses: actions/checkout@v7
       with:
         # We must fetch at least the immediate parents so that if this is
         # a pull request then we can checkout the head.
@@ -73,7 +73,7 @@ jobs:
 
 
     - name: Upload CodeQL Artifacts
-      uses: actions/upload-artifact@v5
+      uses: actions/upload-artifact@v7
       with:
         name: codeql-artifacts
         path: ${{ env.RESULTS_DIR }}

.github/workflows/postman-collection-docs.yml

@@ -0,0 +1,33 @@
+name: Generate Postman Collection Docs
+
+on:
+  push:
+    branches-ignore:
+      - main
+      - develop
+    paths:
+      - 'docker/postman/postman_collection.json'
+  workflow_dispatch:
+
+jobs:
+  generate-docs:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v7
+
+      - uses: actions/setup-python@v6
+        with:
+          python-version: '3.13'
+
+      - name: Generate Markdown from Postman collection
+        run: python docker/postman/generate_collection_docs.py
+
+      - name: Commit generated docs
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+          git add docker/postman/postman_collection.md
+          git diff --cached --quiet || git commit -m "docs: regenerate postman_collection.md"
+          git push

.github/workflows/publish-burnup-chart.yml

@@ -4,6 +4,7 @@ on:
   push:
     branches:
       - main
+      - develop
     paths:
       - 'docs/status/burnup_chart.html'
   workflow_dispatch:
@@ -15,16 +16,16 @@ jobs:
     permissions:
       contents: write
     steps:
-      - name: Checkout main
-        uses: actions/checkout@v6
+      - name: Checkout triggering branch
+        uses: actions/checkout@v7
         with:
-          ref: main
+          ref: ${{ github.ref }}
 
       - name: Save burnup chart before workspace is replaced
         run: cp docs/status/burnup_chart.html /tmp/burnup_chart.html
 
       - name: Checkout gh-pages
-        uses: actions/checkout@v6
+        uses: actions/checkout@v7
         with:
           ref: gh-pages
           path: gh-pages

.github/workflows/secrets-detection.off

@@ -0,0 +1,36 @@
+name: Secret Detection Workflow
+on:
+    push:
+        branches:
+            - main
+    pull_request:
+        branches:
+            - main
+
+jobs:
+    secret-detection:
+        runs-on: ubuntu-latest
+        steps:
+            -
+                name: Checkout code
+                uses: actions/checkout@v7
+            -
+                name: Install detect-secrets
+                run: pip install detect-secrets~=1.5.0
+
+            -
+                name: Verify .secrets.baseline exists
+                run: |
+                    if [ ! -f .secrets.baseline ]; then
+                        echo "❌ No .secrets.baseline found." >&2
+                        echo "" >&2
+                        echo "Run the following locally, audit the results, then commit the baseline:" >&2
+                        echo "  scripts/detect_secrets_baseline.sh scan" >&2
+                        echo "  scripts/detect_secrets_baseline.sh audit" >&2
+                        exit 1
+                    fi
+                    echo "✅ .secrets.baseline found."
+
+            -
+                name: Scan repository for secrets
+                run: scripts/detect_secrets_baseline.sh

.github/workflows/secrets-detection.yaml

@@ -48,14 +48,14 @@ jobs:
         steps:
             -
                 name: 💳 Checkout
-                uses: actions/checkout@v5
+                uses: actions/checkout@v7
                 with:
                     lfs: true
                     token: ${{secrets.ADMIN_GITHUB_TOKEN}}
                     fetch-depth: 0
             -
                 name: 💵 Python Cache
-                uses: actions/cache@v4
+                uses: actions/cache@v6
                 with:
                     path: ~/.cache/pip
                     # The "key" used to indicate a set of cached files is the operating system runner