Merge pull request #511 from NASA-PDS/feature/burnup-charts

Add burnup charts and per-node tracking to registry status reports

Author: jordanpadams

.github/workflows/publish-burnup-chart.yml

@@ -0,0 +1,41 @@
+name: Publish Burnup Chart to GitHub Pages
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - 'docs/status/burnup_chart.html'
+  workflow_dispatch:
+
+jobs:
+  publish:
+    name: Copy burnup chart to gh-pages
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - name: Checkout main
+        uses: actions/checkout@v6
+        with:
+          ref: main
+
+      - name: Checkout gh-pages
+        uses: actions/checkout@v6
+        with:
+          ref: gh-pages
+          path: gh-pages
+
+      - name: Copy burnup chart
+        run: |
+          mkdir -p gh-pages/status
+          cp docs/status/burnup_chart.html gh-pages/status/burnup_chart.html
+
+      - name: Commit and push to gh-pages
+        run: |
+          cd gh-pages
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+          git add status/burnup_chart.html
+          git diff --cached --quiet || git commit -m "docs: update burnup chart"
+          git push

conf/status/loaded_bundles_per_node.json

@@ -0,0 +1,17 @@
+
+{
+    "_source": ["ops:Harvest_Info/ops:node_name", "lidvid", "product_class", "ops:Harvest_Info/ops:harvest_date_time"],
+  "size": 10000,
+  "sort": [{"_id": "asc"}],
+  "query": {
+    "bool": {
+      "filter": [
+        {
+          "match_phrase": {
+            "product_class": "Product_Bundle"
+          }
+        }
+      ]
+    }
+  }
+}

conf/status/loaded_collections_per_node.json

@@ -0,0 +1,17 @@
+
+{
+    "_source": ["ops:Harvest_Info/ops:node_name", "lidvid", "product_class", "ops:Harvest_Info/ops:harvest_date_time"],
+  "size": 10000,
+  "sort": [{"_id": "asc"}],
+  "query": {
+    "bool": {
+      "filter": [
+        {
+          "match_phrase": {
+            "product_class": "Product_Collection"
+          }
+        }
+      ]
+    }
+  }
+}

docs/source/index.rst

@@ -25,3 +25,4 @@ Sitemap
    admin/tasks
    developer/developer
    support/support
+   status

docs/source/status.rst

@@ -0,0 +1,18 @@
+Registry Loading Status
+=======================
+
+The registry loading status reports track the progress of data ingestion across PDS nodes.
+Reports are automatically generated on a schedule and committed to the repository.
+
+Loading Progress
+----------------
+
+The `Burnup Chart <https://nasa-pds.github.io/registry/status/burnup_chart.html>`_ shows
+cumulative products loaded over time, broken down by PDS node.
+
+Raw Data
+--------
+
+CSV reports and full documentation are available in the
+`docs/status directory <https://github.com/NASA-PDS/registry/tree/main/docs/status>`_
+on GitHub.

docs/status/README.md

@@ -5,7 +5,7 @@ This directory contains automatically generated CSV reports that track the statu
 ## Metrics Summary
 
 <!-- METRICS_START -->
-*Last updated: 2026-05-11 20:35:12 UTC*
+*Last updated: 2026-05-13 20:00:31 UTC*
 
 ### Missing Products by Node
 
@@ -15,11 +15,11 @@ This directory contains automatically generated CSV reports that track the statu
 | PDS_ATM | 18 | 8 | 26 | 138 | 81 | 219 |
 | PDS_ENG | 2 | 6 | 8 | 5 | 88 | 93 |
 | PDS_GEO | 11 | 24 | 35 | 166 | 364 | 530 |
-| PDS_IMG | 36 | 35 | 71 | 1589 | 294 | 1883 |
-| PDS_PPI | 15 | 199 | 214 | 89 | 1347 | 1436 |
+| PDS_IMG | 36 | 35 | 71 | 1592 | 294 | 1886 |
+| PDS_PPI | 14 | 199 | 213 | 84 | 1347 | 1431 |
 | PDS_RMS | 8 | 0 | 8 | 23 | 0 | 23 |
-| PDS_SBN | 15 | 30 | 45 | 88 | 93 | 181 |
-| **Total** | **106** | **302** | **408** | **2101** | **2267** | **4368** |
+| PDS_SBN | 16 | 30 | 46 | 89 | 93 | 182 |
+| **Total** | **106** | **302** | **408** | **2100** | **2267** | **4367** |
 
 ### Staged Products by Node
 
@@ -28,11 +28,15 @@ This directory contains automatically generated CSV reports that track the statu
 | PDS_ATM | 3 | 30 |
 | PDS_GEO | 0 | 7 |
 | PDS_IMG | 0 | 64 |
-| PDS_PPI | 0 | 19 |
+| PDS_PPI | 2 | 30 |
 | PDS_RMS | 0 | 1 |
 | PDS_SBN | 26 | 137 |
 | PSA | 902 | 4171 |
-| **Total** | **931** | **4429** |
+| **Total** | **933** | **4440** |
+
+### Loading Progress
+
+[View Burnup Chart](burnup_chart.html) — cumulative products loaded over time vs. target
 
 <!-- METRICS_END -->