Skip to content

Add nbmake-based CI workflow to execute and validate all notebooks #93

Description

@jordanpadams

Summary

There is currently no automated testing for the Jupyter notebooks in this repository. Notebooks can silently break when PDS API responses change, dependencies update, or notebook logic has bugs.

Proposed solution: Add a nbmake-based pytest workflow that executes all notebooks end-to-end and fails if any cell raises an error.

Implementation Plan

Tool: nbmake (pytest plugin for notebook execution)

Workflow structure: One GitHub Actions job per discipline group, using the per-discipline requirements files already in the repo:

strategy:
  matrix:
    include:
      - group: peppi-advanced
        requirements: requirements-peppi-advanced.txt
        path: notebooks/generic/peppi-advanced/
      - group: training
        requirements: requirements-training.txt
        path: notebooks/generic/training_20260305/
      - group: sbn
        requirements: requirements-sbn.txt
        path: notebooks/discipline/sbn/
      # etc.

Known challenges:

  • All notebooks hit the live PDS API — tests are network-dependent
  • Some notebooks are slow (large result sets, SPICE kernel downloads)
  • Heavy deps (spiceypy, pywwt, geopandas) increase CI image setup time

Mitigations:

  • Run on a nightly schedule rather than every push
  • Use --nbmake-timeout to cap slow notebooks
  • Mark known-problematic notebooks to skip via cell tags or --ignore

Acceptance Criteria

  • nbmake added to a test requirements file
  • GitHub Actions workflow that runs notebooks per discipline group
  • Nightly schedule trigger + manual workflow_dispatch
  • Known-slow or API-heavy notebooks documented and handled
  • README updated with instructions for running tests locally

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status
    ToDo
    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions