Skip to content

README.md

Latest commit

 

History

History
98 lines (78 loc) · 6.48 KB

README.md

File metadata and controls

98 lines (78 loc) · 6.48 KB

scripts/ - Quick Reference

Root-level scripts are generic orchestrators for the research pipeline. They coordinate setup, tests, analysis, rendering, validation, copying, LLM review, and executive reporting without implementing project-specific business logic.

Entry Points

Shell

./run.sh
./run.sh --pipeline --resume
./run.sh --all-projects --pipeline
./secure_run.sh --project template_code_project

Python

uv run python scripts/execute_pipeline.py --project template_code_project --core-only
uv run python scripts/execute_multi_project.py --no-llm
uv run python scripts/07_generate_executive_report.py

Pipeline Stages

Stage Script Responsibility
00 00_setup_environment.py Validate Python, dependencies, build tools, and directories
01 01_run_tests.py Run infrastructure and project tests. Project pipelines use --infra-scope pipeline-smoke for a focused real infrastructure contract; explicit repo verification uses --infra-scope full for the coverage-bearing infrastructure suite. With --project-only --all-projects, delegates to infrastructure.core.test_runner.run_per_project_pytest (one pytest process per project, --cov-append, combined coverage gate for local all-project/release sweeps).
02 02_run_analysis.py Discover and run projects/{name}/scripts/
03 03_render_pdf.py Render manuscripts to PDF
04 04_validate_output.py Validate PDFs, markdown, and integrity reports
05 05_copy_outputs.py Copy final deliverables to output/
06 06_llm_review.py Generate LLM reviews or translations when Ollama is available
07 07_generate_executive_report.py Build multi-project executive summaries and dashboards

execute_pipeline.py also supports single-stage execution with keys such as setup, tests, analysis, render_pdf, validate, copy, llm_reviews, llm_translations, and executive_report.

Key Files

  • execute_pipeline.py / execute_multi_project.py - single- and multi-project pipeline runners
  • generate_active_projects_doc.py - regenerates docs/_generated/active_projects.md
  • generate_api_reference_doc.py - regenerates the API reference (CI validate --check)
  • generate_architecture_overview.py - regenerates docs/_generated/architecture_overview.{mmd,svg}
  • generate_coverage_history.py - regenerates the coverage-history page from CI artefacts
  • generate_stage_table_doc.py - regenerates the canonical stage-table marker block
  • generate_exemplar_roster_doc.py - regenerates the public exemplar roster doc (infrastructure.project.exemplar_roster)
  • generate_publication_records_doc.py - regenerates the publication-records doc (infrastructure.documentation.publication_records)
  • 10_repro_bundle.py - builds/verifies reproduction bundles (infrastructure.publishing.repro_bundle)
  • lint_docs.py - runs Mermaid, link, consistency, and doc-pair documentation checks
  • audit_documentation.py - emits the advisory public documentation RedTeam audit
  • verify_no_mocks.py - checks tests for mock usage
  • audit_filepaths.py - repository filepath and reference audit
  • check_tracked_generated_artifacts.py - rejects tracked generated outputs and package metadata
  • ci_local.sh - local CI reproduction (act when available, otherwise a pure-Python CI fallback; see ../docs/maintenance/ci-local.md)
  • Maintenance helpers now live under maintenance/ - manage_workspace.py, show_project_info.py, render_working_projects.py, rerender_working_pdfs.py, organize_executive_outputs.py, merge_test_supplements.py, batch_cogsec_improve.py, setup_pre_commit.py, codegraph_local.py (see maintenance/README.md and maintenance/AGENTS.md). show_project_info.py is a standalone project metadata CLI; it is not invoked by run.sh (the menu's i key prints only the current project name).
  • bash_utils.sh - shared shell helpers for backup/health scripts and integration tests (not sourced by run.sh / secure_run.sh)
  • shell_bootstrap.sh - shared uv bootstrap and sandbox env vars sourced by run.sh and secure_run.sh
  • backup-daily.sh / backup-weekly.sh / backup-full.sh - rsync backup tiers
  • restore-test.sh - non-destructive backup-restore verification
  • health-check.sh - pre-flight system health check (Python, uv, disk, Docker, repo)

Quality gates (thin orchestrator)

Gate Command
Exemplar drift uv run python scripts/check_template_drift.py --strict
Documentation RedTeam audit uv run python scripts/audit_documentation.py --format markdown
Module line count uv run python scripts/gates/module_line_count_check.py
Tracked projects guard uv run python scripts/check_tracked_projects.py
Tracked generated artifacts uv run python scripts/check_tracked_generated_artifacts.py
CodeGraph local scope codegraph files "$(pwd)" --json | uv run python scripts/maintenance/codegraph_local.py verify-scope
Unified health uv run python -m infrastructure.core.health
Opt-in Stage 10 bundle uv run python scripts/08_executable_bundle.py --project {name}
Opt-in Stage 11 archival uv run python scripts/09_archive_publication.py --project {name}

See docs/architecture/thin-orchestrator-summary.md and gates/AGENTS.md.

Output Names

  • projects/{name}/output/reports/test_results.{json,md}
  • projects/{name}/output/reports/validation_report.{json,md}
  • projects/{name}/output/reports/log_summary.txt
  • output/executive_summary/consolidated_report.{json,html,md}
  • output/executive_summary/dashboard.{png,pdf,html}

Notes

  • Project-specific analysis scripts belong in projects/{name}/scripts/.
  • The root scripts stay generic and work with any active project discovered from projects/.
  • Use uv run for direct Python entry points (execute_pipeline.py, execute_multi_project.py, 07_generate_executive_report.py).
  • run.sh and secure_run.sh are thin bootstrap shells: they source shell_bootstrap.sh, then exec uv run python -m infrastructure.orchestration (menu, pipeline, and secure subcommands live in Python). Bare ./run.sh relies on uv run to sync the workspace; pipeline-flag invocations also run uv sync when .venv is missing.

See Also