v0.1.0 — zero known bugs (first minor release) #357
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Stress tests | |
| # Stress tests (#596) are scale-dependent regression tests that | |
| # exercise the runtime at sizes where #570 / #515 / #593-class | |
| # bugs historically manifested. They're skipped from the | |
| # default per-PR pytest run (via `addopts = "-m 'not stress'"` | |
| # in pyproject.toml) because the full suite at 5 minutes wall- | |
| # clock would dominate ordinary CI time. | |
| # | |
| # This workflow runs them in three triggers: | |
| # | |
| # 1. Nightly cron — primary safety net. Catches drift in a | |
| # daily window so the cost of bisecting a regression stays | |
| # small. Failures auto-file (or comment on) a tracking | |
| # issue with the `stress-regression` label. | |
| # 2. Path-filtered PRs touching `vera/codegen/`, `vera/wasm/`, | |
| # `vera/checker/`, or `tests/test_stress.py` — fail-fast for | |
| # PRs that change the code most likely to break stress | |
| # invariants. `vera/checker/` is included because the AST | |
| # shape it produces flows into codegen — a checker change | |
| # that subtly alters the AST can break runtime invariants | |
| # without touching `vera/codegen/` or `vera/wasm/`. Failures | |
| # fail the PR check directly; no tracking issue is filed | |
| # (the PR author already sees the failure in the PR's checks | |
| # tab). | |
| # 3. Manual workflow_dispatch — anyone can trigger from the | |
| # Actions tab for local-suspicious commits. Failures are | |
| # visible to whoever triggered the run; no tracking issue | |
| # is filed. | |
| on: | |
| schedule: | |
| # 06:00 UTC daily — quiet time for most contributor TZs; | |
| # results land before US East Coast morning standups. | |
| - cron: '0 6 * * *' | |
| pull_request: | |
| branches: [main] | |
| paths: | |
| - 'vera/codegen/**' | |
| - 'vera/wasm/**' | |
| - 'vera/checker/**' | |
| - 'tests/test_stress.py' | |
| - '.github/workflows/nightly-stress.yml' | |
| workflow_dispatch: | |
| jobs: | |
| stress: | |
| permissions: | |
| contents: read | |
| # `issues: write` is needed by the failure-reporting step | |
| # below so the workflow can open or comment on a tracking | |
| # issue when the nightly cron fails. Scoped to the job, | |
| # not workflow-wide, so the test-runner step keeps the | |
| # tighter `contents: read` default. | |
| issues: write | |
| runs-on: ubuntu-latest | |
| env: | |
| PYTHONUTF8: 1 | |
| steps: | |
| - uses: actions/checkout@v7 | |
| with: | |
| persist-credentials: false | |
| - name: Set up Python 3.12 | |
| uses: actions/setup-python@v6 | |
| with: | |
| python-version: "3.12" | |
| - name: Install dependencies | |
| run: pip install -e ".[dev]" | |
| - name: Run stress tests | |
| # `-m stress` overrides the default `-m 'not stress'` | |
| # addopts; runs ONLY the stress-marked tests. | |
| run: pytest -v -m stress tests/test_stress.py | |
| # Auto-file a tracking issue on cron failure (option A from | |
| # #669 review). Skipped on `pull_request` (PR's own check | |
| # tab is the reporting surface) and on `workflow_dispatch` | |
| # (whoever triggered the run is already paying attention). | |
| # | |
| # Deduplication: searches for an open issue carrying the | |
| # `stress-regression` label. If one exists, appends a | |
| # comment with the new commit SHA + run URL. If none, files | |
| # a fresh issue with the label. The label is auto-created | |
| # if it doesn't yet exist on the repo. | |
| # | |
| # Open tracking issues stay open across days until manually | |
| # closed; this is by design — auto-closing on success would | |
| # be wrong while a maintainer is mid-debug. | |
| - name: Open or update tracking issue on cron failure | |
| if: failure() && github.event_name == 'schedule' | |
| uses: actions/github-script@v9 | |
| env: | |
| RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} | |
| COMMIT_SHA: ${{ github.sha }} | |
| with: | |
| script: | | |
| const LABEL = "stress-regression"; | |
| const TITLE = "Nightly stress regression on main (tracking)"; | |
| const sha = process.env.COMMIT_SHA; | |
| const sha7 = sha.slice(0, 7); | |
| const runUrl = process.env.RUN_URL; | |
| const commentBody = [ | |
| `Stress nightly failed against \`${sha7}\` (commit ${sha}).`, | |
| ``, | |
| `**Run logs**: ${runUrl}`, | |
| ``, | |
| `This is a regression on \`main\`; bisect against the green commits before this run. See \`tests/test_stress.py\` for the failing test's docstring (each test names the bug class it guards against, which narrows the regression family).`, | |
| ].join("\n"); | |
| // Ensure the label exists (idempotent — create on first failure). | |
| try { | |
| await github.rest.issues.getLabel({ | |
| owner: context.repo.owner, | |
| repo: context.repo.repo, | |
| name: LABEL, | |
| }); | |
| } catch (e) { | |
| if (e.status === 404) { | |
| await github.rest.issues.createLabel({ | |
| owner: context.repo.owner, | |
| repo: context.repo.repo, | |
| name: LABEL, | |
| color: "d73a4a", | |
| description: "Nightly stress test failure on main (#596)", | |
| }); | |
| core.info(`Created label '${LABEL}'`); | |
| } else { | |
| throw e; | |
| } | |
| } | |
| // Find an existing open tracking issue with this label. | |
| // `listForRepo` returns BOTH issues and PRs that carry the | |
| // label; filter to issues only (items with no | |
| // `pull_request` field) so we never accidentally post a | |
| // failure comment onto a PR that happened to be labelled. | |
| // The `stress-regression` label is only ever applied by | |
| // this workflow today, but defensive filtering costs ~3 | |
| // lines and prevents a future misapplication from | |
| // mis-routing notifications. | |
| const { data: candidates } = await github.rest.issues.listForRepo({ | |
| owner: context.repo.owner, | |
| repo: context.repo.repo, | |
| labels: LABEL, | |
| state: "open", | |
| per_page: 10, | |
| }); | |
| const existing = candidates.filter(item => !item.pull_request); | |
| if (existing.length > 0) { | |
| const issueNumber = existing[0].number; | |
| await github.rest.issues.createComment({ | |
| owner: context.repo.owner, | |
| repo: context.repo.repo, | |
| issue_number: issueNumber, | |
| body: commentBody, | |
| }); | |
| core.info(`Posted failure comment on existing tracking issue #${issueNumber}`); | |
| } else { | |
| const { data: created } = await github.rest.issues.create({ | |
| owner: context.repo.owner, | |
| repo: context.repo.repo, | |
| title: TITLE, | |
| body: commentBody, | |
| labels: [LABEL], | |
| }); | |
| core.info(`Filed fresh tracking issue #${created.number}`); | |
| } |