Skip to content

[CRCR] Fix nightly dashboard partial rows at time-window boundary - #8695

Open
subinz1 wants to merge 2 commits into
mainfrom
crcr-nightly-run-level-filter
Open

[CRCR] Fix nightly dashboard partial rows at time-window boundary#8695
subinz1 wants to merge 2 commits into
mainfrom
crcr-nightly-run-level-filter

Conversation

@subinz1

@subinz1 subinz1 commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes the "no signal" cells appearing on the last row of the nightly dashboard, and partially addresses the jobs count discrepancy reported in #8691.

Root cause: The nightly dashboard query filtered by per-job started_at > now() - INTERVAL {days} DAY. In a nightly pipeline, jobs execute sequentially — build starts first, then tests start 1-2 hours later. When the time window clips through a run at the boundary, earlier-starting jobs (build) age out while later-starting jobs (tests) remain. This produces partial SHA rows with empty cells for the disappeared jobs.

Fix: Switch to run-level filtering using a CTE. First, identify eligible_runs where at least one job started within the window. Then include ALL jobs from those runs, regardless of individual started_at. This ensures complete nightly rows at the boundary — no more jobs disappearing one-by-one from the last row.

Before:

WHERE started_at > now() - INTERVAL {days} DAY  -- per-job cutoff

After:

WITH eligible_runs AS (
    SELECT DISTINCT run_id ... WHERE started_at > now() - INTERVAL {days} DAY
)
WHERE run_id IN (SELECT run_id FROM eligible_runs)  -- run-level cutoff

Ref #8691

Test plan

  • Verify nightly dashboard loads without errors
  • Confirm the last SHA row now shows complete data (no "no signal" cells for jobs that were part of the same run)
  • CI passes

Closes #8691 (partial)

Fixes #8691 (partial — companion: #8696)

The nightly dashboard query filtered by per-job started_at, which
caused partial SHA rows when a pipeline's jobs start at different
times. For example, with a 7-day window and a nightly run where the
build job starts first and test jobs start 1-2 hours later, the build
job would age out of the window while tests remained — producing a
row with missing cells ("no signal") for the older jobs.

Switch to run-level filtering: identify eligible run_ids (where at
least one job started within the window), then include ALL jobs from
those runs. This ensures complete nightly rows at the boundary.

Ref #8691
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 1, 2026
@vercel

vercel Bot commented Sep 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
torchci Ignored Ignored Sep 1, 2026 1:42pm UTC

Request Review

@subinz1
subinz1 marked this pull request as ready for review September 1, 2026 14:19
@subinz1
subinz1 requested a review from atalman September 1, 2026 14:52
@subinz1 subinz1 changed the title Fix nightly dashboard partial rows at time-window boundary [CRCR] Fix nightly dashboard partial rows at time-window boundary Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CRCR] Fix discrepancy between jobs and failures reported and the table displayed. Fix invalid no signal on last row

1 participant