feat(frontend): task × agent solve grid in the Eval section - #1100
Conversation
Add a solve-grid heatmap next to the Pareto frontier card: one row per task sorted hardest-first, one column per shown agent, each cell the per-task pass rate colored with the dashboard's status tokens — a pass ramp by solve rate, a fail tint for ran-and-never-passed, and a faint neutral for not-run — with per-cell c/n counts in the tooltip and a legend. Rows scroll under a sticky header for large experiments, and hiding agents via the shared legend drops their columns. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BzsxcETgi6UjLtCnety4sQ
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 68501eb. Configure here.
| if (cell.c === 0) | ||
| return "color-mix(in oklch, var(--paper-fail) 28%, transparent)"; | ||
| const strength = Math.round(20 + 80 * (cell.c / cell.n)); | ||
| return `color-mix(in oklch, var(--paper-pass) ${strength}%, transparent)`; |
There was a problem hiding this comment.
In-progress trials painted as failed
Medium Severity
Every trial increments n, and any cell with c === 0 gets the fail tint meant for ran-and-never-passed. Pending, queued, and running trials have a null reward, so they land in that fail bucket and also pull the hardest-first sort upward. Live experiments then read unfinished work as hard failures, unlike the main trial matrix which keeps those statuses separate.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 68501eb. Configure here.
There was a problem hiding this comment.
Risk: medium. Left a non-blocking comment only — Cursor Bugbot completed as skipped and reported 1 unresolved finding that needs human attention; Cursor Security Agent was not present. No additional reviewers assigned (2 already requested).
Sent by Cursor Approval Agent: Pull Request Router and Approver
2 participants
Stack 3/4 — based on #1099 (merge #1098 → #1099 first; this PR then retargets and needs a rebase/update-branch).
Adds a solve grid heatmap next to the Pareto frontier card in the Eval section: one row per task sorted hardest-first, one column per shown agent, each cell that agent's per-task pass rate.
c/ncounts in the tooltip, a legend below, rows scroll under a sticky header for large experiments, and hiding agents via the shared legend drops their columns.Same feature gate as #1099 (Eval toggle, allowlisted accounts only).
Verified with
tsc --noEmit, ESLint,next build, and rendered screenshots.🤖 Generated with Claude Code
https://claude.ai/code/session_01BzsxcETgi6UjLtCnety4sQ
Generated by Claude Code
Note
Low Risk
Read-only frontend analytics behind the existing Eval feature gate; no API or trial execution changes.
Overview
The experiment page Eval section (same allowlist as the Pareto card) now includes a Solve grid card beside the Pareto frontier: rows are tasks sorted hardest first (mean pass rate across agents that ran the task), columns are visible agent cohorts, and each cell encodes that agent’s per-task pass rate from trial
rewardcounts.Cell styling separates not run (neutral), ran but never passed (fail tint), and partial/full pass (green ramp). Tooltips show
c/n, agent columns respect the shared legend’s hidden agents, and the grid scrolls under a sticky agent header.experiment-trials-tablelazy-loadsTaskSolveHeatmapin the existing two-column Eval layout; CHANGELOG documents the solve grid alongside Pareto.Reviewed by Cursor Bugbot for commit 68501eb. Bugbot is set up for automated code reviews on this repo. Configure here.