Skip to content

Commit 3197d28

Browse files
committed
feat(ui): paint trial glyph by QA verdict, keep shape as outcome
A GOOD_FAILURE trial rendered as a red X with a nearly invisible 4px green corner dot, and the analyzer legend labeled that dot "Pass" — an expected failure was indistinguishable from a bad one, or read as a success. The mirror case (BAD_SUCCESS) showed an unqualified green check. Make the cell semiotics uniform: glyph shape encodes the execution outcome (check/cross), fill color encodes the QA verdict once the trial is classified (green = good, red = bad). A good failure is now a green X, a bad success a red check; good-success and bad-failure look exactly as before. - status-config: getQaGlyphMatrixClass(status, classification) holds the 2x2 rule; shared ANALYSIS_CLASSIFICATION_LABELS. Only binary outcomes take the tone — partials keep the warm ramp, transitional states stay neutral. - trials matrix: apply the tone fill, show the corner dot only when it still carries information (QA running/failed, classified partial or error), and use the classification label in aria-label/tooltip. - legend: rename tone chips Pass/Fail to Good/Bad rendered as a check+X glyph pair, and rewrite the anatomy demo to teach shape=result, color=QA verdict. - trial-detail navigator: same override via the shared helper. Verified with tsc, eslint, and a browser-rendered harness covering all classification x outcome combinations (harness removed after).
1 parent 353be62 commit 3197d28

3 files changed

Lines changed: 147 additions & 19 deletions

File tree

frontend/src/components/experiment-trials-table.tsx

Lines changed: 73 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -76,11 +76,14 @@ import {
7676
taskHasCancellableWork,
7777
} from "@/lib/job-status";
7878
import {
79+
ANALYSIS_CLASSIFICATION_LABELS,
7980
formatPartialRewardBadgeValue,
8081
formatRewardPercent,
8182
formatRewardValue,
8283
getMatrixStatus,
84+
getQaGlyphMatrixClass,
8385
getRewardStyle,
86+
QA_TONE_MATRIX_CLASS,
8487
STATUS_CONFIG,
8588
STATUS_GLYPH_BOX,
8689
type MatrixStatus,
@@ -331,12 +334,12 @@ const ANALYSIS_LEGEND_ITEMS: Array<{
331334
},
332335
{
333336
key: "good",
334-
label: "Pass",
337+
label: "Good",
335338
dotClass: ANALYSIS_CONFIG.GOOD_SUCCESS.dotClass,
336339
},
337340
{
338341
key: "bad",
339-
label: "Fail",
342+
label: "Bad",
340343
dotClass: ANALYSIS_CONFIG.BAD_SUCCESS.dotClass,
341344
},
342345
{
@@ -1562,6 +1565,22 @@ export function ExperimentTrialsTable({
15621565
"analysis-failed": "var(--paper-a-failed)",
15631566
};
15641567

1568+
// Good/bad chips show the tone painted on both outcome shapes — a ✓✗
1569+
// pair — so the legend teaches the cell semiotics (shape = outcome,
1570+
// color = QA verdict) instead of implying green means "pass".
1571+
const renderToneGlyphPair = (tone: "good" | "bad") => (
1572+
<span className="inline-flex items-center gap-[2px]">
1573+
{(["pass", "fail"] as const).map((shape) => (
1574+
<span
1575+
key={shape}
1576+
className={`inline-flex h-[14px] w-[16px] items-center justify-center rounded-[3px] border ${QA_TONE_MATRIX_CLASS[tone]}`}
1577+
>
1578+
<StatusIcon status={shape} className="h-2 w-2" />
1579+
</span>
1580+
))}
1581+
</span>
1582+
);
1583+
15651584
const renderAnalyzerChip = (item: (typeof ANALYSIS_LEGEND_ITEMS)[number]) => {
15661585
const isDimmed = dimmedAnalysisKeys.has(item.key);
15671586
return (
@@ -1575,10 +1594,14 @@ export function ExperimentTrialsTable({
15751594
isDimmed ? "line-through opacity-[0.38]" : ""
15761595
}`}
15771596
>
1578-
<span
1579-
className={`inline-block h-2 w-2 rounded-full ${item.animate ? "animate-pulse" : ""}`}
1580-
style={{ background: ANALYZER_CHIP_COLOR[item.key] }}
1581-
/>
1597+
{item.key === "good" || item.key === "bad" ? (
1598+
renderToneGlyphPair(item.key)
1599+
) : (
1600+
<span
1601+
className={`inline-block h-2 w-2 rounded-full ${item.animate ? "animate-pulse" : ""}`}
1602+
style={{ background: ANALYZER_CHIP_COLOR[item.key] }}
1603+
/>
1604+
)}
15821605
<span>{item.label}</span>
15831606
</Button>
15841607
</TooltipTrigger>
@@ -1593,31 +1616,46 @@ export function ExperimentTrialsTable({
15931616
<Tooltip>
15941617
<TooltipTrigger asChild>
15951618
<div className="flex items-center gap-2.5 border-r border-dashed border-[color:var(--paper-line)] pr-2.5 pl-1.5 font-mono text-[9.5px] leading-tight text-[color:var(--paper-ink-3)]">
1596-
<span className="relative inline-flex">
1619+
<span className="relative inline-flex gap-[3px]">
15971620
<span
15981621
className={`flex items-center justify-center border-transparent bg-[color:var(--paper-pass)] text-white ${STATUS_GLYPH_BOX}`}
15991622
>
16001623
<StatusIcon status="pass" />
16011624
</span>
1625+
<span
1626+
className={`flex items-center justify-center border-transparent text-white ${STATUS_GLYPH_BOX} ${
1627+
showAnalysis
1628+
? "bg-[color:var(--paper-pass)]"
1629+
: "bg-[color:var(--paper-fail)]"
1630+
}`}
1631+
>
1632+
<StatusIcon status="fail" />
1633+
</span>
16021634
{showAnalysis && (
1603-
<span className="absolute -top-[2px] -right-[2px] h-[7px] w-[7px] rounded-full bg-[color:var(--paper-a-good)] ring-[1.5px] ring-[color:var(--paper-surface)]" />
1635+
<span className="absolute -top-[2px] -right-[2px] h-[7px] w-[7px] animate-pulse rounded-full bg-[color:var(--paper-a-analyzing)] ring-[1.5px] ring-[color:var(--paper-surface)]" />
16041636
)}
16051637
</span>
16061638
<span className="flex flex-col gap-0.5">
16071639
<span className="inline-flex items-center gap-1.5 whitespace-nowrap">
1608-
<span className="inline-block h-2.5 w-2.5 rounded-[2px] bg-[color:var(--paper-pass)]" />
1609-
trial result
1640+
<StatusIcon status="pass" className="h-2.5 w-2.5" />
1641+
<StatusIcon status="fail" className="h-2.5 w-2.5" />
1642+
shape = trial result
16101643
</span>
16111644
{showAnalysis && (
16121645
<span className="inline-flex items-center gap-1.5 whitespace-nowrap">
1613-
<span className="mx-[1px] inline-block h-2 w-2 rounded-full bg-[color:var(--paper-a-good)]" />
1614-
QA result
1646+
<span className="inline-block h-2.5 w-2.5 rounded-[2px] bg-[color:var(--paper-pass)]" />
1647+
<span className="inline-block h-2.5 w-2.5 rounded-[2px] bg-[color:var(--paper-fail)]" />
1648+
color = QA verdict
16151649
</span>
16161650
)}
16171651
</span>
16181652
</div>
16191653
</TooltipTrigger>
1620-
<TooltipContent>How to read a cell</TooltipContent>
1654+
<TooltipContent>
1655+
How to read a cell: ✓/✗ is the trial outcome; once QA classifies a
1656+
trial, green means the outcome is good (expected) and red means bad — a
1657+
green ✗ is a good failure. A pulsing dot means QA is still running.
1658+
</TooltipContent>
16211659
</Tooltip>
16221660
);
16231661

@@ -2435,6 +2473,25 @@ export function ExperimentTrialsTable({
24352473
dimmedAnalysisKeys.has(analysisLegendKey)
24362474
? "opacity-25"
24372475
: "";
2476+
// A classified binary outcome takes its QA
2477+
// tone as the glyph fill: shape = outcome,
2478+
// color = QA verdict. A good failure is a
2479+
// green ✗, a bad success a red ✓.
2480+
const classification =
2481+
trial.analysis_status === "success"
2482+
? (trial.analysis?.classification ?? null)
2483+
: null;
2484+
const qaGlyphClass = showAnalysis
2485+
? getQaGlyphMatrixClass(
2486+
status,
2487+
classification
2488+
)
2489+
: null;
2490+
const cellLabel = classification
2491+
? ANALYSIS_CLASSIFICATION_LABELS[
2492+
classification
2493+
]
2494+
: config.shortLabel;
24382495
const baseTitle = getTrialTitle(
24392496
trial,
24402497
status
@@ -2476,9 +2533,9 @@ export function ExperimentTrialsTable({
24762533
trialGroups,
24772534
});
24782535
}}
2479-
className={`relative grid place-items-center gap-0 p-0 leading-none transition-transform hover:-translate-y-px ${STATUS_GLYPH_BOX} ${config.matrixClass} ${isPartial ? "font-mono text-[9.5px] font-semibold tracking-[-0.02em] tabular-nums" : ""}`}
2536+
className={`relative grid place-items-center gap-0 p-0 leading-none transition-transform hover:-translate-y-px ${STATUS_GLYPH_BOX} ${qaGlyphClass ?? config.matrixClass} ${isPartial ? "font-mono text-[9.5px] font-semibold tracking-[-0.02em] tabular-nums" : ""}`}
24802537
style={getRewardStyle(trial.reward)}
2481-
aria-label={`Trial ${trialIndex + 1} ${config.shortLabel}`}
2538+
aria-label={`Trial ${trialIndex + 1} ${cellLabel}`}
24822539
title={fullTitle}
24832540
>
24842541
{isPartial ? (
@@ -2487,7 +2544,7 @@ export function ExperimentTrialsTable({
24872544
<StatusIcon status={status} />
24882545
)}
24892546
</Button>
2490-
{analysisIndicator && (
2547+
{analysisIndicator && !qaGlyphClass && (
24912548
<span
24922549
aria-hidden="true"
24932550
className={`pointer-events-none absolute -top-[1px] -right-[1px] h-[4px] w-[4px] rounded-full ring-[1px] ring-[color:var(--paper-surface)] ${analysisIndicator.dotClass} ${analysisIndicator.animate ? "animate-pulse" : ""}`}

frontend/src/components/trial-detail-panel.tsx

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -74,10 +74,12 @@ import {
7474
sumTaskTrialCost,
7575
} from "@/lib/format";
7676
import {
77+
ANALYSIS_CLASSIFICATION_LABELS,
7778
formatPartialRewardBadgeValue,
7879
formatRewardPercent,
7980
formatRewardValue,
8081
getMatrixStatus,
82+
getQaGlyphMatrixClass,
8183
getRewardStyle,
8284
STATUS_CONFIG,
8385
STATUS_GLYPH_BOX,
@@ -1255,6 +1257,20 @@ export function TrialDetailPanel({
12551257
groupTrial.error_message,
12561258
);
12571259
const groupConfig = STATUS_CONFIG[groupStatus];
1260+
// Same cell semiotics as the experiment matrix: a
1261+
// classified binary outcome takes its QA tone as the
1262+
// glyph fill (good failure = green ✗, bad success = red ✓).
1263+
const groupClassification =
1264+
groupTrial.analysis_status === "success"
1265+
? (groupTrial.analysis?.classification ?? null)
1266+
: null;
1267+
const groupQaGlyphClass = getQaGlyphMatrixClass(
1268+
groupStatus,
1269+
groupClassification,
1270+
);
1271+
const groupLabel = groupClassification
1272+
? ANALYSIS_CLASSIFICATION_LABELS[groupClassification]
1273+
: groupConfig.shortLabel;
12581274
const isPartial = groupStatus === "partial";
12591275
const partialLabel = isPartial
12601276
? formatPartialRewardBadgeValue(groupTrial.reward)
@@ -1270,7 +1286,7 @@ export function TrialDetailPanel({
12701286
className={cn(
12711287
"flex items-center justify-center p-0 leading-none transition hover:opacity-90",
12721288
STATUS_GLYPH_BOX,
1273-
groupConfig.matrixClass,
1289+
groupQaGlyphClass ?? groupConfig.matrixClass,
12741290
isPartial
12751291
? "font-mono text-[9.5px] font-semibold tracking-[-0.02em] tabular-nums"
12761292
: "",
@@ -1279,8 +1295,8 @@ export function TrialDetailPanel({
12791295
: "",
12801296
)}
12811297
style={getRewardStyle(groupTrial.reward)}
1282-
aria-label={`Trial ${index + 1} ${groupConfig.shortLabel}`}
1283-
title={`${groupConfig.shortLabel} • Trial ${index + 1}`}
1298+
aria-label={`Trial ${index + 1} ${groupLabel}`}
1299+
title={`${groupLabel} • Trial ${index + 1}`}
12841300
>
12851301
{isPartial ? (
12861302
partialLabel

frontend/src/lib/status-config.ts

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ import {
1010
} from "lucide-react";
1111
import type { CSSProperties } from "react";
1212

13+
import type { AnalysisClassification } from "@/lib/types";
14+
1315
/**
1416
* Trial status types that map to visual states in the UI.
1517
* These are derived from trial.status and trial.reward values.
@@ -25,6 +27,59 @@ export type MatrixStatus =
2527
| "queued"
2628
| "running";
2729

30+
/**
31+
* QA judgment tone for a classified trial. The glyph semiotics are:
32+
* shape = execution outcome (✓ pass / ✗ fail), fill color = QA verdict
33+
* (green = good, red = bad). This makes an expected "good failure" render
34+
* as a green ✗ instead of a red ✗ with a nearly-invisible green dot, and
35+
* a "bad success" as a red ✓ instead of an unqualified green ✓.
36+
*/
37+
export type QaTone = "good" | "bad";
38+
39+
/**
40+
* Matrix glyph fill for a QA tone. Reuses the pass/fail paper tokens so a
41+
* recolored cell is indistinguishable in hue from an unclassified one —
42+
* only the shape-vs-color meaning changes.
43+
*/
44+
export const QA_TONE_MATRIX_CLASS: Record<QaTone, string> = {
45+
good: "bg-paper-pass text-white border-paper-pass hover:opacity-90",
46+
bad: "bg-paper-fail text-white border-paper-fail hover:opacity-90",
47+
};
48+
49+
/** Human label per QA classification, shared by every trial-glyph surface. */
50+
export const ANALYSIS_CLASSIFICATION_LABELS: Record<
51+
AnalysisClassification,
52+
string
53+
> = {
54+
GOOD_SUCCESS: "Good success",
55+
GOOD_FAILURE: "Good failure",
56+
BAD_SUCCESS: "Bad success",
57+
BAD_FAILURE: "Bad failure",
58+
HARNESS_ERROR: "Harness error",
59+
};
60+
61+
/**
62+
* Glyph fill override for a classified trial, or null when the outcome
63+
* glyph keeps its default color. Only binary outcomes (pass/fail) take the
64+
* QA tone; partials keep their warm ramp and non-terminal or non-scored
65+
* states keep their neutral styling, with the QA tone still surfaced
66+
* through the corner dot. GOOD_SUCCESS pass and BAD_FAILURE fail resolve
67+
* to the same color they already had — the rule stays uniform.
68+
*/
69+
export function getQaGlyphMatrixClass(
70+
status: MatrixStatus,
71+
classification: AnalysisClassification | null | undefined
72+
): string | null {
73+
if (status !== "pass" && status !== "fail") return null;
74+
if (classification === "GOOD_SUCCESS" || classification === "GOOD_FAILURE") {
75+
return QA_TONE_MATRIX_CLASS.good;
76+
}
77+
if (classification === "BAD_SUCCESS" || classification === "BAD_FAILURE") {
78+
return QA_TONE_MATRIX_CLASS.bad;
79+
}
80+
return null;
81+
}
82+
2883
/**
2984
* Status configuration for consistent styling across the UI.
3085
*

0 commit comments

Comments
 (0)