Skip to content

Commit 7baf175

Browse files
baibaichenclaude
andcommitted
[GLUTEN-10134][VL] Add ANSI analysis SKILL entry point
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
1 parent 19df2b6 commit 7baf175

1 file changed

Lines changed: 95 additions & 0 deletions

File tree

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
---
2+
name: ansi-analysis
3+
description: Analyze Gluten ANSI-mode test results (run dev/verify-ansi-expressions.sh, parse JSON tracker output, produce root-cause analysis and fix recommendations). Trigger on user requests like "analyze ANSI tests", "run ANSI matrix", "why is this ANSI test failing".
4+
---
5+
6+
# ANSI Test Analysis Skill
7+
8+
## Step 0 — MUST READ FIRST: shared analysis prompt
9+
10+
Before doing anything else, read the shared prompt that defines the analysis output format and reference source locations:
11+
12+
```
13+
.github/skills/ansi-analysis/shared.md
14+
```
15+
16+
This file is the single source of truth — the same content is consumed by the CI Python pipeline (`.github/skills/ansi-analysis/analyze-ansi.py --ai-analysis`). Your output structure, reference source locations, and self-investigation steps MUST follow it. If the file is missing, STOP and tell the user the repo is in a broken state.
17+
18+
## Step 1 — Decide entry point
19+
20+
Ask the user (or infer from request):
21+
- Run new tests? → Step 2
22+
- Re-analyze existing JSON in `target/ansi-offload/`? → Step 3
23+
- Diagnose a single test failure? → Step 4
24+
25+
## Step 2 — Run the verification script
26+
27+
```bash
28+
./dev/verify-ansi-expressions.sh <category> <spark41|spark40|all> [--clean]
29+
```
30+
31+
Categories: `cast | arithmetic | collection | datetime | math | decimal | string | aggregate | errors | all`
32+
33+
Logs: `/tmp/ansi-matrix/latest/` (bash logs).
34+
JSON: `target/ansi-offload/*.json` (written by `GlutenExpressionOffloadTracker.scala`, this is the structured input for analysis).
35+
36+
Notes from prior runs:
37+
- Use `all` mode in single JVM (~28 min) when full coverage is needed
38+
- After rebase / branch switch, run `./dev/builddep-veloxbe-inc.sh` first to refresh `libvelox.so` / `libgluten.so`
39+
40+
## Step 3 — Analyze JSON results
41+
42+
Two options:
43+
44+
### 3a. Local AI orchestration (this skill, recommended for interactive review)
45+
46+
1. Read `.github/skills/ansi-analysis/shared.md` (Step 0)
47+
2. List `target/ansi-offload/*.json`
48+
3. Read each JSON; extract: suite name, total/passed/failed/ignored counts, per-test `failCause`
49+
4. Apply the analysis template from shared.md verbatim (sections, tables, constraints)
50+
5. For each failure: extract Velox file:line from `failCause`, read those C++ files, verify root cause
51+
6. **Always** grep `isAnsiSupported` in `ep/build-velox/build/velox_ep/velox/functions/sparksql/specialforms/SparkCastExpr.cpp` when the failure involves Cast — most NO_EXCEPTION/Cast failures stem from the small whitelist there
52+
7. Output the markdown report
53+
54+
### 3b. Python script (CI / batch)
55+
56+
```bash
57+
python3 .github/skills/ansi-analysis/analyze-ansi.py \
58+
--json-dir target/ansi-offload/ \
59+
--ai-analysis \
60+
--output ansi-report.md
61+
```
62+
63+
The script loads the same shared prompt and calls the GitHub Models API.
64+
65+
## Step 4 — Single-failure diagnosis
66+
67+
When the user pastes one failing test:
68+
1. Locate its JSON entry under `target/ansi-offload/`
69+
2. Apply the self-investigation steps from shared.md (extract Velox file:line, check `isAnsiSupported`, cross-check `withAnsiEvalMode` in the shim)
70+
3. Output: Symptom / Root Cause / Fix Point / Representative Tests / Estimated Impact
71+
72+
## Step 5 — Optional PR comment
73+
74+
If the user wants the report posted to a PR:
75+
76+
```bash
77+
gh pr comment <pr-number> --body-file ansi-report.md
78+
```
79+
80+
(or use the GitHub MCP server tool when available)
81+
82+
## Environment requirements
83+
84+
For Step 2 (running tests):
85+
- `SPARK_ANSI_SQL_MODE=true`
86+
- `SPARK_TESTING=true`
87+
- `SPARK_SCALA_VERSION=2.13`
88+
- JVM: `-Dspark.gluten.sql.ansiFallback.enabled=false`
89+
- Maven profile: include `-Pdelta`
90+
91+
## What NOT to do
92+
93+
- Do NOT invent reference paths or line numbers — always grep / verify
94+
- Do NOT skip Step 0 — drift between shared.md and your output is the failure mode this skill is designed to prevent
95+
- Do NOT bypass the shared prompt by writing your own analysis structure

0 commit comments

Comments
 (0)