Skip to content

Commit ab5b9d9

Browse files
viktoravelinoviktoravelino
authored
chore: Add accessibility scan reports (#13812)
* feat: add accessibility scan reports * docs: add a11y scan route map * test(a11y): add manifest-backed scans * ci(a11y): summarize scans by route
1 parent 5eaf6f9 commit ab5b9d9

11 files changed

Lines changed: 2227 additions & 47 deletions

File tree

‎.agents/skills/ibm-a11y-automation/SKILL.md‎

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
---
2+
name: ibm-a11y-automation
3+
description: Run Langflow's local accessibility scanner script against frontend routes from src/frontend/src/routes.tsx and summarize the JSON report.
4+
---
5+
6+
# Langflow Accessibility Scanner
7+
8+
Use this skill when asked to scan Langflow frontend pages for accessibility issues.
9+
10+
## Scanner
11+
12+
Use the Python script:
13+
14+
```bash
15+
uv run python scripts/a11y/a11y_scan.py \
16+
--url http://localhost:3000 \
17+
--routes-file scripts/a11y/a11y_routes.json \
18+
--route-group static \
19+
--out /tmp/langflow-a11y-report.json \
20+
--markdown /tmp/langflow-a11y-report.md \
21+
--html /tmp/langflow-a11y-report.html \
22+
--timeout-ms 45000
23+
```
24+
25+
Script options:
26+
27+
- `--url`: base app URL, usually `http://localhost:3000`.
28+
- `--routes-file`: route manifest JSON file. Prefer `scripts/a11y/a11y_routes.json`.
29+
- `--route-group`: manifest group to scan. Default: `static`.
30+
- `--routes`: comma-separated route paths to scan.
31+
- `--route`: one route path; can be repeated instead of `--routes`.
32+
- `--levels`: comma-separated issue levels. Default: `violation`.
33+
- `--out`: JSON report path.
34+
- `--markdown`: optional Markdown report path.
35+
- `--html`: optional self-contained HTML report path.
36+
- `--timeout-ms`: per-route timeout.
37+
- `--quiet-ms`: network quiet window before scanning. Default: `1000`.
38+
- `--states-file`: JSON file with explicit modal/state actions.
39+
- `--headed`: show browser while scanning.
40+
41+
## Route Selection
42+
43+
Use `scripts/a11y/a11y_routes.json` as the source of truth for route selection.
44+
45+
The normal CI/local batch is the manifest `static` group. Prefer that unless the user asks for custom, dynamic, or gated routes.
46+
47+
Common manifest-backed command:
48+
49+
```bash
50+
uv run python scripts/a11y/a11y_scan.py \
51+
--url http://localhost:3000 \
52+
--routes-file scripts/a11y/a11y_routes.json \
53+
--route-group static \
54+
--out /tmp/langflow-a11y-static.json \
55+
--markdown /tmp/langflow-a11y-static.md \
56+
--html /tmp/langflow-a11y-static.html
57+
```
58+
59+
Dynamic routes need real IDs before scanning:
60+
61+
- `/flow/:id/`
62+
- `/flow/:id/view`
63+
- `/playground/:id/`
64+
- `/assets/knowledge-bases/:sourceId/chunks`
65+
66+
For dynamic routes, get IDs from the loaded app, API responses, or existing test data before replacing placeholders.
67+
68+
## Examples
69+
70+
Scan one route:
71+
72+
```bash
73+
uv run python scripts/a11y/a11y_scan.py \
74+
--url http://localhost:3000 \
75+
--route /flows \
76+
--out /tmp/langflow-a11y-flows.json
77+
```
78+
79+
Scan multiple routes:
80+
81+
```bash
82+
uv run python scripts/a11y/a11y_scan.py \
83+
--url http://localhost:3000 \
84+
--routes-file scripts/a11y/a11y_routes.json \
85+
--route-group static \
86+
--out /tmp/langflow-a11y-report.json \
87+
--markdown /tmp/langflow-a11y-report.md \
88+
--html /tmp/langflow-a11y-report.html
89+
```
90+
91+
Scan more than violations:
92+
93+
```bash
94+
uv run python scripts/a11y/a11y_scan.py \
95+
--url http://localhost:3000 \
96+
--routes-file scripts/a11y/a11y_routes.json \
97+
--route-group static \
98+
--levels violation,potentialviolation,recommendation \
99+
--out /tmp/langflow-a11y-expanded.json
100+
```
101+
102+
Scan route plus modal states:
103+
104+
```bash
105+
uv run python scripts/a11y/a11y_scan.py \
106+
--url http://localhost:3000 \
107+
--states-file /tmp/langflow-a11y-states.json \
108+
--out /tmp/langflow-a11y-modal-report.json \
109+
--markdown /tmp/langflow-a11y-modal-report.md \
110+
--html /tmp/langflow-a11y-modal-report.html \
111+
--timeout-ms 45000
112+
```
113+
114+
State file shape:
115+
116+
```json
117+
[
118+
{
119+
"route": "/settings/global-variables",
120+
"states": [
121+
{
122+
"name": "new-global-variable-modal",
123+
"open": [
124+
{ "click": "[data-testid='api-key-button-store']" },
125+
{ "waitFor": "[role='dialog']" }
126+
],
127+
"close": [
128+
{ "press": "Escape" },
129+
{ "waitForHidden": "[role='dialog']" }
130+
]
131+
}
132+
]
133+
}
134+
]
135+
```
136+
137+
Supported state actions:
138+
139+
- `{ "click": "<css selector>" }`
140+
- `{ "clickText": "<visible text>" }`
141+
- `{ "clickRole": { "role": "button", "name": "Create" } }`
142+
- `{ "fill": { "selector": "<css selector>", "value": "text" } }`
143+
- `{ "press": "Escape" }`
144+
- `{ "press": { "selector": "<css selector>", "key": "Enter" } }`
145+
- `{ "waitFor": "<css selector>" }`
146+
- `{ "waitForHidden": "<css selector>" }`
147+
- `{ "waitForText": "<visible text>" }`
148+
- `{ "wait": 500 }`
149+
150+
## Report
151+
152+
The scanner always writes JSON. It can also write Markdown and HTML for presentation.
153+
154+
Use JSON for exact data. Use Markdown for PR comments or issues. Use HTML when the user wants a browsable report.
155+
156+
Summarize:
157+
158+
- report path
159+
- Markdown/HTML report paths, when generated
160+
- total issue count
161+
- per-route issue count
162+
- per-route API request count
163+
- per-route request failure count
164+
- top rule IDs
165+
166+
Use report fields directly:
167+
168+
- `totalIssues`
169+
- `results[].route`
170+
- `results[].state`
171+
- `results[].phase`
172+
- `results[].apiRequests`
173+
- `results[].requestFailures`
174+
- `results[].diagnostics`
175+
- `results[].issues[].ruleId`
176+
177+
## Rules
178+
179+
- Use only scanner output for findings.
180+
- Do not invent route names. Read `routes.tsx`.
181+
- Do not auto-click arbitrary buttons to find modals. Use explicit state actions.
182+
- Avoid destructive modal actions unless the user explicitly asks and data is safe.
183+
- If a route has zero API requests, mention that scan quality may be limited.
184+
- Ask before fixing files unless the user explicitly asks for fixes.

‎.github/workflows/a11y-scan.yml‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -135,11 +135,13 @@ jobs:
135135
run: |
136136
cd src/frontend
137137
if ls coverage/accessibility-reports/*.json > /dev/null 2>&1; then
138-
npm run a11y:report --silent
138+
npm run a11y:html-report --silent
139139
{
140-
echo '```text'
141-
npm run a11y:report --silent
142-
echo '```'
140+
echo '## Accessibility Report'
141+
echo ''
142+
echo 'Full route-by-route HTML report: download the `ibm-a11y-reports-${{ github.run_attempt }}` artifact from this run and open `index.html`.'
143+
echo ''
144+
npm run a11y:job-summary --silent
143145
} >> "$GITHUB_STEP_SUMMARY"
144146
else
145147
echo "No accessibility reports were generated." | tee -a "$GITHUB_STEP_SUMMARY"

0 commit comments

Comments
 (0)