feat: a11y

.agents/skills/ibm-a11y-automation/SKILL.md

@@ -0,0 +1,184 @@
+---
+name: ibm-a11y-automation
+description: Run Langflow's local accessibility scanner script against frontend routes from src/frontend/src/routes.tsx and summarize the JSON report.
+---
+
+# Langflow Accessibility Scanner
+
+Use this skill when asked to scan Langflow frontend pages for accessibility issues.
+
+## Scanner
+
+Use the Python script:
+
+```bash
+uv run python scripts/a11y/a11y_scan.py \
+  --url http://localhost:3000 \
+  --routes-file scripts/a11y/a11y_routes.json \
+  --route-group static \
+  --out /tmp/langflow-a11y-report.json \
+  --markdown /tmp/langflow-a11y-report.md \
+  --html /tmp/langflow-a11y-report.html \
+  --timeout-ms 45000
+```
+
+Script options:
+
+- `--url`: base app URL, usually `http://localhost:3000`.
+- `--routes-file`: route manifest JSON file. Prefer `scripts/a11y/a11y_routes.json`.
+- `--route-group`: manifest group to scan. Default: `static`.
+- `--routes`: comma-separated route paths to scan.
+- `--route`: one route path; can be repeated instead of `--routes`.
+- `--levels`: comma-separated issue levels. Default: `violation`.
+- `--out`: JSON report path.
+- `--markdown`: optional Markdown report path.
+- `--html`: optional self-contained HTML report path.
+- `--timeout-ms`: per-route timeout.
+- `--quiet-ms`: network quiet window before scanning. Default: `1000`.
+- `--states-file`: JSON file with explicit modal/state actions.
+- `--headed`: show browser while scanning.
+
+## Route Selection
+
+Use `scripts/a11y/a11y_routes.json` as the source of truth for route selection.
+
+The normal CI/local batch is the manifest `static` group. Prefer that unless the user asks for custom, dynamic, or gated routes.
+
+Common manifest-backed command:
+
+```bash
+uv run python scripts/a11y/a11y_scan.py \
+  --url http://localhost:3000 \
+  --routes-file scripts/a11y/a11y_routes.json \
+  --route-group static \
+  --out /tmp/langflow-a11y-static.json \
+  --markdown /tmp/langflow-a11y-static.md \
+  --html /tmp/langflow-a11y-static.html
+```
+
+Dynamic routes need real IDs before scanning:
+
+- `/flow/:id/`
+- `/flow/:id/view`
+- `/playground/:id/`
+- `/assets/knowledge-bases/:sourceId/chunks`
+
+For dynamic routes, get IDs from the loaded app, API responses, or existing test data before replacing placeholders.
+
+## Examples
+
+Scan one route:
+
+```bash
+uv run python scripts/a11y/a11y_scan.py \
+  --url http://localhost:3000 \
+  --route /flows \
+  --out /tmp/langflow-a11y-flows.json
+```
+
+Scan multiple routes:
+
+```bash
+uv run python scripts/a11y/a11y_scan.py \
+  --url http://localhost:3000 \
+  --routes-file scripts/a11y/a11y_routes.json \
+  --route-group static \
+  --out /tmp/langflow-a11y-report.json \
+  --markdown /tmp/langflow-a11y-report.md \
+  --html /tmp/langflow-a11y-report.html
+```
+
+Scan more than violations:
+
+```bash
+uv run python scripts/a11y/a11y_scan.py \
+  --url http://localhost:3000 \
+  --routes-file scripts/a11y/a11y_routes.json \
+  --route-group static \
+  --levels violation,potentialviolation,recommendation \
+  --out /tmp/langflow-a11y-expanded.json
+```
+
+Scan route plus modal states:
+
+```bash
+uv run python scripts/a11y/a11y_scan.py \
+  --url http://localhost:3000 \
+  --states-file /tmp/langflow-a11y-states.json \
+  --out /tmp/langflow-a11y-modal-report.json \
+  --markdown /tmp/langflow-a11y-modal-report.md \
+  --html /tmp/langflow-a11y-modal-report.html \
+  --timeout-ms 45000
+```
+
+State file shape:
+
+```json
+[
+  {
+    "route": "/settings/global-variables",
+    "states": [
+      {
+        "name": "new-global-variable-modal",
+        "open": [
+          { "click": "[data-testid='api-key-button-store']" },
+          { "waitFor": "[role='dialog']" }
+        ],
+        "close": [
+          { "press": "Escape" },
+          { "waitForHidden": "[role='dialog']" }
+        ]
+      }
+    ]
+  }
+]
+```
+
+Supported state actions:
+
+- `{ "click": "<css selector>" }`
+- `{ "clickText": "<visible text>" }`
+- `{ "clickRole": { "role": "button", "name": "Create" } }`
+- `{ "fill": { "selector": "<css selector>", "value": "text" } }`
+- `{ "press": "Escape" }`
+- `{ "press": { "selector": "<css selector>", "key": "Enter" } }`
+- `{ "waitFor": "<css selector>" }`
+- `{ "waitForHidden": "<css selector>" }`
+- `{ "waitForText": "<visible text>" }`
+- `{ "wait": 500 }`
+
+## Report
+
+The scanner always writes JSON. It can also write Markdown and HTML for presentation.
+
+Use JSON for exact data. Use Markdown for PR comments or issues. Use HTML when the user wants a browsable report.
+
+Summarize:
+
+- report path
+- Markdown/HTML report paths, when generated
+- total issue count
+- per-route issue count
+- per-route API request count
+- per-route request failure count
+- top rule IDs
+
+Use report fields directly:
+
+- `totalIssues`
+- `results[].route`
+- `results[].state`
+- `results[].phase`
+- `results[].apiRequests`
+- `results[].requestFailures`
+- `results[].diagnostics`
+- `results[].issues[].ruleId`
+
+## Rules
+
+- Use only scanner output for findings.
+- Do not invent route names. Read `routes.tsx`.
+- Do not auto-click arbitrary buttons to find modals. Use explicit state actions.
+- Avoid destructive modal actions unless the user explicitly asks and data is safe.
+- If a route has zero API requests, mention that scan quality may be limited.
+- Ask before fixing files unless the user explicitly asks for fixes.

.github/workflows/a11y-scan.yml

@@ -0,0 +1,157 @@
+name: A11y / IBM Equal Access Scans
+
+on:
+  pull_request:
+  schedule:
+    # Run nightly at 02:00 UTC, after the nightly build window
+    - cron: "0 2 * * *"
+  workflow_dispatch:
+    inputs:
+      ref:
+        description: "(Optional) ref to checkout"
+        required: false
+        type: string
+      assert:
+        description: "Fail the run when scans differ from committed baselines"
+        required: false
+        type: boolean
+        default: false
+
+env:
+  NODE_VERSION: "22"
+  PYTHON_VERSION: "3.13"
+  # Define the directory where Playwright browsers will be installed.
+  # This path is used for caching across workflows
+  PLAYWRIGHT_BROWSERS_PATH: "ms-playwright"
+  PLAYWRIGHT_VERSION: "1.60.0"
+
+jobs:
+  a11y-scan:
+    name: Playwright scan runner
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      # Cron only fires from the default branch, so scheduled runs resolve the
+      # latest release-* branch and scan that (same pattern as nightly_build).
+      - name: Resolve Ref To Scan
+        id: resolve_ref
+        shell: bash
+        run: |
+          if [ -n "${{ inputs.ref }}" ]; then
+            REF="${{ inputs.ref }}"
+          elif [ "${{ github.event_name }}" = "schedule" ]; then
+            REF=$(git ls-remote --heads https://github.com/${{ github.repository }} 'refs/heads/release-*' \
+              | awk '{print $2}' \
+              | sed 's|refs/heads/||' \
+              | grep -E '^release-[0-9]+\.[0-9]+\.[0-9]+$' \
+              | sort -V \
+              | tail -n 1)
+            if [ -z "$REF" ]; then
+              echo "No release-* branch found in ${{ github.repository }}"
+              exit 1
+            fi
+          else
+            REF="${{ github.ref }}"
+          fi
+          echo "ref=$REF" >> "$GITHUB_OUTPUT"
+          echo "Scanning ref: $REF"
+
+      - name: Checkout Repository
+        uses: actions/checkout@v6
+        with:
+          ref: ${{ steps.resolve_ref.outputs.ref }}
+
+      - name: Setup Node.js Environment
+        uses: actions/setup-node@v6
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Get npm cache directory
+        id: npm-cache-dir
+        shell: bash
+        run: echo "dir=$(npm config get cache)" >> "$GITHUB_OUTPUT"
+
+      - name: Cache npm dependencies
+        uses: actions/cache@v5
+        continue-on-error: true
+        with:
+          path: ${{ steps.npm-cache-dir.outputs.dir }}
+          key: ${{ runner.os }}-npm-${{ env.NODE_VERSION }}-${{ hashFiles('src/frontend/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-npm-${{ env.NODE_VERSION }}-
+
+      - name: Install Frontend Dependencies
+        run: npm ci
+        working-directory: ./src/frontend
+
+      - name: Cache Playwright Browsers
+        id: cache-playwright
+        uses: actions/cache@v5
+        continue-on-error: true
+        with:
+          path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
+          key: playwright-${{ env.PLAYWRIGHT_VERSION }}-chromium-${{ runner.os }}
+          restore-keys: |
+            playwright-${{ env.PLAYWRIGHT_VERSION }}-chromium-${{ runner.os }}
+
+      - name: Install Playwright Browser Dependencies
+        if: steps.cache-playwright.outputs.cache-hit != 'true'
+        shell: bash
+        run: |
+          cd ./src/frontend
+          npx playwright install --with-deps chromium
+
+      - name: "Setup Environment"
+        uses: astral-sh/setup-uv@v6
+        with:
+          enable-cache: true
+          cache-dependency-glob: "uv.lock"
+          python-version: ${{ env.PYTHON_VERSION }}
+          prune-cache: false
+
+      - name: Install Python Dependencies
+        run: uv sync
+
+      - name: Run IBM Equal Access Scans
+        shell: bash
+        env:
+          RUN_A11Y: "true"
+          RUN_A11Y_ASSERT: ${{ github.event_name == 'workflow_dispatch' && inputs.assert && 'true' || 'false' }}
+          LANGFLOW_DEACTIVATE_TRACING: "true"
+        run: |
+          cd src/frontend
+          # Only specs that call runA11yScan produce reports; discover them
+          # so new scan hosts are picked up without editing this workflow.
+          SCAN_SPECS=$(grep -rl "runA11yScan(" tests --include="*.spec.ts" | sort)
+          echo "Specs with a11y scans:"
+          echo "$SCAN_SPECS"
+          test -n "$SCAN_SPECS"
+          npx playwright test $SCAN_SPECS --project=chromium --workers=1 --retries=2
+
+      - name: Build IBM Scan Summary
+        if: always()
+        shell: bash
+        run: |
+          cd src/frontend
+          if ls coverage/accessibility-reports/*.json > /dev/null 2>&1; then
+            npm run a11y:html-report --silent
+            {
+              echo '## Accessibility Report'
+              echo ''
+              echo 'Full route-by-route HTML report: download the `ibm-a11y-reports-${{ github.run_attempt }}` artifact from this run and open `index.html`.'
+              echo ''
+              npm run a11y:job-summary --silent
+            } >> "$GITHUB_STEP_SUMMARY"
+          else
+            echo "No accessibility reports were generated." | tee -a "$GITHUB_STEP_SUMMARY"
+          fi
+
+      - name: Upload Accessibility Reports
+        if: always()
+        uses: actions/upload-artifact@v6
+        with:
+          name: ibm-a11y-reports-${{ github.run_attempt }}
+          path: src/frontend/coverage/accessibility-reports
+          retention-days: 30
+          overwrite: true

.github/workflows/a11y-unit-tests.yml

@@ -0,0 +1,48 @@
+name: A11y / Component Unit Tests
+
+on:
+  pull_request:
+    paths:
+      - "src/frontend/**"
+      - ".github/workflows/a11y-unit-tests.yml"
+      - "Makefile.frontend"
+  workflow_call:
+    inputs:
+      ref:
+        description: "(Optional) ref to checkout"
+        required: false
+        type: string
+  workflow_dispatch:
+    inputs:
+      ref:
+        description: "(Optional) ref to checkout"
+        required: false
+        type: string
+
+env:
+  NODE_VERSION: "22"
+
+jobs:
+  a11y-unit-tests:
+    name: jest-axe
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      - name: Checkout Repository
+        uses: actions/checkout@v6
+        with:
+          ref: ${{ inputs.ref || github.ref }}
+
+      - name: Setup Node.js Environment
+        uses: actions/setup-node@v6
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+          cache: "npm"
+          cache-dependency-path: ./src/frontend/package-lock.json
+
+      # These are regression locks: every a11y unit test passes as of the
+      # component fixes in feat/a11y-unit-tests, so a failure here means a
+      # real a11y regression and blocks the PR.
+      - name: Run jest-axe unit tests
+        run: make test_frontend_a11y_unit_ci