Commit 14dafba
fix(docs): resolve documentation site accessibility defects and extend the runtime harness (microsoft#2566)
# fix(docs): resolve documentation site accessibility defects and extend
the runtime harness
## Description
This branch closes out the documentation site accessibility follow-ups
and extends the accessibility skill's runtime harness with the machinery
those fixes needed. Three of the changes are genuine product defects on
the published docs site; the rest is permanent regression coverage plus
the harness work that made the defects reproducible in the first place.
> A recurring theme in this work: several behaviors previously
classified as "manual only" turned out to be automatable once the right
measurement was found. Text zoom is the clearest example — it had been
recorded as undriveable from Playwright, but browser zoom is equivalent
to a scaled viewport at a matching `deviceScaleFactor`, which makes it a
deterministic matrix.
### Documentation site fixes
Three defects were reproduced against a locally served build before any
fix was written, and each fix is now locked by a test proven to fail
without it.
- Fixed the **search clear button** in *src/theme/SearchBar/index.jsx*.
The swizzled component looked for `button[type="reset"],
button[class*="clear"]`, but the upstream theme renders the control as
`<button class="searchClearButton_qk4g">` with no `type` attribute. CSS
attribute matching is case-sensitive, so `[class*="clear"]` never
matched `searchClearButton`. The selector now carries the
case-insensitive flag.
- This one had been masked: the product code and its test shared the
same faulty selector, so the test took a skip branch instead of failing.
- Fixed **navbar overlap at narrow viewports** in *src/css/custom.css*.
The search plugin's container is absolutely positioned and sits outside
the flex flow, so no flex rule on its siblings could resolve the
overlap. The brand title is now visually hidden below 420px while
retaining its accessible name.
- Fixed **search placeholder clipping under text zoom** in the same
file. The keyboard shortcut badge is hidden below 996px, which restores
usable width for the placeholder.
### Regression coverage
- Added *e2e/text-zoom.spec.ts*, which exercises a zoom matrix from 100%
through 250% by pairing scaled viewports with matching device scale
factors, then measures required placeholder width against usable width.
- Added table, search keyboard, arrow navigation, and search page status
specs.
- Extended *e2e/contrast.spec.ts* so every prose link must carry a
non-color cue, and *e2e/focus-management.spec.ts* so every visible
focusable draws an indicator of at least 2 CSS pixels.
- Added `waitForHydration` to *e2e/_helpers/a11yInvariants.ts* and
applied it across twelve specs. Docusaurus sets `data-has-hydrated` on
the document element after React hydration; several specs were asserting
against click handlers, focus styles, and axe results before that point,
which is what produced the intermittent failures.
### Accessibility skill runtime harness
- Split AT case execution, calibration, and visual review into discrete
modules under *scripts/runtime_a11y/runner/*, replacing a single
monolithic path.
- Added an **ARIA-AT catalog** with a JSON Schema and SHA-pinned
upstream provenance covering five patterns. The public posture is
deliberately conservative: entries default to manual-only and JAWS
remains human-led.
- Added `render-artifacts`, `capture-visual-review`, `run-calibration`,
and `run-at-plan` subcommands, along with output-path containment
checks.
- Fixed a **stray browser window leak**. The harness previously invoked
`chrome.exe --version` to read the browser version; on Windows that
opens a window rather than printing and exiting, and the call was made
twice in one expression. Version detection now reads file metadata
instead.
- Introduced a **method-adequacy taxonomy** distinguishing criteria that
static analysis can decide from those it can only inform, with CI
enforcement graduated by tier.
### Documentation
- Added a manual validation guide under *docs/contributing/* organizing
per-behavior steps into seven workstreams with a traceability register
and evidence template.
- Added a shared real screen reader testing runbook under
*docs/planning/runbooks/accessibility/*.
## Related Issue(s)
None
## Type of Change
Select all that apply:
**Code & Documentation:**
* [x] Bug fix (non-breaking change fixing an issue)
* [x] New feature (non-breaking change adding functionality)
* [ ] Breaking change (fix or feature causing existing functionality to
change)
* [x] Documentation update
**Infrastructure & Configuration:**
* [ ] GitHub Actions workflow
* [ ] Linting configuration (markdown, PowerShell, etc.)
* [ ] Security configuration
* [ ] DevContainer configuration
* [ ] Dependency update
**AI Artifacts:**
* [ ] Reviewed contribution with `prompt-builder` agent and addressed
all feedback
* [x] Copilot instructions (`.github/instructions/*.instructions.md`)
* [x] Copilot prompt (`.github/prompts/*.prompt.md`)
* [x] Copilot agent (`.github/agents/*.agent.md`)
* [x] Copilot skill (`.github/skills/*/SKILL.md`)
* [ ] Copilot hook (`.github/hooks/*/*.json`)
* [ ] Eval spec added/updated for changed AI artifacts (`evals/`)
> Note for AI Artifact Contributors:
>
> * Agents: Research, indexing/referencing other project (using standard
VS Code GitHub Copilot/MCP tools), planning, and general implementation
agents likely already exist. Review `.github/agents/` before creating
new ones.
> * Skills: Must include both bash and PowerShell scripts. See
[Skills](../docs/contributing/skills.md).
> * Model Versions: Only contributions targeting the **latest Anthropic
and OpenAI models** will be accepted. Older model versions (e.g.,
GPT-3.5, Claude 3) will be rejected.
> * See [Agents Not
Accepted](../docs/contributing/custom-agents.md#agents-not-accepted) and
[Model Version
Requirements](../docs/contributing/ai-artifacts-common.md#model-version-requirements).
**Other:**
* [x] Script/automation (`.ps1`, `.sh`, `.py`)
* [ ] Other (please describe):
## Sample Prompts (for AI Artifact Contributions)
**User Request:**
"Generate the accessibility coverage matrix for this project and render
the evidence bundle."
**Execution Flow:**
The coverage matrix prompt builds a surface inventory across surfaces
and interaction states, then classifies each criterion by method
adequacy — whether static analysis can decide it or only inform it. It
invokes the runtime harness `render-artifacts` subcommand to produce the
evidence bundle. Criteria in the announcement, interaction, and
adaptive-rendering classes are routed to manual verification rather than
reported as automated passes. Unresolved assistive-technology work is
routed to the shared real screen reader runbook.
**Output Artifacts:**
A six-file bundle: coverage matrix as JSON and Markdown, an EARL JSON-LD
report, manual test plans in Markdown and YAML, and a manifest. For
example, the manual plan opens with the surface and state inventory,
followed by per-criterion steps and an evidence template with a
four-value result vocabulary.
**Success Indicators:**
The manifest lists all six artifacts, the EARL report maps informs-only
methods to `earl:cantTell` rather than a pass, and every criterion that
cannot be decided statically appears in the manual plan rather than
being silently marked as covered.
For detailed contribution requirements, see:
* Common Standards:
[docs/contributing/ai-artifacts-common.md](../docs/contributing/ai-artifacts-common.md)
- Shared standards for XML blocks, markdown quality, RFC 2119,
validation, and testing
* Agents:
[docs/contributing/custom-agents.md](../docs/contributing/custom-agents.md)
- Agent configurations with tools and behavior patterns
* Prompts:
[docs/contributing/prompts.md](../docs/contributing/prompts.md) -
Workflow-specific guidance with template variables
* Instructions:
[docs/contributing/instructions.md](../docs/contributing/instructions.md)
- Technology-specific standards with glob patterns
* Skills: [docs/contributing/skills.md](../docs/contributing/skills.md)
- Task execution utilities with cross-platform scripts
## Testing
Automated validation captured on this branch **before** the most recent
merge from `main`:
| Check | Result |
|-------|--------|
| `npm run test:node` | Passed, 120 of 120 |
| Playwright e2e suite | Passed, 248 passed and 2 skipped |
| Docusaurus `npm run typecheck` | Passed |
| `npm run lint:md` | Passed |
| `npm run lint:frontmatter` | Passed, 823 files with no errors or
warnings |
Re-run **after** the merge, on the tree as proposed:
| Check | Result |
|-------|--------|
| `npm run lint:frontmatter` | Passed |
| `npm run validate:skills` | Passed, 56 skills with no errors or
warnings |
Each new lock was proven non-vacuous by confirming it fails before the
corresponding fix and passes after. The contrast lock moved from 0 to 19
offenders when the fix was reverted; the focus indicator lock moved from
0 to 77; the zoom matrix moved from 4 passing to 2 failing.
The e2e suite went from 217 passing with 16 skips to 248 passing with 2
skips, and full-suite runtime dropped from 12.0 minutes with 4 failures
to 2.9 minutes with none.
> [!IMPORTANT]
> The Node-based checks could not be re-run after the merge. A TLS
handshake failure between this workstation and the public npm registry
currently blocks dependency installation, which leaves both the
repository root and `docs/docusaurus` without a complete `node_modules`.
That is an environment fault on the authoring machine, not a defect in
this branch. CI installs from a clean environment and should be treated
as the authoritative signal for every unchecked item below.
Manual testing has not yet been performed. A manual validation pass
covering the remaining behaviors is planned as a follow-up using the
runbook added in this branch.
Security review of the diff found no secrets, tokens, credentials, or
personal data, and no non-inclusive language. All new Python and Node
modules carry Microsoft copyright and SPDX identifiers. The skill-local
lockfile resolves from the public npm registry.
## Checklist
### Required Checks
* [x] Documentation is updated (if applicable)
* [x] Files follow existing naming conventions
* [x] Changes are backwards compatible (if applicable)
* [x] Tests added for new functionality (if applicable)
### AI Artifact Contributions
<!-- If contributing an agent, prompt, instruction, or skill, complete
these checks -->
* [ ] Used `/prompt-analyze` to review contribution
* [ ] Addressed all feedback from `prompt-builder` review
* [ ] Verified contribution follows common standards and type-specific
requirements
### Required Automated Checks
The following validation commands must pass before merging:
* [ ] Markdown linting: `npm run lint:md`
* [ ] Spell checking: `npm run spell-check`
* [x] Frontmatter validation: `npm run lint:frontmatter`
* [x] Skill structure validation: `npm run validate:skills`
* [ ] Link validation: `npm run lint:md-links`
* [ ] PowerShell analysis: `npm run lint:ps`
* [ ] Eval spec schema and coverage (if AI artifacts changed): `npm run
eval:lint:schema`
* [ ] Plugin freshness: `npm run plugin:generate`
* [ ] Docusaurus tests: `npm run docs:test`
## Security Considerations
<!-- 1 parent 720ea72 commit 14dafba
197 files changed
Lines changed: 27562 additions & 548 deletions
File tree
- .github
- agents/accessibility
- subagents
- instructions/accessibility
- prompts/accessibility
- skills/accessibility/accessibility
- references
- ci
- frameworks
- phases
- scripts/runtime_a11y
- matrix
- runner
- drivers
- tests/runtime_a11y
- fixtures
- runner
- probe-smoke
- workflows
- docs
- agents
- accessibility
- code-review
- contributing
- customization
- docusaurus
- e2e
- _helpers
- screen-reader
- plugins
- src
- css
- data
- __tests__
- theme
- DocSidebarItem/Category
- Layout
- SearchBar
- SearchPage
- static
- ns/accessibility
- planning
- runbooks/accessibility
- evals/agent-behavior
- stimuli
- scripts
- ci
- evals/Modules
- lib/Modules
- linting
- schemas
- tests
- linting
- security
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
| 131 | + | |
| 132 | + | |
131 | 133 | | |
132 | 134 | | |
133 | 135 | | |
| |||
195 | 197 | | |
196 | 198 | | |
197 | 199 | | |
| 200 | + | |
198 | 201 | | |
199 | 202 | | |
200 | 203 | | |
201 | 204 | | |
202 | 205 | | |
203 | 206 | | |
204 | 207 | | |
| 208 | + | |
205 | 209 | | |
206 | 210 | | |
207 | 211 | | |
208 | 212 | | |
209 | 213 | | |
210 | 214 | | |
211 | 215 | | |
| 216 | + | |
212 | 217 | | |
213 | 218 | | |
214 | 219 | | |
| |||
Lines changed: 1 addition & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
| 26 | + | |
36 | 27 | | |
37 | 28 | | |
38 | 29 | | |
| |||
Lines changed: 9 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
| |||
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| 42 | + | |
41 | 43 | | |
42 | 44 | | |
43 | 45 | | |
| |||
84 | 86 | | |
85 | 87 | | |
86 | 88 | | |
| 89 | + | |
| 90 | + | |
87 | 91 | | |
88 | 92 | | |
89 | 93 | | |
90 | 94 | | |
91 | 95 | | |
92 | | - | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
93 | 99 | | |
94 | 100 | | |
95 | 101 | | |
| |||
114 | 120 | | |
115 | 121 | | |
116 | 122 | | |
117 | | - | |
| 123 | + | |
| 124 | + | |
Lines changed: 20 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
60 | 69 | | |
61 | 70 | | |
62 | 71 | | |
| |||
162 | 171 | | |
163 | 172 | | |
164 | 173 | | |
165 | | - | |
166 | | - | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
167 | 177 | | |
168 | 178 | | |
169 | 179 | | |
| |||
175 | 185 | | |
176 | 186 | | |
177 | 187 | | |
178 | | - | |
179 | | - | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
180 | 191 | | |
181 | 192 | | |
182 | 193 | | |
| |||
198 | 209 | | |
199 | 210 | | |
200 | 211 | | |
201 | | - | |
202 | | - | |
203 | | - | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
204 | 217 | | |
205 | 218 | | |
206 | 219 | | |
| |||
0 commit comments