accessibility-testing

<objective> Make an application usable by people who rely on keyboards and assistive technology, and prove it with tests that run in CI. A button that passes `toBeVisible()` can still be unreachable by keyboard; a page with zero axe violations can still be impossible to operate with a screen reader. Automated tools catch 30-40% of accessibility issues — this skill covers the automated scan plus the keyboard, screen reader, and ARIA-state testing that catch the other 60-70%. </objective>

Discovery Questions

Check .agents/qa-project-context.md first — if it exists, use it as the foundation and skip anything already answered.

Requirements and compliance (sets the target level and audit obligations)

• What WCAG conformance level is required — A, AA, or AAA? AA is the practical legal default.
• What laws apply — ADA, EAA/EN 301 549, Section 508, AODA? Each maps to a WCAG level.
• Is there a VPAT or accessibility statement to maintain, or contractual a11y clauses from enterprise/government customers?

Current state (tells you whether you're auditing or preventing regressions)

• Has an audit run before? What were the findings, and what's already in the backlog?
• Does the design system carry accessibility guidance and accessible components?

Testing infrastructure (determines what you can automate vs. must do by hand)

• Is automated a11y testing already in CI?
• Which screen readers does the team test with — VoiceOver, NVDA, JAWS, TalkBack?

Core Principles

1. Automated testing catches 30-40% of issues — no more. axe-core finds missing alt text, low contrast, missing labels, and invalid ARIA. It cannot tell you whether alt text is meaningful, whether tab order is logical, or whether a custom widget is operable. Passing axe is necessary, not sufficient. Crucially, axe ships no automated rule for several WCAG 2.2 success criteria (2.4.11 focus-not-obscured, 2.5.7 dragging movements, and 2.5.8 target-size only partially) — a green axe run does not equal 2.2 AA conformance.

2. Semantic HTML first, ARIA as last resort. Native elements (<button>, <nav>, <input>, <dialog>) carry built-in semantics, keyboard behavior, and screen reader support. A <div role="button"> also needs tabindex, Enter/Space handlers, focus styles, and ARIA state — all of which a <button> gives you free. Reach for ARIA only when no native element fits.

3. Test in impact order: keyboard, screen reader, automated. Keyboard issues physically block users from features (highest impact). Screen reader issues confuse with wrong announcements. Automated checks catch the mechanical remainder. Start where the damage is worst, not where the tooling is easiest.

4. Accessibility is a quality attribute, not a feature. Test it continuously like performance or security — every new component, every PR. Retrofitting accessibility onto a finished product costs 10-100x more because inaccessible patterns get baked into the component library.

5. Test with real assistive technology. Browser DevTools and axe extensions are dev aids, not substitutes for VoiceOver (macOS/iOS), NVDA (Windows), and TalkBack (Android), which each behave differently. Reserve manual AT passes for complex custom widgets.

Automated Scanning with axe-core + Playwright

Install @axe-core/playwright pinned to an exact version from npm view @axe-core/playwright version (4.14.0 on 2026-10-21; its minor tracks axe-core's, so a new rule set arrives with every bump and must be reviewed, not auto-updated). Wrap it in a reusable checkAccessibility(page, testInfo, options) helper that filters to the WCAG tags ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa']. Require an expected URL and an application-owned readiness assertion before scanning: blank, error, and redirected pages can pass axe. Attach the full results JSON, reject a scan that evaluated no rules, and assert violations.toHaveLength(0). The exported scanAccessibility helper returns the violation summary for known-gap baselines; checkAccessibility requires zero violations. Loop it over every key page and over interactive states (modal open, menu expanded, signed-in account), not just the default load. For a third-party page you cannot fix, assert the exact set of known rule ids instead of skipping the scan, so a new violation or a fixed one changes the test.

Suppress a rule only with a documented justification (a tracking issue or inline comment) and exclude third-party widgets you don't own rather than disabling the rule globally.

See references/recipes.md for the install command, RGAA tag caveat, the full helper, the page-loop and interactive-state specs, rule suppression, and CI integration.

Manual Testing Checklist

Automated scanning is the floor. These checks need a human (or a keyboard-driven Playwright spec — see references/recipes.md for the keyboard specs).

Keyboard navigation audit

• Tab order is logical — left-to-right, top-to-bottom for LTR. No surprise focus jumps.
• All interactive elements reachable via Tab / Shift+Tab.
• Focus indicator visible on every focused element. No outline: none without a replacement.
• Skip link works — first Tab reveals "Skip to main content"; Enter moves focus to <main>.
• Enter activates buttons/links; Space activates buttons and toggles checkboxes.
• Escape closes modals, dropdowns, tooltips; focus returns to the trigger.
• Arrow keys navigate within tabs, menus, radio groups, tree views.
• No keyboard traps (modal dialogs intentionally trap until dismissed — that's allowed).
• Custom widgets operable without a mouse (sliders, date pickers, drag-and-drop).

Screen reader testing

• Page title announced on navigation.
• Headings form a navigable outline (h1 → h2 → h3, no skipped levels).
• Images have descriptive alt text (or alt="" for decorative).
• Form inputs announce their labels when focused.
• Required fields announced as required; errors associated with their input.
• Live regions announce dynamic content (toasts, loading states).
• Buttons/links announce their purpose (no "click here").

Color contrast and visual

• Normal text: 4.5:1 minimum (WCAG AA). Large text (18pt+ / 14pt+ bold): 3:1.
• UI components and graphical objects: 3:1 against adjacent colors.
• Information never conveyed by color alone — add icons, patterns, or text.

Form and error accessibility

• Every input has a visible <label> tied via for/id (placeholder is not a label).
• Required fields indicated visually and programmatically (required / aria-required).
• Errors use aria-describedby to link to the input and role="alert" to announce.
• Focus moves to the first error on submission failure.
• Related fields grouped with <fieldset> and <legend>.

WCAG 2.2 Quick Reference

Level A (must fix)

Level AA (most common legal requirement)

Level AAA (nice to have)

Accessible Patterns

Test on the accessible tree (roles, names, ARIA state), not on CSS. The patterns you need runnable tests for:

• Forms — error linked via aria-describedby, aria-invalid='true', focus on first error.
• Modal/dialog — aria-modal='true', aria-labelledby, focus trapped, Escape returns focus.
• Interactive states — opened dropdown (role="menu" or role="listbox", aria-expanded), loading skeleton (aria-busy='true' during fetch), toast (aria-live='polite'). These carry different ARIA per state, so click to trigger the state change and assert the open-state ARIA — the default page snapshot never exercises them.
• Data tables — columnheader roles, aria-sort reflects the active sort.
• Landmarks — exactly one main; banner, navigation, contentinfo present.

See references/patterns.md for the full runnable tests for every pattern above.

ARIA Snapshots

Playwright's toMatchAriaSnapshot() captures the accessible tree as YAML and asserts against it — the fastest way to catch a regression where a visual change silently breaks semantics (a <div> restyled to look like a button, a heading demoted to plain text). It checks structure and accessible names, not pixels, so it's complementary to visual-testing, not a replacement. Scope snapshots to a stable container; whole-page snapshots over async content go flaky. See references/patterns.md for navigation and form snapshot examples.

Legal Compliance Mapping

Practical target: if you serve US or EU users, WCAG 2.2 AA is the target for new development — the EAA is in force, EN 301 549 is expected to update to 2.2, and ISO/IEC 40500:2025 (published Sept 2025) codifies WCAG 2.2 internationally. WCAG 2.1 AA is the legacy minimum where 2.2 can't be reached immediately.

WCAG 3 status: W3C published an updated WCAG 3 working draft in March 2026 that renamed "Outcomes" to "Requirements" and moved away from binary pass/fail grading; it lists ~174 requirements. It remains a working draft — Candidate Recommendation is targeted for Q4 2027 and a Recommendation not before 2028. Plan for WCAG 2.2 today; track WCAG 3 but do not test against it yet.

Audit evidence to collect: automated scan results per page, manual checklists with tester/date, screen reader results with AT versions, accessibility statement, VPAT for enterprise sales, and a remediation plan for known issues. For the full legal/VPAT/consent mapping, see compliance-testing.

Anti-Patterns

Only automated testing

Running axe, finding zero violations, and declaring the product accessible. Automated tools miss 60-70% of real issues and skip several WCAG 2.2 criteria entirely. Fix: pair every axe run with the keyboard and screen reader checklist; gate releases on both, not just the scan.

ARIA overuse

Adding role, aria-label, and aria-describedby to elements that already have native semantics, creating double announcements. Fix: delete the redundant ARIA and use the native element — a <button> never needs role="button".

Ignoring keyboard users

Features that work by mouse and touch but not keyboard — click-only dropdowns, drag-and-drop with no keyboard path, hover-only tooltips. Fix: give every mouse interaction a keyboard equivalent and cover it with a keyboard.spec.ts test (see references/recipes.md).

Retrofitting accessibility

Waiting until the product is "finished," by which point inaccessible patterns are baked into the component library at 10-100x the fix cost. Fix: add an axe check to the Definition of Done so every new component is gated on accessibility before merge.

Treating accessibility as optional

Deprioritizing a11y tickets because "nobody complained" — users with disabilities can't complain through a product they can't use, so they leave silently, and US web-accessibility lawsuits have climbed year over year since 2018. Fix: track a11y as a release blocker with the same severity rules as functional bugs, and report open a11y issues in the release readiness check.

Testing only the happy path

Scanning only the default page state, missing the modals, expanded dropdowns, error messages, and loading skeletons that carry different ARIA. Fix: drive each interactive state in the test (click to open the menu, trigger the fetch) and re-run the scan against the changed DOM.

Failure Modes

Verification

Prove the suite actually exercises the page — an a11y test that passes vacuously (wrong URL, axe scanning an error page, snapshot never reached) is worse than none.

1. Confirm axe is scanning real content. Point the helper at a page you know has a violation (e.g. temporarily remove a <label>) and run it — the test must FAIL and name the rule:

   npx playwright test e2e/tests/a11y/pages.spec.ts
   

   Copy
   If a page with a planted defect still passes, AxeBuilder is scanning the wrong DOM (redirect, blank page, or wrong selector) — fix that before trusting any green run.
2. Confirm the keyboard specs reach the app, not a 404. Run npx playwright test e2e/tests/a11y/keyboard.spec.ts and open the trace for the skip-link test — the first Tab should land on the skip link, not nowhere. A test that "passes" because the page never loaded is a false green.
3. Confirm the CI gate blocks. Introduce one serious violation on a branch and push — the a11y job must exit non-zero and fail the PR check. Revert after.
4. Spot-check an ARIA snapshot. Run toMatchAriaSnapshot once with --update-snapshots, then again without — the second run must pass. If it flakes, the snapshot scope includes async/reordering content; narrow it to a stable container.

Done When

• axe-core integrated into the E2E suite and run automatically on all key user-facing pages identified in the test strategy.
• CI reports zero critical or serious axe violations and blocks merge when any are introduced (see references/recipes.md for the workflow).
• Keyboard navigation tested end-to-end for all interactive flows (forms, modals, dropdowns, navigation menus).
• Interactive-state ARIA verified for at least one dropdown/menu (aria-expanded + role), one loading region (aria-busy), and one live region (aria-live).
• Color contrast validated for the full brand palette against WCAG AA thresholds (4.5:1 normal text, 3:1 large text and UI components).
• Screen reader test notes documented for complex custom widgets (date pickers, data tables, drag-and-drop), including which screen reader and version was used.

Related Skills

• playwright-automation — the test runner for both axe scans and keyboard/ARIA snapshot tests; this skill adds the accessibility-specific patterns on top.
• compliance-testing — legal/regulatory testing including cookie consent (GDPR/CMP), VPAT generation, and EAA/Section 508 reporting. Go there for consent banners and formal compliance documentation; stay here for WCAG conformance testing.
• visual-testing — pixel-diff screenshot regression. Use it for visual rendering changes; use this skill's ARIA snapshots for semantic-tree regressions and contrast for the a11y-specific color thresholds.
• ci-cd-integration — running a11y tests in CI and blocking merges on violations.
• risk-based-testing — prioritizes which pages and components to audit first.