trussworks
[feat] Automated accessibility check to our development pipeline.
Does your feature request relate to a specific USWDS component?
No
What USWDS Version is this feature present in?
Any.
Is your feature request related to a problem? Please describe.
We should add an automated accessibility check to our development pipeline.
- As government websites are required to meet section 508 compliance accessibility should be a north star rather than an afterthought.
- Adding this check into the pipeline can help remediate one of the challenges when working on accessibility issues - finding out if the problem is tied to your configuration or the library
Describe the solution you'd like
- Add an automated accessibility check to our development pipeline, specially section 508 / WCAG 2.0.
- Continuous Integration tests incorporate accessibility tools like axe and errors block commits.
- An automated process set up to optimize the use of developer time.
Describe alternatives you've considered
On a previous project we used Jest-axe (app testing) Storybook a11y add on (component level testing). @gidjin, @abbyoung, or @jcbcapps may be able to shed more light on that.
Well-known solutions to consider
- Deque's aXe has changed the playing field for site reviews. Because of its flexible design, it can be executed from the command line, from within the browser, or even from a WYSIWYG editor. Being executed from the command line means that it is much easier to simply add accessibility checking to any scripts that are being run. It can be appended to a script crawling to ensure security best practices are maintained, or simply added to a continuous integration script.
- Pa11y-ci is a continuous integration (CI) runner that can test a few pages or an entire site for accessibility issues.
- pa11y-ci in their CI pipelines and generate reports as artifacts or comments with the results of the testing.
- Pa11y by default uses the HTML_CodeSniffer accessibility engine, but it is recommended to switch to Pa11y with axe, which is described in this UK Government Accessibility Manual.
- Cypress is a JavaScript testing framework that is used for automated testing. Axe core is a popular accessibility testing engine.
- Take a look at Setting up Cypress with axe for accessibility article that illustrates the setup.
- Cypress-axe
- Jest-axe / Testing with Jest on GitHub
- Nightwatch aXe-core / Testing with Nightwatch on GitHub
Additional context
- How I do automated accessibility testing for my website by Daniel Mundra
Most of this information is from the CivicAction accessibility site. They put together some really impressive documentation on this topic.
Noting that I looked into adding Playwright, but that would only be useful locally (not as a GH-run action). We could have it run locally on commit, but I think we'd rather have it run on the repo as well to ensure no a11y bugs creep in.
We already use @storybook/addon-a11y in development. @AnnaGingle are you recommending that we include it in the production version of storybook so that users looking at the deployed storybook instance can see the a11y scan results?
We do use a linter, eslint-plugin-jsx-a11y in the development process. And the description suggests using Cypress/axe, though I only know of Cypress' use as an integration test tool. Since react-uswds is a library of components, @AnnaGingle how do you see Cypress being utilized in this case?
Ah, didn't realize Cypress was integration only — I imagined this feature would either run similar to the other Vitest/Jest unit tests (e.g. expect(component).toBeAccessible) or run on more complex stories without needing to be run locally (see my Playwright experiment)
Recommendation from @JasonLin0991: https://www.deque.com/axe/core-documentation/api-documentation/