a11y job fails looking for `proof-a11y.json`

[Peeja]

Describe the bug

In a new design system, the a11y CI job fails with `Error: ENOENT: no such file or directory, open 'proof-a11y.json'.

To Reproduce

1. Follow the Getting Started instructions to generate and set up a new design system.
2. Run CI.

Expected behavior

Something would tell me what proof-a11y.json and how to create it (or create it for me).

I assume this is just something missing in the docs?

[orther]

I'm running into the same issue. Here is the full output of the a11y job:

#!/bin/bash -eo pipefail 
node ./.circleci/proof-a11y.js 

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     3  100     3    0     0     29      0 --:--:-- --:--:-- --:--:--    29
yarn run v1.22.4
$ /home/circleci/design-system/node_modules/.bin/ds proof -u undefined --add-all --a11y --headless --skip-tests
Selenium installation [] 0% 41.6sSelenium installation [] 6% 0.2sSelenium installation [] 13% 0.2sSelenium installation [] 22% 0.2sSelenium installation [] 36% 0.1sSelenium installation [] 45% 0.1sSelenium installation [] 56% 0.1sSelenium installation [] 64% 0.1sSelenium installation [] 65% 0.1sSelenium installation [] 70% 0.1sSelenium installation [] 79% 0.0sSelenium installation [] 88% 0.0s🌟  complete     sessionId 6fd2b58ebbd696d9bd294673fa5467f4
💾  info         Sauce Labs URL undefined
🌟  complete     sessionId 8fdefb89ac684a077333fd67eb469b61
💾  info         Sauce Labs URL undefined
[proof.....] › 🚒  error        Error: element ("#storybook-preview-iframe") still not existing after 1000ms    at new WaitUntilTimeoutError (/home/circleci/design-system/node_modules/webdriverio/build/lib/utils/ErrorHandler.js:149:12)
    at /home/circleci/design-system/node_modules/webdriverio/build/lib/commands/waitUntil.js:29:19
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
fs.js:114
    throw err;
    ^

Error: ENOENT: no such file or directory, open 'proof-a11y.json'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at main (/home/circleci/design-system/.circleci/proof-a11y.js:21:8)
    at Object.<anonymous> (/home/circleci/design-system/.circleci/proof-a11y.js:53:1)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)

Exited with code exit status 1 
CircleCI received exit code 1 

[tylerkrupicka]

Hey, sorry for the lack of replies so far on this thread. This error is caused by proof, our test runner. When you run --ally it creates a proof-ally.json file with the results. If your tests crash that file may not exist. @Peeja were you using CircleCI as well? We might just need to update the CircleCI default pipeline to not try to open that file if it doesn't exist.

[Peeja]

@tylerkrupicka Yep, that was on CircleCI.