Runtime issue: Axe is already running. Use `await axe.run()` to wait for the previous run to finish before starting a new run.

[dNicolle]

<< Please describe how you encountered this issue >>

axe-core version: undefined axe-extension version: 4.24.5

it@moz-extension://a1c77a94-b956-419f-b37e-7d85e421f84d/axe-versions/latest/axe.js:12:78706
e/</axe.run@moz-extension://a1c77a94-b956-419f-b37e-7d85e421f84d/axe-versions/latest/axe.js:12:355675
48269/<@moz-extension://a1c77a94-b956-419f-b37e-7d85e421f84d/content.bundle.js:1:21730
async*72257/h</<@moz-extension://a1c77a94-b956-419f-b37e-7d85e421f84d/content.bundle.js:61:5845

[straker]

Thanks for the issue. Are you able to share the url that this error occurred, or maybe a code snippet that causes the error?

[dNicolle]

It's seam's to be an incompatibility with an other extension (colorzilla). When i haved use Axe after have disabled her and reload firefox, il was ok.

[straker]

Thanks for the info. I'll let the appropriate team know and we'll investigate the problem.

[professorplumb]

Ran into this same problem; in my case the issue was that I'd wrapped axe.run in an async function, but forgotten to call that function with await in 2+ instances, which then clashed when run together.

Commenting here in case others find this through Google like I did and happened to make the same mistake.

[priyang12]

I am facing the same error. I am using jest-axe for testing and I got catch this error in the block when running multiple tests.

jest-axe@^6.0.0:
  version "6.0.0"
  resolved "https://registry.yarnpkg.com/jest-axe/-/jest-axe-6.0.0.tgz#e5ca8e9a0363c8c5f4e00760317bd7d7d29b90b4"
  integrity sha512-gAh/2zoWii4Rbhe6IUIo5TTiseGJDCitFnDFwBNpIuaOciyQgVZue6jtd4W7oMoKHewKoRSuIol7t/MuGx+mqg==
  dependencies:
    axe-core "4.4.1"
    chalk "4.1.0"
    jest-matcher-utils "27.0.2"
    lodash.merge "4.6.2"

Should I update the version?

[AlanSl]

Is there any solution for this issue when multiple tests are run concurrently? For example:

• In a monorepo where multiple packages use axe-core (or a dependent like jest-axe)
• Within one repo / package where the same tests are run concurrently with different env vars or contexts

[w-a-t-s-o-n]

@AlanSl did you find a solution? I believe this is the problem I'm having as I only see it pop up when two packages within a Turbo monorepo are running tests that use jest-axe concurrently.

[AlanSl]

@w-a-t-s-o-n I'm afraid not, it seemed to be a side effect of Axe's origins as a bookmarklet-like client-side script that it assumes it'll be used in a one-shot single thread context.

I'm no longer working on the project where I saw this issue and I didn't have much time to dig deeper; the only options I can really see are:

• Work around it by moving axe tests to a separate no-concurrency runner and design CI around the face that the axe tests task will be a slow, long-running task (e.g. only run it after a branch is pushed?). A possible bonus of this is, if the axe tests are running on their own, they're less likely to hit flakey timeouts than they do when they're mixed in a lot of parallel processes that might unpredictably make them slow (we had a lot of problems with timeouts in the same project, even after bumping the timeout limits very high for axe tests).
• Maybe, try to fix it by somehow creating a new instance of the underlying axe object for each potentially-parallel test. Like maybe do a dynamic import or deep clone in a beforeEach, instead of always using the same static import whereby every test references the same global object?

[w-a-t-s-o-n]

@AlanSl Yeah that's a little disappointing but makes sense! I'm working on a monorepo that's meant to scale to include many, large projects, so I'm definitely hesitant to add an extra CI step. Your second solution seems interesting and I'll probably give it a go. We use Chromatic as well so I might see if it's feasible to only run a11y tests in their platform.

[davidyoung77]

@AlanSl Yeah that's a little disappointing but makes sense! I'm working on a monorepo that's meant to scale to include many, large projects, so I'm definitely hesitant to add an extra CI step. Your second solution seems interesting and I'll probably give it a go. We use Chromatic as well so I might see if it's feasible to only run a11y tests in their platform.

@AlanSl did you figure out a solution to this? I just encountered this same problem. This short term solution I did is to run the tests sequentially with --runInBand, and increase the jest timeout to 20 seconds, this is not a long term solution though.

[benkeen]

Hey folks! We just encountered this same issue. We'd just upgraded to jest 29 and axe-core 4.7.0, from jest 26 and axe-core 4.4.3. I'll need more time making real sense of what's going on, but I was able to whittle it down to this block. For all the successful tests, the resolve/rejects are firing in that block, but in the case where we were seeing this error, nothing fired. No errors, no nothing - it just reached the end of the block. Since the promise isn't resolved/rejected, the calling code thinks axe.run() is still running, so on the next test that runs, it throws the error described in this ticket. I can't speak to why it's occurring for us yet, or what's different about some of our tests. Running the test individually works fine, it's only when run en-masse does that particular one consistently fail + cause this error on all subsequent tests.

I found a quick workaround hack was adding a resolve([]); after line 509, just to ensure the promise was resolved in all cases. Once that was added, our tests were green again. It's obviously not a viable solution though. I'll keep looking at it to see if I can make more sense of it.

[benkeen]

A little more info. In our case it's due to the DOM content in the problematic tests being particularly large. It doesn't appear to be the specifics of the DOM content itself - commenting out different areas allows them to pass, it's only when it hits a certain size does this problem show up.

[WilcoFiers]

I'm not convinced axe-core ever should support parallel runs. The problem isn't years-old architectural decisions. It's that axe doesn't support changing the page while it's running, which is the only real use case I know for wanting parallel runs at all. Axe-core caches things like selectors, CSS properties, bounding boxes, ID references, etc. These are too slow to do over and over again. It's a 10x performance loss not to have these things cached. If the page changes as axe is running, the cache becomes invalid. It also introduces race conditions that will make the results unpredictable.

Even if you guaranteed the element under test isn't changed, elements outside the context of a run do impact the results of a run. Think of changing heading order, id labels disappearing, layout shifts causing an element's position to change, even the computed styles of an element can change by having other elements added or removed from the DOM. It just doesn't work.

Since jest-axe does work in a more constrained environment than what axe-core needs to support there might be things they can do that can't be put into axe-core itself. I would recommend opening an issue with them for it. If there's a solution that we could easily support in axe-core we might consider it. Perhaps running the rules synchronously is the answer here. That's a non-trivial thing to create, but now that axe has its runVirtualRule method it's definitely possible.

[benkeen]

To be clear, this isn't a parallelism issue - keeping it sequential is totally fine. When this problem occurs for us (it's still an issue) it's just because the previous run hadn't been marked as "done" (pass or fail) internally within axe-core, so the next run thinks it's being ran in parallel, which isn't the case, and it throws this error. What I've been unable to figure out is why whatever is failing doesn't get picked up by axe-core so it can mark the previous test as failed & why it's left in an invalid state internally. I'm hoping to get more time to return to this problem soon because it's blocked our jest upgrade.

[michaelfaith]

@benkeen were you able to make any progress on this by chance?

[benkeen]

Well, yes and no. I ended up upgrading our monorepo piecemeal - updating the packages to the latest jest that didn't have axe tests first, then doing them last. In the end, when I updated those final packages the problem went away. That suggested it was some sort of package dependency problem, but I don't think so... the problem occur every now and then, even now. Just haven't been able to identify what triggers it. Sorry. :(

[oheyjesse]

We see this issue ourselves when using the axe-core/react plugin on our react development environments. We've got this set up to only initialize in the development environment, rather than production, but still, It seems to occur in dev when our page hot reloads after a change, or something similar- I can only assume that because the process is running, it doesn't exit gracefully. It's been hard to nail down, as it's intermittent- but it's seemingly occurring in more than just jest testing runs in CI.

Screenshot of webpack's fullscreen overlay error. Interestingly, even though this error fires, it appears the axe react tooling continues to run - almost as if it recovers after the error fires.

[WilcoFiers]

Agreed, axe-core/react races axe-core against its debounce timer. That's a problem that should be fixed. I've opened an issue on axe-core/react. On jest-axe again, that's not something we can fix in axe-core.

To be clear, this isn't a parallelism issue - keeping it sequential is totally fine. When this problem occurs for us (it's still an issue) it's just because the previous run hadn't been marked as "done" (pass or fail) internally within axe-core, so the next run thinks it's being ran in parallel, which isn't the case, and it throws this error

The only way to get this error is if you don't wait for a run to resolve with the results before starting the next one (or if you override axe._running). Here's the code for it: https://github.com/dequelabs/axe-core/blob/af0b548a61d7156f5e71194f8f2172ce6d5521fa/lib/core/public/run.js#L40-L43

One way you could run into that would be if you use the callback rather than await. For example the following could cause that error. By the time axe-core invokes the first callback the test runner may already have started run axe again. The way to avoid these issues is to use async/await

// RACE CONDITION! DO NOT USE THIS CODE
it('runs axe', () => {
  axe.run((results) => console.log(results))
}):
it('runs axe again', () => {
  axe.run((results) => console.log(results))
}):

[catandthemachines]

I found this issue fixed itself when I changed my jest-axe package from ^8.0.0 to ^6.0.0