Speeding up Testaro tests with Promise.all

[tgrushka]

Hi Jonathan,

Thank you and CVS for such a great test suite! I'd like to make a small contribution in terms of test speed on some of the Testaro tests.

The caveat is that this improvement can only be made on tests that are not affected by:

• order of execution;
• concurrent execution;
• interaction within the each locator loop using the Playwright API such as await page.keyboard.press (e.g. as in focAll.js).

Taking focOp.js as an example, this test inspects each focusable element (with tabIndex === 0 ... another question: why not tabIndex >= 0? I know, setting tabIndex to a value > 0 is bad practice but another test could be added for focus order ... at least tabIndex > 0 elements I believe are still keyboard focusable...). The isOperable(loc) just inspects the computedStyle and other props of the element. Therefore, we should be able to run these in parallel.

We can simply replace:

  // For each locator:
  for (const loc of all.allLocs) {
  ...
  }

with:

  // For each locator:
  await Promise.all(
    all.allLocs.map(async loc => {
    ...
    })
  )

On a page with a sample size of 100, running only the loop 20 times, I got the following averages:

• 4295 ms running consecutively
• 413 ms running concurrently

which is roughly 10.4 times faster.

Another example, focInd: On a page with a sample size of 100, running only the loop 20 times, I got the following averages:

• 4120 ms running consecutively
• 880 ms running concurrently

which is roughly 4.7 times faster.

Should add up significantly over multiple tests and thousands of pages.

A more involved step would be to bypass the allLocs altogether, and thus serialization/deserialization overhead between node and chromium, and just put the entire test loop into a single page.evaluate call with a document.querySelectorAll.

Any reason this could be a problem that I haven't considered?