Testcafe fails to run Angular 17/18 tests on Firefox

[equipatuequipo]

trafficstars

What is your Scenario?

I'm trying to create tests for an Angular 18 app.

What is the Current behavior?

The tests fail while running them on Firefox, the browser console shows an error related to observables.

Digging a little bit deeper I found out the issue comes from hammerhead, a part of Testcafe:

https://github.com/DevExpress/testcafe-hammerhead/blob/dcc14065ec63616c6aff9fdb01a663a58f41052a/src/processing/script/header.ts#L55

What is the Expected behavior?

The tests to work in all browsers

What is the public URL of the test page? (attach your complete example)

Given that you need a browser to run the tests this only can be tested locally. I've setup a repo with instructions to reproduce: https://github.com/equipatuequipo/testcafe-angular-18

What is your TestCafe test code?

import { Selector, fixture, test } from 'testcafe';

fixture`Error example`.page`./`;

test('Empty page', async (t) => {
  await t.expect(Selector('h1').innerText).eql('Test');
});

Your complete configuration file

/** @type {TestCafeConfigurationOptions} */
module.exports = {
  src: "src/tests/**/*.test.ts",
  browsers: ["firefox"],
  baseUrl: "http://localhost:4200",
  debugMode: true,
  speed: 0.5,
};

Your complete test report

The test does not finish, the error is visible in the browser's console:

Screenshots

No response

Steps to Reproduce

1. Clone the example repo
2. Install all dependencies: npm i
3. Start the webserver: npm start
4. Run the tests in a separate terminal: npm run test
5. Check the browser's console

TestCafe version

3.6.0

Node.js version

20.11.0

Command-line arguments

None, all the config is in .testcaferc.js

Browser name(s) and version(s)

Firefox 127

Platform(s) and version(s)

Windows 10

Other

Tested on Angular 17 and 18, problem persists.

[github-actions[bot]]

We appreciate you taking the time to share information about this issue. We reproduced the bug and added this ticket to our internal task queue. We'll update this thread once we have news.

[Bayheck]

Hello,

After taking a closer look at the issue, we found out that TestCafe has a technical limitation when used with the rxjs library. TestCafe uses testcafe-hammerhead to proxy your page, and rxjs has an operator function named 'window', which interferes and replaces the global window. It breaks hammerhead attached methods like __get$ and __set$, which results in the issue you are facing.

When you use a function from the rxjs library like 'map' in your example, the whole module is sent to the client with the above mentioned 'window' operator function, which causes the issue. As a workaround, try using an alternative package for the rxjs 'map' function in your application.

[equipatuequipo]

Hello, Thanks so much for your response! I'm a bit confused about how this issue only occurs in Firefox, while everything works perfectly in Chrome. If what you mentioned were the case, wouldn't it affect all browsers? Regarding the alternative you suggested, I've tried five different libraries, but none have managed to work properly with observables. Do you have any other suggestions? Thanks again for your help!

[Bayheck]

Hello,

The issue occurs only in Firefox because Testcafe has Native Automation mode only in Chromium-based browsers.

https://testcafe.io/documentation/404237/guides/intermediate-guides/native-automation-mode

Could you please illustrate your attempts with alternative libraries using the example repo you shared before?

It is possible there is another issue.

[equipatuequipo]

Hello,

We have tested the map function from the following libraries:

• lodash
• ramda
• observable-fns
• rxjs-operators
• Our own implementation

None of them worked correctly (they gave an error in Angular). rxjs is the library that Angular uses internally for observables, and it seems there is no alternative to this.

[Bayheck]

Hello, this is a temporary technical limitation of Testcafe in Firefox.

For now, I recommend to use chromium based browsers with enabled Native Automation.

Stay tuned for any updates.

[TheGameKnave]

This error is also present in Safari and Opera. Strangely, it works on Edge (all on OSX Ventura).

Really looking forward to a fix, as I adopted TestCafe on the promise of supporting all browsers.

TestCafe supports a wide array of modern browsers. The latest versions of the following browsers work without any extra configuration:

[sorin-davidoi]

After taking a closer look at the issue, we found out that TestCafe has a technical limitation when used with the rxjs library. TestCafe uses testcafe-hammerhead to proxy your page, and rxjs has an operator function named 'window', which interferes and replaces the global window. It breaks hammerhead attached methods like __get$ and __set$, which results in the issue you are facing.

I think this can be fixed by tweaking the environment detection code. The actual window object should be accessed via globalThis.window in order to avoid variables shadowing it.

I can confirm that applying this change in https://github.com/DevExpress/testcafe-hammerhead/blob/dcc14065ec63616c6aff9fdb01a663a58f41052a/src/processing/script/header.ts#L55 solves this for my particular use-case.

Support for globalThis looks good both in browsers and in NodeJS, so we should be able to use it without worrying about breaking compatibility.

The one thing I'm not sure if this change needs to be applied to other parts of the codebase. @Bayheck would you accept a patch that does this change in header.ts (and possibly other places)?

[Bayheck]

Thank you for your help. Replacing the window with a globalThis.window does help resolve the issue. We are already working on the PR and making progress. However, we believe your proposed patch is not necessary at this time.

Thanks for your contribution anyway.

[github-actions[bot]]

Release v3.7.1-rc.1 addresses this.