Testcafe test runner object will timeout even with valid URL on Edge/Chrome, when running Testcafe from CLI works

[Cicerio]

What is your Scenario?

I'm trying to use a node test-runner object (test_runner.ts) to run Testcafe with my test.ts file. Once the site is running, I want to run node tests/e2e/test_runner.ts and have it run tests using the test.ts file tests.

What is the Current behavior?

When I try to run node tests/e2e/test_runner.ts (while the site is running on my local), it brings up the webpage in a blank chrome tab (like usual), but it won't run the tests on the webpage. After 2 minutes (the default timeout), it closes the webpage and displays an error: Error: Cannot establish one or more browser connections. Even though my webpage is running locally on my system just fine and running the test.ts file through the CLI works as intended.

What is the Expected behavior?

I expect when node tests/e2e/test_runner.ts is run while the webpage is active on my local, the webpage should be brought up in the browser, Testcafe should connect to it and then run the tests outlined in test.ts on the page before closing the browser and displaying the results in the CLI.

What is the public URL of the test page? (attach your complete example)

https://angular-node-boilerplate-c873a67b06f7.herokuapp.com/

What is your TestCafe test code?

tests/e2e/test_runner.ts:

const createTestCafe = require("testcafe");

(async () => {
  const testcafe = await createTestCafe("localhost", 4200);

  try {
    const runner = testcafe.createRunner();

    const failedCount = await runner
      .browsers(["chrome --window-size=1280,1024"])
      .src(["tests/e2e/test_homepage/test.ts"])
      .run();

    console.log("Tests failed: " + failedCount);
  } finally {
    await testcafe.close();
  }
})();

tests/e2e/test_homepage/test.ts

import { AngularSelector, waitForAngular } from 'testcafe-angular-selectors';
import { takeSnapshot } from 'testcafe-blink-diff';
import { Selector, test } from 'testcafe';
import { ClientFunction } from 'testcafe';
import { getThreshold } from '../data/constants';

// making memory exception. Might want to move to global ts file
declare global {
    interface Performance {
        memory: {
            jsHeapSizeLimit: number;
            totalJSHeapSize: number;
            usedJSHeapSize: number;
        };
    }
}
// page load time stuff
const isPageLoaded = ClientFunction(() => {return document.readyState === 'complete';});

// helper functions for Measure Memory
const getMemory = async t => JSON.parse(
    await t.eval(() => {
    const { jsHeapSizeLimit, totalJSHeapSize, usedJSHeapSize } = window.performance.memory;
    console.log({ jsHeapSizeLimit, totalJSHeapSize, usedJSHeapSize });
    return JSON.stringify({ jsHeapSizeLimit, totalJSHeapSize, usedJSHeapSize });
}));
//compare current memory against memory threshold
const validateMemory = (memoryVal: { jsHeapSizeLimit: number; usedJSHeapSize: number; totalJSHeapSize: number }, threshold: number) => {
    const { jsHeapSizeLimit, usedJSHeapSize, totalJSHeapSize } = memoryVal;
    const isMemoryLowerThanThreshold =  usedJSHeapSize < threshold;
    return isMemoryLowerThanThreshold;
};

fixture `App tests`
    .page('http://localhost:4200')
    .beforeEach(async () => {
        await waitForAngular();
    });
// measures page load time, fails if load time over threshold
test('Measure Page Load Time', async t => {
    const startTime = await t.eval(() => performance.now());

    await t.expect(isPageLoaded()).ok();

    const endTime = await t.eval(() => performance.now());
    const pageLoadTime = endTime - startTime;
    await t.expect(pageLoadTime).lt(getThreshold("pageLoad"));
    console.log(`Page load time: ${pageLoadTime} milliseconds`);
});
test('Click Button Once', async t => {
    const elementOne = Selector('app-example-one');
    const elementTwo = Selector('app-example-two'); 
    await t
    // checks the DOM for the elements that change during the button press
        .expect(elementOne.exists).ok()
        .expect(!elementTwo.exists).notOk()

    // then clicks the button, looks for changes
        .click('button')
        .expect(elementTwo.exists).ok()
        .expect(!elementOne.exists).notOk();

        await takeSnapshot(t);
});
test('Click Button Twice', async t => {
    const elementOne = Selector('app-example-one');
    const elementTwo = Selector('app-example-two'); 
    await t
        .expect(elementOne.exists).ok()
        .expect(!elementTwo.exists).notOk()
        
        .click('button')
        .expect(elementTwo.exists).ok()
        .expect(!elementOne.exists).notOk()
        
        .click('button')
        .expect(elementOne.exists).ok()
        .expect(!elementTwo.exists).notOk();
    await takeSnapshot(t);
});
test('Measure Memory Usage', async t => {
    const memoryVal = await getMemory(t);
    await t.expect(validateMemory(memoryVal, getThreshold("memory"))).ok(`Memory usage exceeds the threshold: ${JSON.stringify(memoryVal)}`);
});

Your complete configuration file

module.exports = {
    "src": "tests/e2e",
    "retryTestPages": true,
    "hostname": "localhost",
}

Your complete test report

The "src" option from the configuration file will be ignored.
Error: Cannot establish one or more browser connections.
1 of 1 browser connections have not been established:
- chrome --window-size=1280,1024

Hints:
- Increase the Browser Initialization Timeout if its value is too low (currently: 2 minutes for local browsers and 6 minutes for remote browsers). The timeout determines how long TestCafe waits for browsers to be ready.
- The error can also be caused by network issues or remote device failure. Make sure that your network connection is stable and you can reach the remote device.
    at new BrowserConnectionError (D:\Repositories\angular-boilerplate\node_modules\testcafe\src\errors\runtime\index.js:196:9)
    at BrowserSet.createBrowserConnectionError (D:\Repositories\angular-boilerplate\node_modules\testcafe\src\runner\browser-set.ts:126:16)
    at Function.from (D:\Repositories\angular-boilerplate\node_modules\testcafe\src\runner\browser-set.ts:108:30)
    at async Promise.all (index 0)
    at Bootstrapper._bootstrapParallel (D:\Repositories\angular-boilerplate\node_modules\testcafe\src\runner\bootstrapper.ts:427:38)
    at Bootstrapper.createRunnableConfiguration (D:\Repositories\angular-boilerplate\node_modules\testcafe\src\runner\bootstrapper.ts:455:25)
    at Runner._getRunTaskOptions (D:\Repositories\angular-boilerplate\node_modules\testcafe\src\runner\index.js:605:75)

Screenshots

This is the state it sits on for 2 minutes after opening the browser. Afterward, the browser closes and gives a timeout error.

Steps to Reproduce

1. Set up the work environment with node, Testcafe, and the test_runner.ts / test.ts files
2. Make sure the URL is the Heroku site
3. Run test_runner.ts using Node (node [filepath]/test_runner.ts)
4. Should produce the timeout error after loading up Testcafe and waiting for the timeout timer (without running tests

TestCafe version

8.19.2

Node.js version

18.12.1

Command-line arguments

node tests/e2e/test_runner.ts

Browser name(s) and version(s)

Chrome v125.0.6422.142, Edge v125.0.2535.85, Firefox v123.0

Platform(s) and version(s)

Windows 10 Home

Other

These tests are run on a locally hosted version of the website. For this issue, we launched a Heroku-hosted version of the same app for testing purposes. I have tried using Chrome and Edge, but the error persists in both browsers, with the local hosted site and the live site. However, in Firefox, the bug doesn't occur and runs the tests fine using the test runner, with both the live site and the local host.

[Bayheck]

Hello, I tried to reproduce your example, but I got the following error:

Could you please check if the application is deployed correctly?

In addition, I commented out the line with getThreshold and its subsequent uses:

import { getThreshold } from '../data/constants';

Could you please also share the file/files?

[Cicerio]

Oh, apologies, for the first issue, it's a bug in the testing code. I realized this after I posted the bug, and it's an unneeded line. In test.ts, comment out:

fixture `App tests`
    .page('http://localhost:4200')
///    .beforeEach(async () => {
///        await waitForAngular(); - This line is causing the bug you're experiencing
///    });

And for the thresholds, I forgot to include that file, here it is. constants.ts:

export const thresholds = {
    pageLoad: 40,
    memory: 80000000
}
export function getThreshold(thresholdKey: string){
    return thresholds[thresholdKey] * 1.2;
}

[github-actions[bot]]

This issue was automatically closed because there was no response to our request for more information from the original author. Currently, we don't have enough information to take action. Please reach out to us if you find the necessary information and are able to share it. We are also eager to know if you resolved the issue on your own and can share your findings with everyone.

[Cicerio]

@Bayheck Apologies for the late response, but what did you need clarification on?

[Bayheck]

Hello, thank you for the clarification.

I managed to run tests in your example.

However, I did not encounter any issues you mentioned.

All tests were passed:

I used TestCafe version 3.6.1. Could you please send TestCafe version you are using? In addition, try running your tests on a different machine.

[github-actions[bot]]

This issue was automatically closed because there was no response to our request for more information from the original author. Currently, we don't have enough information to take action. Please reach out to us if you find the necessary information and are able to share it. We are also eager to know if you resolved the issue on your own and can share your findings with everyone.