Roles do not work (show blank page between tests and fail) on M1 Airbook

[qualityshepherd]

trafficstars

What is your Scenario?

I'm attempting to run tests that use Roles on an M1 airbook; 2020; Big Sur 11.2

What is the Current behavior?

Tests login then get stuck on a blank page and fail further tests in the fixture.

Occurs on latest chrome (96.0.4664.55 ) and safari (14.0.3 ) Does not occur on coworkers Windows machine, nor our linux pipeline

What is the Expected behavior?

Should work. Running the same code without Roles work just fine.

What is your public website URL? (or attach your complete example)

See code...

What is your TestCafe test code?

import { Role, Selector as $, t } from 'testcafe';

const user = Role('http://automationpractice.com/index.php?controller=authentication', async t => {
  await t
    .typeText($('#email'), '[email protected]')
    .typeText($('#passwd'), 'test123')
    .click($('#SubmitLogin'))
});

fixture `MyFixture`
  .beforeEach(async t => {
    await t.useRole(user)
  })

test('Test1', async t => {
  await t
    .click($('.logout'))
    .expect($('.logout').exists).notOk()
})

test('Test2', async t => {
  await t
    .click($('.logout'))
    .expect($('.logout').exists).notOk()
})

Your complete configuration file

No response

Your complete test report

No response

Screenshots

Steps to Reproduce

TestCafe version

1.17.1

Node.js version

v14.17.4

Command-line arguments

testcafe chrome test.js

Browser name(s) and version(s)

No response

Platform(s) and version(s)

No response

Other

No response

[AlexanderMoiseev]

Hi,

Thank you for your input. We have reproduced the problem. Please stay tuned.

[qualityshepherd]

thanks y'all... this is obviously a pretty big blocker for me... I was surprised nobody else has seen it 🤔

[qualityshepherd]

also, removed that unnecessary navigateTo that I had in there while regressing... woops!

[mikoz93]

I've had a similar problem when using the current LTS version of node (16.13.0). After the t.useRole code would finish executing, the browser would refresh and lose all cookies stored during sign-in. This happened when using one of the latest versions of Chrome. To solve this, I had to drop back to one of the 80+ versions of Chrome.

[AlexanderMoiseev]

Hi, @Mikoz93

Thank you for pointing this out. Once we have any news, we will post them here. Please stay tuned.

[mikoz93]

@AlexanderMoiseev Any news on this?

[github-actions[bot]]

No updates yet. Once we get any results, we will post them in this thread.

[teebash]

Just checking the status of this ticket, as this is also affecting.

[connorpwilliams]

I've had a similar problem when using the current LTS version of node (16.13.0). After the t.useRole code would finish executing, the browser would refresh and lose all cookies stored during sign-in. This happened when using one of the latest versions of Chrome. To solve this, I had to drop back to one of the 80+ versions of Chrome.

This is seemingly the exact same use case for me. Maybe some insight:

I was using version 1.15.2 and this was working fine. When updating to latest (1.18.4) this stopped working for me.

[VasilyStrelyaev]

@thecpdubguy, thank you for pointing out that this issue is a regression.

@teebash, there are no updates regarding this issue yet. We will update this thread once we have any news.

[gg-kialo]

FWIW, some people on my team are also affected by this. They've been adding additional awaits in places that they aren't needed and it seems like they skirt around the issue. i.e. effectively await t.useRole(await (async () => (await role))()).

I don't have an M1 so can't reproduce these issues myself.

[VasilyStrelyaev]

Thank you for sharing your workaround!

[qualityshepherd]

Still no movement on this. I'm really considering moving to another framework.

Anyway, here's an updated example that shows the problem (original site was nuked).

import { Role, Selector as $, t } from 'testcafe';

const user = Role('https://www.saucedemo.com/', async t => {
  await t
    .typeText($('#user-name'), 'standard_user')
    .typeText($('#password'), 'secret_sauce')
    .click($('#login-button'))
});

fixture `MyFixture`
  .beforeEach(async t => {
    await t.useRole(await user)
  })

test('Test1', async t => {
  await t
    .click($('#react-burger-menu-btn'))
    .click($('#logout_sidebar_link'))
    .expect($('#login-button').exists).ok()
})

test('Test2', async t => {
  await t
    .click($('#react-burger-menu-btn'))
    .click($('#logout_sidebar_link'))
    .expect($('#login-button').exists).ok()
})

[qualityshepherd]

Also, I REALLY wish the fix for this is to bag, Roles; implement a beforeAll and let me roll my own Roles (see what I did there?).

[aleks-pro]

Hello @qualityshepherd ,

Thank you for the updated example. We will revisit this issue priority.

[vileen]

@qualityshepherd do you also get redirected to about:blank after logging in? I wonder if it's the exact same scenario for you Workaround from @gg-kialo didn't produce any different outcome than simple await role, but it makes the test stay on about:blank a lot shorter before navigating further But still, the refresh happens so my tests are broken, and they're running just fine on Windows...

[qualityshepherd]

To be clear, gg-kialo's workaround does NOT fix this issue. My example still fails.

[Artem-Babich]

Hi @qualityshepherd,

I just tested your example and found that you didn't specify the URL on which you wish to run tests. As a result, you are getting a blank page after calling the useRole method: Activate Roles. Also, note that in your example, when going to the main page (https://www.saucedemo.com), even a logged user will not be redirected to inventory.html. Instead, you will be asked to log in again.

So, I had to change your test as follows:

.beforeEach(async t => {
    await t.useRole(user)
    await t.navigateTo('https://www.saucedemo.com/inventory.html')
})

After this, I tried running this test on the following machines with Node 18.12:

1. macOS Ventura 13.4 - M1 chip, Chrome 114.0.5735.106, and Safari 16.5 (18615.2.9.11.4)
2. Ubuntu 22.04.2 LTS, Chrome 113.0.5672.126
3. Windows 11 Pro, Chrome 114.0.5735.110

The behavior is the same on all machines - all tests are passed.

Would you please apply the changes above and test the example on your machine again? If the issue is still reproducible, please share more details so we can reproduce it on our side.

[qualityshepherd]

@Artem-Babich I think you're misunderstanding how Role works. This is a bug... and anyway, my example is just an example. Try it on Windows or non-m1 and it'll work.

[Artem-Babich]

Hi @qualityshepherd,

Thank you for the reply. Would you please elaborate on how Roles should work in this case and why you think this behavior is incorrect?

Previously, I shared a link where the behavior you encountered is discussed: Activate Roles.

The first time you activate a role, TestCafe navigates to the login page and executes login actions. When the authentication is complete, TestCafe goes back to the page that was open before Role activation. Use the preserveUrl option to disable this behavior.

So, when you logged in with a Role, the page is redirected back to about:blank. That is why I corrected your test's code. After this change, I was able to run it on all of the OS I mentioned above. If you see a different behavior, please share the following details:

1. TestCafe version.
2. NodeJS version.
3. OS version.
4. Browser and its version.
5. Command line arguments.
6. Exact test code.
7. A screencast illustrating the problematic behavior.

I look forward to your response.

[qualityshepherd]

Thank you for the reply. Would you please elaborate on how Roles should work in this case and why you think this behavior is incorrect?

Um... no.

[qualityshepherd]

And sorry... what I mean is, I've written bugs against Testcafe... and they go nowhere. Testcafe would be the best framework going if:

1. it had a real beforeAll (which would also allow us to NOT use Roles
2. threaded tests at the file level instead of the test level

That's it. But it's not happened in years. I've been using Testcafe for years... but that's gonna change soon.

[Artem-Babich]

Thank you for your feedback. We will take it into the account.

Since we identified the cause of the incorrect behavior in the example, we are closing this issue. However, if you still believe that you encounter a bug, feel free to reopen this issue and share an example with instructions on how to reproduce the problematic behavior.

[qualityshepherd]

This is a bug but closing it anyway is what I've come to expect from Testcafe. I won't bother writing up future bugs...

[vileen]

@qualityshepherd you digress, could you stick to the topic and explain the issue? There is no point in being salty, it doesn't produce any value and you might be surprised if something comes out of this discussion.

[vileen]

Thanks @Artem-Babich, seems like you were right, I even made a repo to try and reproduce the issue https://github.com/vileen/refresh-after-logging-in-reproduction

But seems like simply adding preserveUrl: true fixes the issue in this simple example. Now I have to figure out why tests are not working in more complex setup which is using cucumber... There is something causing reload after logging in, before the process even reaches the test case so the errors disappears and test fails.

[Artem-Babich]

Hi @vileen,

Thank you for letting us know that you were able to resolve the initial issue. Regarding the issue with the cucumber integration, if the test suit works with TestCafe but fails when using cucumber - it is better to create a separate issue in the cucumber's repository.