cypress
cypress copied to clipboard
cypress-io
When accessing multiple sessions with origin I get "cannot read properties of undefined reading 'message'" on azureAD with Chrome
Current behavior
I have a test that tries to login with different session_id (string, string[], object) and with the use of origin on each step, with the same login provider (Azure AD). When the tests are run on Chrome or Electron (headless/browser) after the 1st login/session for an approx 80% of the times the rest of the tests will fail (see attached error).
When I run on firefox the same tests will have lower chance of failing (approx 10% -20%)
I couldnt manage to scope the reason that it fails but it could be either the security of chrome (it is disabled) or the cookies.
Another lead is that some times the failed step is showed that has no "url" during the call of the step when I check the snapshots of the stack trace
Desired behavior
No response
Test code to reproduce
-----Login helper commands ----------
cy.session(session as string | string[] | object, () => {
loginWithAzureAD(username, password);
cy.visit('/');
});
export const loginWithAzureAD = (username: string, password: string): void => {
cy.origin(
'https://login.microsoftonline.com',
{ args: { username, password } },
({ username, password }) => {
cy.visit('/');
// Set email and wait
cy.get('[type="email"]').should('be.visible').type(username);
cy.get('[type="submit"]').click();
//cy.wait(3000);
// Validation that the email input and progress bar do not exist
cy.get('[type="email"]', { timeout: 5000 }).should('not.exist');
cy.get('[class=progress]', { timeout: 5000 }).should('not.exist');
// Set password and wait
cy.get('[type="password"]', { timeout: 5000 })
.should('be.visible')
.type(password);
cy.get('[type="submit"]').click();
//cy.wait(2000);
// Validation that the password input does not exist
cy.get('[type="password"]', { timeout: 5000 }).should('not.exist');
}
);
};
-------Spec ----------------------
describe('Test Azure AD Authentication with origin', () => {
// beforeEach(function () {
// //In a real scenario here we place the cy.loginAzureADwithOrigin command
// //The login will cache the cookies and local storage
// });
it('Direct login with the exported cridentials', () => {
cy.login('ad');
cy.visit('/');
cy.get('[data-testid="search-icon"]').should('exist');
});
it('Direct login with the exported cridentials and custom session id', () => {
cy.login('ad', 'test_user');
cy.visit('/');
cy.get('[data-testid="search-icon"]').should('exist');
});
it('Custom login with credentials in array, and unique session id', () => {
cy.login('ad', [Cypress.env('username'), Cypress.env('password')]);
cy.visit('/');
cy.get('[data-testid="search-icon"]').should('exist');
});
it('Custom login with credentials in object, and unique session id', () => {
cy.login('ad', {
username: Cypress.env('username'),
password: Cypress.env('password'),
});
cy.visit('/');
cy.get('[data-testid="search-icon"]').should('exist');
});
});
Cypress Version
10.8.0
Node version
v14.17.0
Operating System
macOS 12.5.1
Debug Logs
No response
Other
No response
Sorry you're encountering this issue.
Do you have experimentalModifyObstructiveThirdPartyCode set to true in your cypress.config.ts?
Sorry for the late response,
yes I have both: experimentalSessionAndOrigin: true, experimentalModifyObstructiveThirdPartyCode: true,
@io-sar, could you try this on cypress 10.9.0, we recently released some stability fixes for cy.origin and i'm wondering if that has had any effect on your error message.
@mjhenkes I have tried it. Unfortunately it is not fixed. However there is a success as now I recieve also the following message :
The following error originated from your test code, not from Cypress.
> Failed to construct 'URL': Invalid URL
When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.
which confirms my comment "some times the failed step is showed that has no url"
What is the stack trace provided when that error is thrown? Anything useful?
@mjhenkes not really... :(
at loginWithAzureAD (webpack:///../cypress-utils/src/lib/commands/login/login-with-azure-ad.ts:11:5)
at Object.eval [as setup] (webpack:///../cypress-utils/src/lib/commands/login/login.ts:39:25)
From previous event:
at Context.origin (https://my.baseurl/__cypress/runner/cypress_runner.js:142366:14)
at wrapped (https://my.baseurl/__cypress/runner/cypress_runner.js:157357:43)
at <unknown> (https://my.baseurl/__cypress/runner/cypress_runner.js:156095:15)
From previous event:
at CommandQueue.runCommand (https://my.baseurl/__cypress/runner/cypress_runner.js:156073:8)
at next (https://my.baseurl/__cypress/runner/cypress_runner.js:156220:19)
at <unknown> (https://my.baseurl/__cypress/runner/cypress_runner.js:156249:16)
From previous event:
at next (https://my.baseurl/__cypress/runner/cypress_runner.js:156220:39)
From previous event:
at <unknown> (https://my.baseurl/__cypress/runner/cypress_runner.js:171645:77)
From previous event:
at CommandQueue.run (https://my.baseurl/__cypress/runner/cypress_runner.js:171640:21)
at CommandQueue.run (https://my.baseurl/__cypress/runner/cypress_runner.js:156283:15)
at $Cy.runQueue (https://my.baseurl/__cypress/runner/cypress_runner.js:157333:14)
at cy.<computed> [as login] (https://my.baseurl/__cypress/runner/cypress_runner.js:157442:12)
at runnable.fn (https://my.baseurl/__cypress/runner/cypress_runner.js:157625:19)
at callFn (https://my.baseurl/__cypress/runner/cypress_runner.js:107910:21)
at ../driver/node_modules/mocha/lib/runnable.js.Runnable.run (https://my.baseurl/__cypress/runner/cypress_runner.js:107897:7)
at <unknown> (https://my.baseurl/__cypress/runner/cypress_runner.js:164934:30)
From previous event:
at Object.onRunnableRun (https://my.baseurl/__cypress/runner/cypress_runner.js:164919:53)
at $Cypress.action (https://my.baseurl/__cypress/runner/cypress_runner.js:153707:28)
at Runnable.run (https://my.baseurl/__cypress/runner/cypress_runner.js:162582:13)
at ../driver/node_modules/mocha/lib/runner.js.Runner.runTest (https://my.baseurl/__cypress/runner/cypress_runner.js:108569:10)
at <unknown> (https://my.baseurl/__cypress/runner/cypress_runner.js:108695:12)
at next (https://my.baseurl/__cypress/runner/cypress_runner.js:108478:14)
at <unknown> (https://my.baseurl/__cypress/runner/cypress_runner.js:108488:7)
at next (https://my.baseurl/__cypress/runner/cypress_runner.js:108390:14)
at <unknown> (https://my.baseurl/__cypress/runner/cypress_runner.js:108456:5)
at timeslice (https://my.baseurl/__cypress/runner/cypress_runner.js:102382:27)
@io-sar I am starting to take a look into this. I threw together a reproduction repo here. Can you help me fill in the blanks so I can reproduce this on my end? Also curious to see if you continue to have this issue with cypress 10.10.0
I believe I'm also seeing this issue, though not in AD (just a Win 10 Pro local machine).
Upgrading from 10.3 to 10.10 did not help.
10.10 stack:
index.656179c6.js:99335 TypeError: The following error originated from your application code, not from Cypress.
> Cannot read properties of undefined (reading 'message')
When Cypress detects uncaught errors originating from your application it will automatically fail the current test.
This behavior is configurable, and you can choose to turn this off by listening to the `uncaught:exception` event.
Because this error occurred during a `before each` hook we are skipping the remaining tests in the current suite: `Integration Manager: Kick`
at makeErrFromObj (http://localhost:8080/__cypress/runner/cypress_runner.js:160728:30)
at errorFromProjectRejectionEvent (http://localhost:8080/__cypress/runner/cypress_runner.js:161091:10)
at Object.errorFromUncaughtEvent (http://localhost:8080/__cypress/runner/cypress_runner.js:161097:65)
at <unknown> (http://localhost:8080/__cypress/runner/cypress_runner.js:157758:74)
From previous event:
at CommandQueue.run (http://localhost:8080/__cypress/runner/cypress_runner.js:171664:21)
at CommandQueue.run (http://localhost:8080/__cypress/runner/cypress_runner.js:156298:15)
at $Cy.runQueue (http://localhost:8080/__cypress/runner/cypress_runner.js:157348:14)
at cy.<computed> [as serveHtmlFile] (http://localhost:8080/__cypress/runner/cypress_runner.js:157457:12)
at __stackReplacementMarker (http://localhost:8080/__cypress/runner/cypress_runner.js:156804:13)
at runnable.fn (http://localhost:8080/__cypress/runner/cypress_runner.js:157640:19)
at callFn (http://localhost:8080/__cypress/runner/cypress_runner.js:107910:21)
at ../driver/node_modules/mocha/lib/runnable.js.Runnable.run (http://localhost:8080/__cypress/runner/cypress_runner.js:107897:7)
at http://localhost:8080/__cypress/runner/cypress_runner.js:164958:30
From previous event:
at Object.onRunnableRun (http://localhost:8080/__cypress/runner/cypress_runner.js:164943:53)
at $Cypress.action (http://localhost:8080/__cypress/runner/cypress_runner.js:153717:28)
at Runnable.run (http://localhost:8080/__cypress/runner/cypress_runner.js:162595:13)
at next (http://localhost:8080/__cypress/runner/cypress_runner.js:108412:10)
at http://localhost:8080/__cypress/runner/cypress_runner.js:108456:5
at timeslice (http://localhost:8080/__cypress/runner/cypress_runner.js:102382:27)
The project uses experimentalSessionAndOrigin but not experimentalModifyObstructiveThirdPartyCode. I don't believe this is a test issue as the tests run past this point in CI.
Using the test case mentioned above, I get the following:
index.656179c6.js:99335 CypressError: Timed out retrying after 5000ms: The command was expected to run against origin `https://login.microsoftonline.com` but the application is at origin `https://login.live.com`.
This commonly happens when you have either not navigated to the expected origin or have navigated away unexpectedly.
This error occurred while creating session. Because the session setup failed, we failed the test.
at loginWithAzureAD (webpack:///./cypress/support/e2e.ts:23:5)
at Object.eval [as setup] (webpack:///./cypress/support/e2e.ts:49:21)
From Your Spec Code:
at loginWithAzureAD (webpack:///./cypress/support/e2e.ts:23:5)
at Object.eval [as setup] (webpack:///./cypress/support/e2e.ts:49:21)
This was all under Node 16, if it helps.
Hello @AtofStryker, unfortunately it is not fixed on 10.10 and I have tried to use your PR. The main problem is that this is not using the azureAD on an application level thus when you try to login it redirects you during the password phase to a different origin 'https://login.live.com'.
What I would try to do (currently not too much of a time) is to either have an app behind an azure ad free account. Or, try to rework with the baseUrl starting on https://login.microsoftonline.com with all the query parameters and use origin during the 'https://login.live.com'. For the latter I am not sure if it is a valid test though.
I saw also the "cy.origin() use is not currently supported in the cy.origin() callback, but is planned for a future release." message. (cool feature!!)
@turt2live @io-sar sounds like my reproduction isn't very helpful here 😅 . I think the most critical thing here is to get a fully working reproduction, which I know is a bit difficult. Worst case, private reproduction repositories where core team members are invited is also an option. Once I can see that, I have a feeling I can determine the problem.
We also do have a free azure AD account within Cypress to test these scenarios. If you give me instructions on how you set up your application/tenant with a test case, I also might be able to reproduce it on my end.
Unfortuantely I haven't been able to come up with a smaller reproduction case, though our application's use of localStorage and IndexedDB appear to be causing problems, somehow. Removing some of the idb or localstorage code resolves the issue, at least.
Is there potentially something in there which sets off a lightbulb around origins?
Is there potentially something in there which sets off a lightbulb around origins?
@turt2live nothing off the top of my head, but I wonder if it could be something related to us trying to clear localStorage too quickly when the AUT hasn't navigated. Does it happen every single time for you or intermittently?
Very nearly 100% of the time when we set specific keys related to authentication in our app, but only about 10-15% when not setting those keys (but the tests fail anyways).
Is there a way we could quickly test the navigation timing within the context of our app? Somewhat worth noting that our CI (Linux) appears to be fine, which has concerns about platform or machine dependence.
@turt2live @AtofStryker , I dont know if it helps, but wanted to point again that in my case the tests are failing on chrome based browsers. Runnig the tests on Firefox works well (nearly 100% of the time). I am not sure if ff works differently with the localStorage
@turt2live have you tried to run your failing tests with firefox ?
I can't get Firefox to run at all, though I've also possibly been experiencing an environmental issue this whole time. Turns out the tests run perfectly fine in a fresh Windows VM, but not on my local machine.
The hunt continues...
Right now there doesn't seem to be enough information to reproduce the problem on our end. We'll have to close this issue until we can reproduce it. This does not mean that your issue is not happening - it just means that we do not have a path to move forward. You may also want to try upgrading to Cypress 10.11.0 which has a few additional fixes in cy.origin.
Please open a new issue with a reproducible example and link to this issue. Here are some tips for providing a Short, Self Contained, Correct, Example and our own Troubleshooting Cypress guide.
Started noticing this error in a cy.origin in our test cases. It started happening without changes to the test or code.
We did upgrade Cypress from 13.6.8 to 13.7.2, but downgrading didn't fix it. While we got this mostly happening locally (100% of the time now), we also got 1 failure of it in Cypress Cloud.
I realize the issue has been closed due to challenges in reproducing, so happy to share the Cypress Cloud execution link with Cypress support if it helps investigating this issue.
Same issue: cross-origin call that was working fine to Stripe Checkout with experimentalModifyObstructiveThirdPartyCode turned on suddenly gives this error.
The issue happens when the application throws undefined instead of a valid error or object (in @Helveg's case and my case, Stripe is throwing that undefined exception for no good reason).
And Cypress does not seem to be expecting an undefined error.
As a developer, we unfortunately have no control over this issue: Cypress fails to parse undefined, meaning that Cypress.on('uncaught:exception', …) is not even called. So our test fails, whether we set an uncaught:exception listener or not.
EDIT: I contacted the Stripe support to report the problem from their side, but even if it ends up being resolved, I think Cypress should handle cases like this more gracefully.
This is causing us problems in automated tests that include Stripe checkout due to them raising an undefined exception.
Does this need to be raised as a new issue or will this be picked up from here since this issue is already closed?
@michaelkay-wrisk We should probably open a new issue and refer this one, especially since this one has too many misleading details about azureAD and so on.
It would be helpful to get a reproducible example. I'm trying to spin up a Stripe project we have, not sure it will duplicate with exactly what y'all are seeing.
@jennifer-shehane I built a minimal example, it does not require Stripe, it's a CodeSandbox app that does a throw undefined 100 ms after loading.
Then to test it in any Cypress project:
Cypress.on("uncaught:exception", (err, runnable) => {
// returning false here prevents Cypress from
// failing the test
return false
});
it.only("Should catch the exception", () => {
cy.visit("https://www.cypress.io/")
cy.visit("https://9p6zh4.csb.app");
cy.origin("9p6zh4.csb.app", () => {
cy.visit("/");
cy.log("Success!");
});
});
As you should see (screenshot below), uncaught:exception will not catch the exception and Success! will not be logged. It's because Cypress always expects thrown values to have a message property. But throwing undefined or any value without message, although a bad practice, is valid JavaScript.
CodeSandbox source: https://codesandbox.io/p/sandbox/cy-origin-uncaught-exception-9p6zh4?file=%2Fsrc%2FApp.js%3A17%2C2
Thank you for taking a look :)
@BertrandBordage Yay! I was trying out a few different situations. I do see an uncaught 'undefined' error that is not being caught with the uncaught exception handler. I don't see this exact messaging that others seemed to be reporting:
Cannot read properties of undefined (reading 'message')
Is there another piece of code to write to get to that error?
@jennifer-shehane Good point. I'm not sure what causes this variant of the error. I face it in a complex project but could not isolate it yet. I can share the traceback though, which is different from the other error:
cypress-1 | TypeError: Cannot set property message of which has only a getter
cypress-1 |
cypress-1 | Because this error occurred during a `after each` hook we are skipping the remaining tests in the current suite: `REDACTED`
cypress-1 | at modifyErrMsg (http://norishare.localhost/__cypress/runner/cypress_runner.js:75141:15)
cypress-1 | at Object.appendErrMsg (http://norishare.localhost/__cypress/runner/cypress_runner.js:75146:10)
cypress-1 | at Runner.<anonymous> (http://norishare.localhost/__cypress/runner/cypress_runner.js:162598:68)
cypress-1 | at Runner.emit (http://norishare.localhost/__cypress/runner/cypress_runner.js:146530:7)
cypress-1 | at Runner.fail (http://norishare.localhost/__cypress/runner/cypress_runner.js:155453:8)
cypress-1 | at Runner.fail (http://norishare.localhost/__cypress/runner/cypress_runner.js:145735:25)
cypress-1 | at Runner.failHook (http://norishare.localhost/__cypress/runner/cypress_runner.js:1554[91](REDACTED):8)
cypress-1 | at Hook.<anonymous> (http://norishare.localhost/__cypress/runner/cypress_runner.js:155565:14)
cypress-1 | at next (http://norishare.localhost/__cypress/runner/cypress_runner.js:163070:24)
cypress-1 | at http://norishare.localhost/__cypress/runner/cypress_runner.js:1630[97](REDACTED):13
cypress-1 | at tryCatcher (http://norishare.localhost/__cypress/runner/cypress_runner.js:1807:23)
cypress-1 | at Promise._settlePromiseFromHandler (http://norishare.localhost/__cypress/runner/cypress_runner.js:1519:31)
cypress-1 | at Promise._settlePromise (http://norishare.localhost/__cypress/runner/cypress_runner.js:1576:18)
cypress-1 | at Promise._settlePromise0 (http://norishare.localhost/__cypress/runner/cypress_runner.js:1621:10)
cypress-1 | at Promise._settlePromises (http://norishare.localhost/__cypress/runner/cypress_runner.js:1701:18)
cypress-1 | at Promise._fulfill (http://norishare.localhost/__cypress/runner/cypress_runner.js:1645:18)
cypress-1 | at Promise._resolveCallback (http://norishare.localhost/__cypress/runner/cypress_runner.js:1439:57)
cypress-1 | at Promise._settlePromiseFromHandler (http://norishare.localhost/__cypress/runner/cypress_runner.js:1531:17)
cypress-1 | at Promise._settlePromise (http://norishare.localhost/__cypress/runner/cypress_runner.js:1576:18)
cypress-1 | at Promise._settlePromise0 (http://norishare.localhost/__cypress/runner/cypress_runner.js:1621:10)
cypress-1 | at Promise._settlePromises (http://norishare.localhost/__cypress/runner/cypress_runner.js:1701:18)
cypress-1 | at Promise._fulfill (http://norishare.localhost/__cypress/runner/cypress_runner.js:1645:18)
@jennifer-shehane Worth noting I did open a ticket with Cypress support before this issue got reopened. I shared with support (and was told that it was shared with Cypress engineering) a very ful debug log of this issue, along with Cypress Cloud run
I don't have a minimally reproducible example, but the Cypress team should have enough resources at this point to either reproduce the issue or ask questions to narrow it down.
@jennifer-shehane I managed to reproduce this error! Cannot read properties of undefined (reading 'message')
I could only get it with Stripe, so I isolated it with the demo checkout page from Stripe. In case the generated Stripe URL I used is not going to work in the future:
- Go to https://checkout.stripe.dev/preview
- Inspect the code to copy the full iframe URL (including its anchor,
#…) - Paste the iframe URL in the
cy.visitof the snippet below. - Run the snippet, with or without
experimentalModifyObstructiveThirdPartyCode. In both cases it should fail.
Cypress.on("uncaught:exception", (err, runnable) => {
// returning false here prevents Cypress from
// failing the test
return false
});
it.only("Should catch the exception", () => {
cy.visit("https://www.cypress.io/");
cy.origin("checkout.stripe.com", () => {
cy.visit("https://checkout.stripe.com/c/pay/cs_test_b1ME8Ug7h3E2qvMqgXJKiR0fQV0F5lTmRujFdpktg37k3v4cCocLmXCTO0?demoWallet=applePay&demoPolicies=false#fidkdWxOYHwnPyd1blpxYHZxWjA0TUM1Yl9GT1c0a25sZjdSa2ppakZgQzdxVz1qYk9pcUBnNTc9Z11kVnc9b0F%2FREgxfE5oQ31Dd2dGME9sQVx2Tm1wc3RyRGhPZjIwTzRLYmd3TnJDSjJMNTVJMUBtbWNMQycpJ2N3amhWYHdzYHcnP3F3cGApJ2lkfGpwcVF8dWAnPydocGlxbFpscWBoJyknYGtkZ2lgVWlkZmBtamlhYHd2Jz9xd3BgeCUl");
cy.location("host").should("equal", "checkout.stripe.com");
cy.log("Success!");
});
});
Moving the uncaught:exception listener inside cy.origin or disabling it has no effect on the error.
For info, I got a reply from the Stripe support, they don't plan on fixing this error but are interested in seeing Cypress resolve this side of the problem :facepalm:
@BertrandBordage Thanks, I can reproduce.
Stack trace:
"TypeError: Cannot read properties of undefined (reading 'message')
at makeErrFromObj (https://checkout.stripe.com/__cypress/runner/cypress_cross_origin_runner.js:91576:30)
at errorFromProjectRejectionEvent (https://checkout.stripe.com/__cypress/runner/cypress_cross_origin_runner.js:91897:10)
at Object.errorFromUncaughtEvent (https://checkout.stripe.com/__cypress/runner/cypress_cross_origin_runner.js:91902:65)
at https://checkout.stripe.com/__cypress/runner/cypress_cross_origin_runner.js:197375:72"
I did look into this a bit yesterday. It's not as simple of a fix as accepting Strings in this case, as it was on this line here:
https://github.com/cypress-io/cypress/blob/develop/packages/driver/src/cypress/error_utils.ts#L192
I can fix the message error by making a new Error if the obj is undefined on that line above, but the uncaught:exception handler still doesn't register properly, where the test keeps failing.
You can throw a debugger after this line when running the above project through Cypress locally (Instructions here to watch) and see that the results are returning an empty array [] instead of ['false'] like it does for normal errors to indicate there was a handler that returned false.
https://github.com/cypress-io/cypress/blob/develop/packages/driver/src/cypress/cy.ts#L898