dotnet
Update selenium dependency versions
![azure-pipelines[bot] avatar](https://avatars.githubusercontent.com/in/9426?v=4 "Published by a pub.dev verified publisher"){kind=small}
Some big jumps. Hope they don't include breaking changes…
Yeah, it looks like the last time this was updated was August 1st so we're a little behind. Hopefully, not any breaking changes here...
Due to the https://github.com/dotnet/aspnetcore/blob/07a563e1ed6d443a8e8dce6c4fe78a6aad5302df/src/Components/test/E2ETest/.npmrc file, you'll need to get vsts-npm-auth and use vsts-npm-auth -C {that file} before running yarn install in that directory
Due to the https://github.com/dotnet/aspnetcore/blob/07a563e1ed6d443a8e8dce6c4fe78a6aad5302df/src/Components/test/E2ETest/.npmrc file, you'll need to get
vsts-npm-authand usevsts-npm-auth -C {that file}before runningyarn installin that directory
Where are we supposed to run this?
Where are we supposed to run this?
In the same directory as that .npmrc and the changed package.json file.
The system automatically loads packages from upstream repositories into the one listed in the .npmrc file when an authorized person runs yarn install. That means only authorized people (like you) can change what we depend on.
It may also be necessary to clear your local yarn cache before installing and updating the lock file.
@dougbu what I meant is, does vsts-npm-auth need to go on a ~~yarn~~ YAML file?
No, execute it from the command line using the .npmrc file I mentioned. Then, you're authenticated and yarn install should do the right thing, assuming your cache is clear and you don't have a node_modules/ folder in the directory.
Not sure but always-auth=true may temporarily (don't check the change in) need to be added to the .npmrc file.
https://learn.microsoft.com/en-us/azure/devops/artifacts/tutorials/protect-oss-packages-with-upstream-sources?view=azure-devops&tabs=npm%2Cnpmrestore is pretty general but has a bit more context.
@captainsafia @javiercn where exactly is this PR❔ Do one of you already have updated lock files on your machines❔
@dotnet/aspnet-blazor-eng I updated the lock files and we're getting further. But it appears the newer packages include a breaking change:
> selenium-standalone start --config /home/vsts/work/1/s/src/Shared/E2ETesting/selenium-config.json --javaArgs=-Dwebdriver.chrome.driver=/usr/local/share/chrome_driver/chromedriver -- -port 44457
[xUnit.net 00:00:02.56] Microsoft.AspNetCore.Components.E2ETests:
[xUnit.net 00:00:02.83] Microsoft.AspNetCore.Components.E2ETests: Unrecognized option: -
[xUnit.net 00:00:02.83] Microsoft.AspNetCore.Components.E2ETests: Error: Could not create the Java Virtual Machine.
[xUnit.net 00:00:02.83] Microsoft.AspNetCore.Components.E2ETests: Error: A fatal exception has occurred. Program will exit.
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: /home/vsts/work/1/s/src/Components/test/E2ETest/node_modules/selenium-standalone/lib/check-started.js:44
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: throw new Error(
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: ^
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests:
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: Error: Selenium exited before it could start with code 1
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: Stdout:
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: Stderr:
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: at checkStarted (/home/vsts/work/1/s/src/Components/test/E2ETest/node_modules/selenium-standalone/lib/check-started.js:44:13)
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: at async Object.start (/home/vsts/work/1/s/src/Components/test/E2ETest/node_modules/selenium-standalone/lib/start.js:150:3)
[xUnit.net 00:00:03.27] Microsoft.AspNetCore.Components.E2ETests: at async Object.start (/home/vsts/work/1/s/src/Components/test/E2ETest/node_modules/selenium-standalone/bin/selenium-standalone:23:16)
Didn't we hit something similar the last time we tried to move to a newer selenium-standalone❔
This seems like something the @dotnet/aspnet-blazor-eng team should investigate and fix. It's gotten too complex for normal build-ops to try to figure out.
Several tests take longer than usual and show a [Long Running Test] warning in the log. Restarted the step to check if it's transient of something to investigate.
https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_apis/build/builds/93516/logs/140
Timeouts still happening, some investigations on why some tests became too slow will be necessary.
Have we tried bumping timeouts for the aspnetcore-components-e2e pipeline yet @sebastienros❔ Won't help if a test is just hanging but might work if we're just dealing w/ a general slowdown.
It is much slower, like before 0:40 minutes and now timing out at 1:15, with the last part growing from 0:10 to 0:45 already, plus an estimate of another 30 minutes from what I see in the old logs.
I can try to bump to 2h to have an idea of how long it will take, but are we ok with that much carbon impact?
I can try to bump to 2h to have an idea of how long it will take, but are we ok with that much carbon impact?
Good question. In this case, the impact of increasing the timeout should be more than offset by avoiding useless retries to validate PRs touching this area. (We don't require the pipeline to pass but should be diligent if a PR may affect these tests.)
I have started the investigation to try understanding why the test are taking longer. I still need to check what is the behavior before the update, but I notice that when running locally the test is retrying ( i believe it might be the expected behavior), however, after the update I found a lot of warning entries like this:
{"traceId": "9878be833f94e8b6b4242beead9ee63d","eventTime": 1669933376674284037,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "localhost:39509","http.method": "POST","http.request_content_length": "437","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.0.0 (.net linux)"}}
Just for context, although it would certainly be beneficial to be able to merge a Selenium update (if it doesn't regress CI perf badly), there is a totally different approach we're considering at https://github.com/dotnet/aspnetcore/pull/45284 which would allow us to drop the Selenium dependency completely.
It might be possible to get that done in the next couple of months. So we should bear this in mind before spending too much effort on the Selenium upgrade.
@SteveSandersonMS I'm fine w/ your timing except for the likelihood Chrome gets updated to be incompatible w/ our Selenium versions other than selenium-standalone. Might be best to leave that change (which I think is the main bump related to the Components E2E tests) out of this PR and get this in to be on the safe side.
That may however require unravelling some of the workarounds in this PR.
Of course, the validation running now might just work 😀
Reminder to all: Do not squish n' merge unless aspnetcore-components-e2e is successful. (It's not normally required but should be here.)
Made some additional bumps for Chrome 108 and associated selenium packages. Reverted the selenium-standalone bump for now.
Latest packages aren't available in our feeds, have reached out offline to the appropriate folks.
Progress?
Exception in thread "main" com.beust.jcommander.ParameterException: Was passed main parameter '--port' but no main parameter was defined in your arg class
at com.beust.jcommander.JCommander.initMainParameterValue(JCommander.java:936)
at com.beust.jcommander.JCommander.parseValues(JCommander.java:752)
at com.beust.jcommander.JCommander.parse(JCommander.java:340)
at com.beust.jcommander.JCommander.parse(JCommander.java:319)
at org.openqa.grid.selenium.GridLauncherV3.parse(GridLauncherV3.java:218)
at org.openqa.grid.selenium.GridLauncherV3.lambda$buildLaunchers$3(GridLauncherV3.java:241)
at org.openqa.grid.selenium.GridLauncherV3.lambda$launch$0(GridLauncherV3.java:86)
at java.base/java.util.Optional.map(Optional.java:265)
at org.openqa.grid.selenium.GridLauncherV3.launch(GridLauncherV3.java:86)
at org.openqa.grid.selenium.GridLauncherV3.main(GridLauncherV3.java:70)
/home/vsts/work/1/s/src/Components/test/E2ETest/node_modules/selenium-standalone/lib/check-started.js:23
throw new Error(`Selenium exited before it could start with code ${cpState.code}\n`);
^
I think this may be related to the arg parsing @BrennanConroy mentioned above.
Hmm I'm seeing:
selenium-standalone start --config /home/vsts/work/1/s/src/Components/test/E2ETest/bin/Release/net7.0/modifiedConfig.json -- --port 44615
But that looks like it's expected:
https://github.com/dotnet/aspnetcore/pull/44834/files#diff-220c96679ae5c8b66a7c08d0514422c79c41bbf522874580097ce4dd6b6d8f1eL114-R118
@BrennanConroy are you able to share some more context on your earlier change?
Taking a shot by reverting the --port to the original -port: 5624bc7
Okay, that worked, no more parameter error! Onto the next;
System.InvalidOperationException : Couldn't create a Selenium remote driver client. The server is irresponsive
---- OpenQA.Selenium.WebDriverException : The newSession command returned an unexpected error.
Selenium
Whoops! The URL specified routes to this help page.
For more information about Selenium please see the docs and/or visit the wiki. Or perhaps you are looking for the Selenium console.
Happy Testing!
So running into some routing issue now.
Reverted the selenium standalone workaround.