core
core copied to clipboard
owncloud
webUI acceptance test Mink-selenium dependencies
overnight the UI acceptance tests started failing, e.g. see test PR #37111 https://drone.owncloud.com/owncloud/core/23882/86/15
Scenario: simple user login # /drone/src/tests/acceptance/features/webUILogin/login.feature:12
Given these users have been created with default attributes and without skeleton files: # FeatureContext::theseUsersHaveBeenCreatedWithDefaultAttributesAndWithoutSkeletonFiles()
| username |
| user1 |
When user "user1" logs in using the webUI # WebUILoginContext::logInWithUsernameUsingTheWebUI()
Fatal error: Call to a member function open() on null (Behat\Testwork\Call\Exception\FatalThrowableError)
Then the user should be redirected to a webUI page with the title "Files - %productname%" # WebUIGeneralContext::theUserShouldBeRedirectedToAWebUIPageWithTheTitle()
SCENARIO RESULT: (fail)
Seems to be related to the new releases of behat/mink-selenium2-driver and behat/mink that were done overnight.
https://github.com/minkphp/MinkSelenium2Driver/releases/tag/v1.4.0 https://github.com/minkphp/Mink/releases/tag/v1.8.1
- [x] pin the version of whatever dependencies to get tests passing again #37112
- [ ] bump dependencies to whatever newer/well-defined versions will work reliably
- [ ] investigate what "official" versions have trouble, and why, and either fix our code or report upstream...
ToDo this week - I will look out how we can move forward with the newer version 1.8.1 of minkphp\Mink
Still on the "to-do" list. But it will be important to sort this out sooner rather than later.
@phil-davis I could not find what's causing this. I found one line in mink release notes that may or may not be related to the issue but not sure where this is used
Auto-start the session on first call to visit. Mink::getSession() will no longer start the session automatically.
We might need to look into this when PHP 8.* support moves forward. But for now, the classic web UI tests are running fine with the pinned mink-selenium dependencies.
the issue is not blocked with any specific reason, moving this to QA backlog.
the issue is not blocked with any specific reason, moving this to QA backlog.
yes, someone could investigate again if there are newer combinations of dependencies plus selenium plus browser that will work together.
@SwikritiT @amrita-shrestha I added this to the current project board. Related discussion https://github.com/owncloud-docker/server/pull/390#discussion_r1169770353
Please get someone to try the "old" classic web UI acceptance tests with newer chrome-selenium-docker and see if there is some way to make it work.
Findings:
While upgrading behat/mink version we get Fatal error: Call to a member function open() on null (Behat\Testwork\Call\Exception\FatalThrowableError) error .
Discussed in: https://github.com/owncloud/core/pull/37112
Could be related to: https://github.com/sensiolabs/BehatPageObjectExtension/issues/92
public function open(array $urlParameters = array())
{
$url = $this->getUrl($urlParameters);
$this->getDriver()->visit($url);
$this->verify($urlParameters);
return $this;
}
The solution for behat/mink version is to modify above code as
public function open(array $urlParameters = array())
{
$url = $this->getUrl($urlParameters);
$this->getSession()->visit($url);
$this->verify($urlParameters);
return $this;
}
@SwikritiT @amrita-shrestha I added this to the current project board. Related discussion owncloud-docker/server#390 (comment)
Please get someone to try the "old" classic web UI acceptance tests with the newer chrome-selenium docker and see if there is some way to make it work.
Discussion in this issue : https://github.com/SeleniumHQ/docker-selenium/issues/922
It still doesn't work with the selenium/standalone-chrome: latest docker image. Currently, if we use the selenium/standalone-chrome:latest docker image we get this error
And user "Alice" has logged in using the webUI # WebUILoginContext::theUserHasLoggedInUsingTheWebUI()
Could not open connection: Could not start a new session. New session request timed out
Host info: host: 'xxxxx-OptiPlex-xxxx', ip: '127.0.1.1'
Build info: version: 'x.xx.x', revision: 'xxxxxxx'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: 'x.xx.x-xx-generic', java.version: 'xx.x.xx'
Driver info: driver.version: unknown (Behat\Mink\Exception\DriverException)
with chromium-selenium-docker, it works fine
SELENIUM_IMAGE=seleniarm/standalone-chromium:4.0.0-beta-1-20210215. But with the latest chromium-selenium-docker, it fails.
The idea of using chromium comes from a web docker image of selenium which uses chromium instead of chrome. https://github.com/owncloud/web/pull/5415/files#diff-25825897929bda864a1f3d8a946fe6b8dd1273f556b6712cc8dd92350e42560b
Related issue discussion: https://github.com/minkphp/MinkSelenium2Driver/issues/293
- Maybe we could try out different drivers https://mink.behat.org/en/latest/drivers/
What do you mean by ...?
We are currently using chromium image in docker instead of chrome.
The issue is related to behat mink not with the nightwatch of web
What do you mean by ...?
We are currently using chromium image in docker instead of chrome.
The issue is related to behat mink not with the nightwatch of web
sorry if I gave the wrong hint with my word. I said we are using chromium instead of chrome in the docker. I didn't mention nightwatch on the web. I just mentioned web because the docker code is present in the web repo
That image is only used for mac m1 device. Selenium image is used only for the nightwatch tests and we use selenium/standalone-chrome in CI (with pinned tag)
Running with selenium/standalone-chrome:104 and w3c disabled works but in the CI the tests can fail due to the issue related to memory spaces.
Also, using various chrome options is not helping.
--disable-gpu
--disable-gpu-sandbox
--disable-dev-shm-usage
--no-sandbox
--test-type=ui
I think we won't be putting more effort into maintaining oc10 test infrastructure
agree - the web UI tests with Behat and chrome/firefox will be stuck with whatever versions of Behat/Gherkin/Mink/selenium are working now. It seems that Behat support for Mink/selenium to control newer browser versions is never going to happen.