minkphp
Run tests also with Selenium 3.x
There were 3 errored and 2 failing tests, see post below for analysis and how they were fixed.
Codecov Report
Merging #354 (e84a258) into master (5d15043) will decrease coverage by
2.28%. The diff coverage is100.00%.
@@ Coverage Diff @@
## master #354 +/- ##
============================================
- Coverage 90.02% 87.73% -2.29%
+ Complexity 150 149 -1
============================================
Files 1 1
Lines 421 367 -54
============================================
- Hits 379 322 -57
- Misses 42 45 +3
| Impacted Files | Coverage Δ | |
|---|---|---|
| src/Selenium2Driver.php | 87.73% <100.00%> (-2.29%) |
:arrow_down: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
1) Behat\Mink\Tests\Driver\Basic\IFrameTest::testIFrame
WebDriver\Exception\InvalidArgument: data did not match any variant of untagged enum FrameId at line 3 column 1
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Session.php:286
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:426
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/behat/mink/src/Session.php:321
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Basic/IFrameTest.php:17
test must be fixed, only integer indexes are accepted, when string id is passed, the following exception is shown:
name key tested, but it is not supported, not handle, nor element itself (might be supported when serialized correctly, but the latest driver does not support it)
WebDriver\Exception\InvalidArgument: invalid argument: 'id' can not be string
(Session info: chrome=84.0.4147.105)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'DX6B48F375QD', ip: '10.8.136.130', os.name: 'Windows 10',
os.arch: 'amd64', os.version: '10.0', java.version: '12.0.2'
2) Behat\Mink\Tests\Driver\Js\ChangeEventTest::testSetValueChangeEvent with data set "file" ('the-file', '/home/runner/work/MinkSeleniu...e1.txt', '/home/runner/work/MinkSeleniu...e2.txt')
WebDriver\Exception\InvalidArgument: File not found: /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/../../web-fixtures/file1.txt
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:234
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Container.php:241
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:679
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/behat/mink/src/Element/NodeElement.php:105
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/ChangeEventTest.php:50
the uploaded paths are newly validated if exist and canonical, of course from browser, thus we must use some paths available within the browser container
3) Behat\Mink\Tests\Driver\Js\WindowTest::testWindow
WebDriver\Exception\InvalidArgument: missing field `handle` at line 3 column 1
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Session.php:269
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:418
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/behat/mink/src/Session.php:311
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/WindowTest.php:17
Selenium 3.x + Chrome is working, Firefox is broken - and only handle must be passed, main window handle must be retrived from webdriver first, null/empty is not supported, nor name is supported (/w or /wo handle)
impossible to fix https://stackoverflow.com/questions/25760357/get-title-of-window-without-switching-in-selenium-webdriver, developer must pass window handle instead of window name (which can also be duplicate)
(only) handle support fixed in https://github.com/instaclick/php-webdriver/pull/119
null handle fixed by retriving all handles and using the 1st one (the results is ordered, so the 1st one should always be main windows handle)
4) Behat\Mink\Tests\Driver\Custom\TimeoutTest::testInvalidTimeoutSettingThrowsException
Failed asserting that exception of type "\Behat\Mink\Exception\DriverException" is thrown.
/w Selenium 2x the following exception is thrown:
Behat\Mink\Exception\DriverException: Error setting timeout: Unknown wait type: invalid
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:336
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:322
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/tests/Custom/TimeoutTest.php:33
Caused by
WebDriver\Exception\UnknownError: Unknown wait type: invalid
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Session.php:319
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:333
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:322
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/tests/Custom/TimeoutTest.php:33
/w Selenium 3.x no exception is thrown with Firefox, in Chrome, exception is thrown, but a different one - WebDriver\Exception\InvalidArgument, test depending on the webdriver impl. removed
5) Behat\Mink\Tests\Driver\Js\EventsTest::testRightClick
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'right clicked'
+'single clicked'
/home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Js/EventsTest.php:45
natively via webdriver it seems impossible/broken, contextClick method is also not supported, can be probably fixed by using syn.js https://github.com/minkphp/MinkSelenium2Driver/pull/358 for right click - https://github.com/bitovi/syn/blob/v0.15.0/test/mouse_button_test.js#L439 (non upgraded syn.js does not trigger any click $this->trigger($xpath, 'rightClick'); (but 'click' is delivered correctly via syn.js))
$this->wdSession->buttondown(array('button' => 2));
$this->wdSession->buttonup(array('button' => 2));
is not emitting right click either
the only fix seems to migrate to new webdriver actions - https://stackoverflow.com/questions/2171589/how-can-you-do-right-click-using-selenium/19123275#19123275
1) Behat\Mink\Tests\Driver\Basic\IFrameTest::testIFrame WebDriver\Exception\InvalidArgument: data did not match any variant of untagged enum FrameId at line 3 column 1 /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Exception.php:198 /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/AbstractWebDriver.php:167 /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/instaclick/php-webdriver/lib/WebDriver/Session.php:286 /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/src/Selenium2Driver.php:426 /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/behat/mink/src/Session.php:321 /home/runner/work/MinkSelenium2Driver/MinkSelenium2Driver/vendor/mink/driver-testsuite/tests/Basic/IFrameTest.php:17test must be fixed, only integer indexes are accepted, when string
idis passed, the following exception is shown:
namekey tested, but it is not supported, not handle, nor element itself (might be supported when serialized correctly, but the latest driver does not support it)WebDriver\Exception\InvalidArgument: invalid argument: 'id' can not be string (Session info: chrome=84.0.4147.105) Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53' System info: host: 'DX6B48F375QD', ip: '10.8.136.130', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '12.0.2'
Does Selenium 3 still follow the JsonWire protocol, that allows id parameter to be a string (see https://www.selenium.dev/documentation/legacy/json_wire_protocol/#sessionsessionidframe)?
Otherwise, an upgrade Selenium is only removing stuff instead of adding it :( .
Offtopic: Some time ago, in the https://github.com/minkphp/MinkSelenium2Driver/issues/298 , we're discussing another driver creation (like WebDriverDriver), that will use https://github.com/php-webdriver/php-webdriver (previously developed at Facebook) internally instead of https://github.com/instaclick/php-webdriver, which supports Selenium 2 only (as per the repo's README.md file).
There are also unmerged/closed PRs with attempts to use https://github.com/php-webdriver/php-webdriver in this driver instead.
Maybe we should move in that direction, @stof ?
