helium icon indicating copy to clipboard operation
helium copied to clipboard

Support for selenium==4.1.0, gekodriver=0.30.0 and chromedriver=97

Open petrisorionel opened this issue 3 years ago • 15 comments

Support for latest Selenium version 4.1.0

petrisorionel avatar Jan 17 '22 17:01 petrisorionel

@mherrmann please don't merge.

still trying to fix 4 failing tests and 1 warning.

selenium-python-helium\helium\_impl\__init__.py:110: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  result = Chrome(
FAIL: test_click_top_left (tests.api.test_point.PointTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "github\selenium-python-helium\tests\api\test_point.py", line 43, in test_click_top_left
    self.assert_result_is(
  File "github\selenium-python-helium\tests\api\test_point.py", line 107, in assert_result_is
    self.assert_around(
  File "github\selenium-python-helium\tests\api\test_point.py", line 38, in assert_around
    self.assertIn(
AssertionError: 50 not found in [-1, 0, 1] : Offset (50, 15) is not in expected range (0+-1, 0+-1).

petrisorionel avatar Jan 17 '22 17:01 petrisorionel

@mherrmann it is a way that I can run the tests without headless option?

petrisorionel avatar Jan 17 '22 17:01 petrisorionel

@petrisorionel you can edit tests/api/init.py line number 65.

mherrmann avatar Jan 17 '22 18:01 mherrmann

@mherrmann maybe you have an idea how to fix these ones (on Chrome)

======================================================================
FAIL: test_click_top_left (tests.api.test_point.PointTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 43, in test_click_top_left
    self.assert_result_is(
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 107, in assert_result_is
    self.assert_around(
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 38, in assert_around
    self.assertIn(
AssertionError: 50 not found in [-1, 0, 1] : Offset (50, 15) is not in expected range (0+-1, 0+-1).

======================================================================
FAIL: test_doubleclick_top_left (tests.api.test_point.PointTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 84, in test_doubleclick_top_left
    self.assert_result_is(
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 107, in assert_result_is
    self.assert_around(
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 38, in assert_around
    self.assertIn(
AssertionError: 50 not found in [-1, 0, 1] : Offset (50, 15) is not in expected range (0+-1, 0+-1).

======================================================================
FAIL: test_hover_top_left (tests.api.test_point.PointTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 56, in test_hover_top_left
    self.assert_result_is(
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 107, in assert_result_is
    self.assert_around(
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 38, in assert_around
    self.assertIn(
AssertionError: 50 not found in [-1, 0, 1] : Offset (50, 15) is not in expected range (0+-1, 0+-1).

======================================================================
FAIL: test_rightclick_top_left (tests.api.test_point.PointTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 69, in test_rightclick_top_left
    self.assert_result_is(
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 107, in assert_result_is
    self.assert_around(
  File "\desktop\github\selenium-python-helium\tests\api\test_point.py", line 38, in assert_around
    self.assertIn(
AssertionError: 50 not found in [-1, 0, 1] : Offset (50, 15) is not in expected range (0+-1, 0+-1).

----------------------------------------------------------------------
Ran 306 tests in 207.003s

FAILED (failures=4)
Test failed: <unittest.runner.TextTestResult run=306 errors=0 failures=4>
error: Test failed: <unittest.runner.TextTestResult run=306 errors=0 failures=4>

petrisorionel avatar Jan 17 '22 19:01 petrisorionel

Firefox failing tests.

======================================================================
ERROR: test_interact_with_found_elements (tests.api.test_find_all.FindAllTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "\desktop\github\selenium-python-helium\tests\api\test_find_all.py", line 31, in test_interact_with_found_elements
    id_ = text_field.web_element.get_attribute('id')
  File "\Desktop\GitHub\selenium-python-helium\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 155, in get_attribute
    attribute_value = self.parent.execute_script(
  File "\Desktop\GitHub\selenium-python-helium\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 878, in execute_script
    return self.execute(command, {
  File "\Desktop\GitHub\selenium-python-helium\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "\Desktop\GitHub\selenium-python-helium\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Web element reference not seen before: {"element-6066-11e4-a52e-4f735466cecf":"f692518c-cc44-46fb-945c-350a365e33be"}
Stacktrace:
WebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:183:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.jsm:395:5
element.resolveElement@chrome://remote/content/marionette/element.js:673:11
evaluate.fromJSON@chrome://remote/content/marionette/evaluate.js:254:26
evaluate.fromJSON/<@chrome://remote/content/marionette/evaluate.js:245:38
evaluate.fromJSON@chrome://remote/content/marionette/evaluate.js:245:20
evaluate.fromJSON@chrome://remote/content/marionette/evaluate.js:262:29
receiveMessage@chrome://remote/content/marionette/actors/MarionetteCommandsChild.jsm:79:29


======================================================================
ERROR: test_access_attributes_across_iframes (tests.api.test_iframe.IframeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "\desktop\github\selenium-python-helium\tests\api\test_iframe.py", line 20, in test_access_attributes_across_iframes
    self.assertEqual("This text is inside an iframe.", text.value)
  File "\desktop\github\selenium-python-helium\helium\__init__.py", line 679, in value
    return self._impl.value
  File "\desktop\github\selenium-python-helium\helium\_impl\__init__.py", line 840, in value
    return self.first_occurrence.text
  File "\desktop\github\selenium-python-helium\helium\_impl\selenium_wrappers.py", line 86, in f_decorated
    return f(self, *args, **kwargs)
  File "\desktop\github\selenium-python-helium\helium\_impl\selenium_wrappers.py", line 128, in text
    return self.target.text
  File "\Desktop\GitHub\selenium-python-helium\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 77, in text
    return self._execute(Command.GET_ELEMENT_TEXT)['value']
  File "\Desktop\GitHub\selenium-python-helium\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 710, in _execute
    return self._parent.execute(command, params)
  File "\Desktop\GitHub\selenium-python-helium\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "\Desktop\GitHub\selenium-python-helium\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Web element reference not seen before: {"element-6066-11e4-a52e-4f735466cecf":"8b38df02-6d89-4244-9b77-d9724ee4ae1e"}
Stacktrace:
WebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:183:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.jsm:395:5
element.resolveElement@chrome://remote/content/marionette/element.js:673:11
evaluate.fromJSON@chrome://remote/content/marionette/evaluate.js:254:26
evaluate.fromJSON@chrome://remote/content/marionette/evaluate.js:262:29
receiveMessage@chrome://remote/content/marionette/actors/MarionetteCommandsChild.jsm:79:29


----------------------------------------------------------------------
Ran 306 tests in 333.169s

FAILED (errors=2, skipped=2)
Test failed: <unittest.runner.TextTestResult run=306 errors=2 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=306 errors=2 failures=0>

petrisorionel avatar Jan 17 '22 19:01 petrisorionel

I won't have time to help other than to review and merge the PR, sorry.

mherrmann avatar Jan 17 '22 22:01 mherrmann

@mherrmann for selenium==3.141.0 and geckodriver=0.26.0 there are the same failing tests as for selenium==4.1.0 and geckodriver=0.30.0 - test_access_attributes_across_iframes (tests.api.test_iframe.IframeTest), test_interact_with_found_elements (tests.api.test_find_all.FindAllTest).

but when I run them separately they pass without any problem.

so, from my point of view on Firefox we are good.

petrisorionel avatar Jan 18 '22 09:01 petrisorionel

@mherrmann seams to be an issue on Selenium side, I tried the same scenario with selenium==4.1.0 and seams to fail.

you can see this example in the code in tests.api.test_point.PointTest.test_click_top_left_selenium

	def test_click_top_left_selenium(self):
		driver = self.driver
		el = driver.find_element(By.XPATH, "//button[text()='Button 1']")
		action = ActionChains(driver)
		action.move_to_element_with_offset(el, 0, 0)
		action.click()
		action.perform()
		self.assert_result_is(
			"Button 1 clicked at offset (0, 0).", offset_delta=(1, 1)
		)
return

Traceback (most recent call last):
  File "\Desktop\GitHub\selenium-python-helium\tests\api\test_point.py", line 56, in test_click_top_left_selenium
    self.assert_result_is(
  File "\Desktop\GitHub\selenium-python-helium\tests\api\test_point.py", line 120, in assert_result_is
    self.assert_around(
  File "\Desktop\GitHub\selenium-python-helium\tests\api\test_point.py", line 41, in assert_around
    self.assertIn(
AssertionError: 49 not found in [-1, 0, 1] : Offset (49, 14) is not in expected range (0+-1, 0+-1).

also I made a separated project.

test_top_left.py::test_click_top_left_selenium FAILED                    [100%]
test_top_left.py:6 (test_click_top_left_selenium)
Button 1 clicked at offset (49, 14). != Button 1 clicked at offset (0, 0).

Expected :Button 1 clicked at offset (0, 0).
Actual   :Button 1 clicked at offset (49, 14).
<Click to see difference>

def test_click_top_left_selenium():
        driver = webdriver.Chrome()
        driver.get("file:///C:/Users/user/Desktop/GitHub/selenium-python-helium/tests/api/data/test_point.html")
        el = driver.find_element(By.XPATH, "//button[text()='Button 1']")
        action = ActionChains(driver)
        action.move_to_element_with_offset(el, 0, 0)
        action.click()
        action.perform()
>       assert driver.find_element(By.ID, "result").text == "Button 1 clicked at offset (0, 0)."
E       AssertionError: assert 'Button 1 clicked at offset (49, 14).' == 'Button 1 clicked at offset (0, 0).'

test_top_left.py:15: AssertionError

also @dragosdaniil can confirm that we have the same issue on Java.

petrisorionel avatar Jan 18 '22 12:01 petrisorionel

If we agree I will remove the test_click_top_left_selenium from the code and we can start the review.

I opened a bug - https://github.com/SeleniumHQ/selenium/issues/10261

petrisorionel avatar Jan 18 '22 12:01 petrisorionel

If I understand correctly, then this PR makes one additional test fail: test_click_top_left. This test does not fail with Selenium 3. So my question is what is the advantage of using Selenium 4 that would outweigh breaking existing functionality?

mherrmann avatar Jan 18 '22 14:01 mherrmann

(And thank you for the PR. I do appreciate it but have the doubt described above.)

mherrmann avatar Jan 18 '22 14:01 mherrmann

Selenium 4 it is more fast then 3 because is W3C compliant now. and add some extra features like

  • Relative Locators (I know that we already have this in Helium)
  • Better Windows/Tab management
  • New API for Chrome DevTools Protocol

I saw some changes in Actions Class,, we can keep this PR open and I will do more research for these changes.

petrisorionel avatar Jan 18 '22 16:01 petrisorionel

@mherrmann see https://github.com/SeleniumHQ/selenium/issues/10261#issuecomment-1016011257

I will try to fix the tests.

petrisorionel avatar Jan 19 '22 12:01 petrisorionel

@mherrmann because selenium 4.1.0 is now W3C compliant now and if we follow the documentation.

Let x element and y element be the result of calculating the in-view center point of element. and https://w3c.github.io/webdriver/#dfn-center-point

I think we need to do something like this.

	def test_click_top_left(self):
		click(Button("Button 1").top_left + (1, 1))
		self.assert_result_is(
			"Button 1 clicked at offset (0, 0).", offset_delta=(1, 1)
		)

Ran 306 tests in 204.848s

OK

(venv) C:\Users\user\Desktop\GitHub\selenium-python-helium>

petrisorionel avatar Jan 20 '22 09:01 petrisorionel

So does click(Button(...).top_left) click the top left corner or the center? I'm a little confused why the + (1, 1) in the test fixes the issue.

mherrmann avatar Jan 20 '22 15:01 mherrmann

@petrisorionel Can you explain the logic behind your (1,1) tuple fixing the delta tests? Thanks! 🌮

Islati avatar Jan 28 '23 16:01 Islati

I already explained above in a comment.

petrisorionel avatar Jan 28 '23 19:01 petrisorionel