zeppelin icon indicating copy to clipboard operation
zeppelin copied to clipboard

[ZEPPELIN-6195] Fix Selenium Runtime.evaluate error by downgrading Chrome/ChromeDriver to 125 and upgrading selenium-java to 4.20.0

Open ParkGyeongTae opened this issue 6 months ago • 2 comments

What is this PR for?

This PR addresses a test failure in the GitHub Actions job test-selenium-with-spark-module-for-spark-3-4. The failure was caused by ChromeDriver not recognizing the Runtime.evaluate command, which is internally used by Selenium when executing JavaScript.

To resolve this issue, two changes were applied:

  1. Upgraded Selenium (selenium-java) to 4.20.0 to ensure compatibility with modern DevTools commands.
  2. Explicitly downgraded Chrome and ChromeDriver to version 125.0.6422.60, using Chrome for Testing binaries, to avoid compatibility issues introduced in newer versions (e.g., Chrome 136+).

What type of PR is it?

Bug Fix

Todos

  • [x] - Verify other integration tests (if any) are not affected
  • [x] - Confirm long-term compatibility with Chrome for Testing approach

What is the Jira issue?

  • Jira: https://issues.apache.org/jira/browse/ZEPPELIN/ZEPPELIN-6195

How should this be tested?

  • Run the test-selenium-with-spark-module-for-spark-3-4 job in GitHub Actions.
  • Ensure that AuthenticationIT and other affected Selenium tests pass without triggering Runtime.evaluate errors.
  • Confirm that the installed Chrome and ChromeDriver versions match (125.0.6422.60).

Screenshots (if appropriate)

N/A

Questions:

  • Does the license files need to update? No.
  • Is there breaking changes for older versions? No.
  • Does this needs documentation? No.

ParkGyeongTae avatar Jun 07 '25 03:06 ParkGyeongTae

Thank you for wanting to correct the Selenium tests. I see that your solution works. However, I find the downgrade very problematic, as it is not future-proof.

I had also looked at the problem some time ago and found a problem with the maximize method. https://github.com/apache/zeppelin/blob/94ae845cd4a7fd6058e936d82f817e1ef8618a64/zeppelin-integration/src/test/java/org/apache/zeppelin/WebDriverManager.java#L159

The problem with Selenium is analyzed in more detail here. https://github.com/SeleniumHQ/selenium/issues/15358

I would suggest a fixed height and width. Unfortunately, my initial tests were not successful. Perhaps you can find suitable values.

I like the update of the Selenium version, but not the downgrade of Chrome.

Reamer avatar Jun 10 '25 07:06 Reamer

@Reamer I always appreciate your feedback. I understand your point, and I’ve reviewed the discussion in the link you shared. I’ll look into a way to keep Chrome up to date while ensuring the tests run without errors.

ParkGyeongTae avatar Jun 10 '25 09:06 ParkGyeongTae

The issue looks to be there for a long time, if it is specific to Chrome, how about switching the test to Firefox?

pan3793 avatar Aug 01 '25 14:08 pan3793

Firefox has other bugs as far as I remember

Reamer avatar Aug 02 '25 11:08 Reamer

@pan3793 @Reamer Thanks for the suggestion! I switched the tests to Firefox, and they are now passing successfully. Please take a look and let me know if you have any further concerns.

ParkGyeongTae avatar Aug 02 '25 15:08 ParkGyeongTae

I see the failed run-e2e-tests-in-zeppelin-web still prints

[INFO] [09:56:05] I/launcher - Running 1 instances of WebDriver
[INFO] [09:56:05] I/direct - Using ChromeDriver directly...

pan3793 avatar Aug 06 '25 10:08 pan3793

@ParkGyeongTae thanks for fixing this, could you please update the PR title and description to reflect the final solution?

pan3793 avatar Aug 08 '25 08:08 pan3793

@pan3793 I’ve updated the PR title and description, as well as the Jira issue title and description.

ParkGyeongTae avatar Aug 08 '25 08:08 ParkGyeongTae

Merged into master and branch-0.12

Reamer avatar Aug 08 '25 09:08 Reamer