code-dot-org icon indicating copy to clipboard operation
code-dot-org copied to clipboard

Upgrade eyes_selenium to 6.3.7

Open snickell opened this issue 1 year ago • 2 comments

While upgrading Ruby to 3.3 (PR), we discovered two compatibility issues between our old version of eyes_selenium 3.18.4 and Ruby 3.3.

  1. eyes_selenium 3.18.4 has a dependency on the state_machine gem, which is not compatible with Ruby >= 3.2, and appears not to be receiving updates either. state_machine gem appears to have been forked into the state_machines gem, which is used by newer eyes_selenium versions and is fixed.
  2. eyes_selenium passes the faraday gem a hardcoded SSL cert for the applitools server in a way that breaks openssl on Ruby >= 3.1.

snickell avatar Oct 15 '24 22:10 snickell

A previous (merged to test) attempt saw numerous eyes failures: applitools eyes failures during eyes_selenium upgrade

snickell avatar Oct 15 '24 22:10 snickell

Possible Timing Issues

Solution eval dialog not popped up

image

Videos not rendered

image

Timing Etc

image

Microscopic differences in font rendering:

Example 1

image

Example 2

image

Possible example 3

image

This is a microscopic difference, there's a little pinch of white pixel in the non-basline.

Time travelling

image

"Saved In a few seconds" whereas baseline is "Saved a few seconds ago" 🤯

Teacher Feedback tab not present

image

Viewport Issues

Captured full page but baseline only shows viewport

image

Captured page wider than baseline

image

Baseline includes empty capture beyond viewport

In this case, it almost seems like the baseline PNG is wrong: image

Captured page cuts off viewport

image

Note that this may be capturing wider than the viewport. Both of these might be considered buggy, perhaps this is CSS stitching mode or something?

snickell avatar Oct 15 '24 22:10 snickell