sitespeed.io Chrome Mobile Device Emulation Not Working with Newer Devices

Have you read the documentation?

[X] Yes, I've read the how to make a reproducable bug guide
[X] Yes, I've read the how to debug my script guide

URL

https://www.sitespeed.io

What are you trying to accomplish

From Chrome 97 onwards there are a number of new emulated browsers to test with. However when attempting to use these (I have sent the log output for iPhone XR), I get two errors. The first is "ERROR: No matching viewPort for device iPhone XR. Will use default viewport" and this seems to be coming from here: https://github.com/sitespeedio/browsertime/blob/bcdaeaebbbaa40c845e6171526c642668852c573/lib/support/getViewPort.js The second issue is [2022-02-23 20:25:07] INFO: chrome failed to start, trying 0 more time(s): invalid argument: cannot parse capability: goog:chromeOptions from invalid argument: cannot parse mobileEmulation from invalid argument: 'iPhone XR' must be a valid device from unknown error: must be a valid device

What browser did you use?

Chrome

How to reproduce

docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:23.0.1 https://www.sitespeed.io/ --browsertime.chrome.mobileEmulation.deviceName "iPhone XR"

Relevant log output

Google Chrome 98.0.4758.80 
Mozilla Firefox 97.0
Microsoft Edge 98.0.1108.43 
[2022-02-23 20:25:06] INFO: Versions OS: linux 5.10.76-linuxkit nodejs: v16.13.2 sitespeed.io: 23.0.1 browsertime: 15.0.1 coach: 7.1.0
[2022-02-23 20:25:06] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:06] INFO: Running tests using Chrome - 3 iteration(s)
[2022-02-23 20:25:06] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:06] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 2 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 1 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 0 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: BrowserError: Could not start chrome with 3 tries
    at SeleniumRunner.start (/usr/src/app/node_modules/browsertime/lib/core/seleniumRunner.js:87:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Iteration.run (/usr/src/app/node_modules/browsertime/lib/core/engine/iteration.js:113:7)
    at async Engine.runByScript (/usr/src/app/node_modules/browsertime/lib/core/engine/index.js:266:20)
    at async Object.analyzeUrl (/usr/src/app/lib/plugins/browsertime/analyzer.js:152:21)
    at async Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:150:26)
[2022-02-23 20:25:07] ERROR: No data to collect
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 2 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 1 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 0 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: BrowserError: Could not start chrome with 3 tries
    at SeleniumRunner.start (/usr/src/app/node_modules/browsertime/lib/core/seleniumRunner.js:87:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Iteration.run (/usr/src/app/node_modules/browsertime/lib/core/engine/iteration.js:113:7)
    at async Engine.runByScript (/usr/src/app/node_modules/browsertime/lib/core/engine/index.js:266:20)
    at async Object.analyzeUrl (/usr/src/app/lib/plugins/browsertime/analyzer.js:152:21)
    at async Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:150:26)
[2022-02-23 20:25:07] ERROR: No data to collect
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 2 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 1 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: No matching viewPort for device iPhone XR. Will use default viewport
[2022-02-23 20:25:07] INFO: chrome failed to start, trying 0 more time(s): invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device
[2022-02-23 20:25:07] ERROR: BrowserError: Could not start chrome with 3 tries
    at SeleniumRunner.start (/usr/src/app/node_modules/browsertime/lib/core/seleniumRunner.js:87:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Iteration.run (/usr/src/app/node_modules/browsertime/lib/core/engine/iteration.js:113:7)
    at async Engine.runByScript (/usr/src/app/node_modules/browsertime/lib/core/engine/index.js:266:20)
    at async Object.analyzeUrl (/usr/src/app/lib/plugins/browsertime/analyzer.js:152:21)
    at async Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:150:26)
[2022-02-23 20:25:07] ERROR: No data to collect
[2022-02-23 20:25:07] ERROR: Caught error from Browsertime TypeError: Cannot read properties of undefined (reading 'alias')
    at Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:167:52)
[2022-02-23 20:25:08] INFO: HTML stored in /sitespeed.io/sitespeed-result/www.sitespeed.io/2022-02-23-20-25-06

Feb 23 '22 20:02 shaqb

Hi @shaqb thanks for creating the issue!

The first part Is that we need to internally have the viewport, so we can set it correct size for the video (if I remember it correctly). Can you help me with which devices you wanna use and I can add them.

The other error comes directly from Chrome, I need to look into that.

Feb 23 '22 20:02 soulgalore

Hi again @shaqb it looks like a bug in Chromedriver. You can enable the log with --browsertime.chrome.enableChromeDriverLog.

[1645650039.690][INFO]: Starting ChromeDriver 98.0.4758.48 (d869ab3eda60629b9fabbd4e30c0f833466c83fd-refs/branch-heads/4758@{#415}) on port 55348
...
[1645650039.710][INFO]: [415553500af60d2d5c311b275aa6ba8d] RESPONSE InitSession ERROR invalid argument: cannot parse capability: goog:chromeOptions
from invalid argument: cannot parse mobileEmulation
from invalid argument: 'iPhone XR' must be a valid device
from unknown error: must be a valid device

If you could try latest beta Chromedriver 99 and see if its fixed there: https://chromedriver.storage.googleapis.com/index.html?path=99.0.4844.35/

If that do not work, do you have to create an issue at https://bugs.chromium.org/p/chromedriver/issues/list? Else I can create an issue on Friday.

Feb 23 '22 21:02 soulgalore

Hi @soulgalore @shaqb

Looks like a ChromeDriver bug is indeed required.

Similar to the previous issue here: https://bugs.chromium.org/p/chromedriver/issues/detail?id=3474

The following file contains a list of supported devices - and this hasn't been updated since 08/2020 (ChromeDriver 87): https://source.chromium.org/chromium/chromium/src/+/master:chrome/test/chromedriver/chrome/mobile_device_list.cc;bpv=1

So, any newly supported devices added since then will not be supported until ChromeDriver is updated accordingly.

Feb 24 '22 10:02 barryperez

Hi @soulgalore @barryperez - raised https://bugs.chromium.org/p/chromedriver/issues/detail?id=4044. Hopefully they should update to the latest in an upcoming version!

Feb 24 '22 11:02 shaqb

Thank you @shaqb !

Feb 24 '22 11:02 soulgalore

This is fixed in Browsertime now, waiting on Chromedriver.

Mar 12 '22 17:03 soulgalore

Hey @soulgalore! So looks like this now works on ChromeDriver 113. See the status here: https://bugs.chromium.org/p/chromedriver/issues/detail?id=4044.

Are you able to test this in the next version of sitespeed.io when you incorporate ChromeDriver 113?

Only took them a year to fix it!

Mar 29 '23 10:03 shaqb

Yes I can test that, 113 isn't released yet though, but maybe the beta will be there in a couple of days.

Mar 29 '23 10:03 soulgalore

I verified with XR and this works now.

Dec 27 '23 09:12 soulgalore