mdn-bcd-collector icon indicating copy to clipboard operation
mdn-bcd-collector copied to clipboard

Published 20 hours ago • verified publisher icon openwebdocs

Collect Selenium results for beta browsers

Open Elchi3 opened this issue 1 year ago • 2 comments

I've been manually collecting results on beta browsers but ideally we want to collect results automatically. I think this PR should do that. This will help to reduce manual BCD PRs that people usually submit during the beta phase of browsers. See also https://github.com/mdn/community-meetings/blob/main/23-12-11/minutes.md#bcd where Ruth asked about this.

Elchi3 avatar Dec 13 '23 18:12 Elchi3

In Beta browsers there are sometimes A/B tests which can skew the test results.

For Chrome, I found https://chromium.googlesource.com/chromium-variations/ and so open -a Google\ Chrome\ Beta --args --fake-variations-channel=stable should hopefully disable the variation and make Beta act like a stable browser release. Is it somehow possible to tell Selenium to launch Chrome beta with --fake-variations-channel=stable? (https://github.com/mdn/browser-compat-data/pull/23207#issuecomment-2141479252)

For Firefox, something similar exists, but I haven't done the research yet how to prevent it from happening there.

Elchi3 avatar May 31 '24 08:05 Elchi3