jgraham

FWIW I don't think the numbers should be massaged simply to help people make cheap cross-browser comparisons. The differences we are seeing are presumably real, and provide a clue that...

I don't think we have that information, and I don't think it's particularly useful even if we do. We don't have the information because we have no way of knowing...

I think people can notice that different implementations ran different numbers of tests, and infer that something is worth investigating. Evidence: you noticed and filed this bug. I also think...

``` TypeError: results.import is undefined[Learn More] wpt-error-report:30:7 https://bocoup.github.io/wpt-error-report/:30:7 ```

Thanks for fixing the page (and for doing this work in the first place of course). It's disappointing that you're still using a nonstandard feature (HTML imports) in an interoperability...

> As an aside, I think we could consider guarding against tests that don't run to completion in all target browsers with the PR build bot I think that probably...

Using Xvfb is more like a real user experience, certainly. Xvfb uses the normal Firefox code but renders to a framebuffer that is never drawn to the screen. Headless mode...

I think that we should make `--headless` an option to wptrunner that abstracts the details of enbling headless mode across different browsers. It could make sense to enable that for...

Hah. `cffi` seems to be a dep of `pyOpenSSL`, which depends on `cffi [required: >=1.8,!=1.11.3, installed: 1.11.5]`. I don't think that pinning transitive dependencies is supported in any *good* way,...

Ooh, pip seems to have a `--prefer-binary` option. I think that might help here.