interop
interop copied to clipboard
Monitoring for tests with precondition failures, harness errors, or other problems with the tests
In https://github.com/web-platform-tests/interop/issues/319 there was a problem with "PRECONDITION_FAILED" in some WebCodecs tests.
I think the conclusion is that we can't have tests like these in Interop, because there's no good way to score them.
We should have a system for finding such tests and periodically review it. It could be as simple as a (very long) wpt.fyi query.
Something like https://wpt.fyi/results/?label=master&label=experimental&aligned&view=interop&q=label%3Ainterop-2024-indexeddb%20status%3Aerror is a small subset of such a query, currently reveling a harness error in Chrome.