devtools-protocol

devtools-protocol copied to clipboard

Would there be value in having a shared repo of protocol test cases for the ones which already pass in multiple implementations? IIRC @brendyna said they rely on a copy of a bunch of the Chrome tests.

Pragmatically it may be most convenient to store any tests in web-platform-tests (probably marked tentative). That way we get at least automatic 2-way-sync, so Chromium and Firefox devs at least can use and contribute to them without any additional overhead.

At TPAC this year, @thejohnjansen noted that the EDP implementation had heavily used Chrome's tests, and in fact only on those tests and not reverse engineering CDP in Chrome. Quite clearly, sharing these tests would be valuable, perhaps moreso than any spec prose we write.

@foolip @thejohnjansen and I were chatting this morning, and we actually ended up writing a significant number of tests for EDP's implementation in addition to what was there in CDP as we found the coverage for individual APIs was relatively low. Those were written in TypeScript but could be massaged/ported into web-platform-tests with some work. We also see that sharing as valuable from our end.

@brendyna that sound really exciting! Could you paste a representative test so I can understand the structure a bit? When we integrate this into web-platform-tests they will need some special setup just like WebDriver tests, I'd like to understand mostly what the code sending CDP commands looks like and what sort of assertions are made.

Sure, let me start an thread w/you & one of our dev leads to get an example of this pulled up. I'll ask that he put the example here, but we'll kick off the convo in email as he's not part of this WICG.