test262.fyi
test262.fyi copied to clipboard
CanadaHonk
Babel tests
We run test262 on Babel and core-js regularly, and it would be very nice to show the results on https://test262.fyi.
I just noticed that our test262 runner on CI is completely broken and it's not reporting results, but for the purpose of this issue let's assume that it works, while I fix it 😅
We run test262 on every commit, and store the result in a test262-result artifact -- you can find the last one https://github.com/babel/babel/actions/runs/5179541745.
We run tests by compiling them to ES5 using Babel, injecting the core-js polyfill, and running the resulting code in a very old version of Node.js (0.10) which only supported ES5. This means that our test results aren't significant for pre-ES6 behavior, but they are useful for features introduced since then.
We use https://github.com/babel/babel-test262-runner/ to run them, which produces tap output. The readme doesn't mention it, but it's possible to specify the path to a different test262 checkout using the TEST262_PATH=.... env variable.
Babel CI and test runner fixed! The test262-results artifact at https://github.com/babel/babel/actions/runs/5198849397 not contains the results.
Something is still broken, many more tests than expected are failing.
Also btw @nicolo-ribaudo, I maintain a fork of eshost (no node changes yet) and test262-stream (more accurate results) if you have requests / want to switch to them later in Babel's runner for best results.
Thank you, I'll take a look!
I fixed for real Babel's test262 runner, it's now generating the results at https://github.com/babel/babel/actions/runs/5198849397
After a lot of tinkering, Babel builds should now be generated with the same harness (after some updates) while using Babel's test262 transpile script.
Oh awesome :) Let's keep this issue open for a while, so I can check if there is anything that can be improved for the Babel runner and I'll post comments here.
