svaarala
Test262 results for 2.3.0 using C++ harness
Hello!
Today I ran Test262 against ver. 2.3.0 of the runtime, in MacOS. For that, I used a test harness I created in C++ for both Windows and MacOS. The results show that 20141 of the 35887 tests did not pass.
Most of the results make complete sense, since they refer to functionality that is not yet implemented in the runtime - and in some cases, to things that the test harness itself did not implement. There are a few, however, that do not seem to have a clear explanation - /built-ins/Array/prototype/filter/15.4.4.20-3-12.js comes to mind.
Attached are the results generated by the test harness: test_results.txt.zip
I just thought you might want to have these results handy for future releases.
The harness I used is currently located at https://github.com/Izhido/test262_harness_cpp .
(I do realize I might be out of place by posting something like this in the issue tracker. If so, a thousand apologies - just let me know).
Update: I just ran the test in a Windows machine, as well. Results are now 20144 of 35887.
I should probably mention that the harness specifically excludes 3 tests from running, since they cause a stack overflow in Windows:
- /RegExp/S15.10.2.5_A1_T5.js
- /RegExp/S15.10.2.9_A1_T5.js
- /RegExp/S15.10.2.8_A3_T15.js
Results are attached here: test_results_win.zip
Thanks, I'll check the results out. For Duktape testing the best test case set would be the ES5.1 subset; although some ES2015+ features are implemented, naturally most ES2015+ tests will fail due to missing functionality.
For the test cases that fail due to stack overflow, you can probably work around it by increasing the stack size when compiling the test harness and/or Duktape. The default stack size of 1MB is very small compared to most platforms and woefully inadequate for some use cases.
The make test262test target runs the ES5 test suite: https://github.com/svaarala/duktape/blob/master/Makefile#L750-L757