xbox_kernel_test_suite icon indicating copy to clipboard operation
xbox_kernel_test_suite copied to clipboard

Automatic regression testing

Open PatrickvL opened this issue 7 years ago • 5 comments

It would be very valuable if the kernel tests could be ran as part of a build process, so that projects like Cxbx-Reloaded, xqemu, OpenXbox or other parties can somehow run these kernel tests, and compare it against a reference output, so that regressions can be spotted directly after making changes.

PatrickvL avatar Jan 04 '18 00:01 PatrickvL

The problem here is that Travis (our CI) is running Linux, and CxBx-Reloaded is currently Windows only. It might be possible to install wine, running tests and the upload the results, but this will be very hacky.

Luca1991 avatar Mar 06 '18 20:03 Luca1991

Ouch, that could prove difficult. But what if we'd do it the other way around - let the Cxbx-Reloaded AppVeyor build run the most recent kernel test suite?

PatrickvL avatar Mar 06 '18 23:03 PatrickvL

Yeah, this might be possible. It should be possible to expand the appveyor build script to download the latest build of the kernel test suite and run it just after compiling cxbx.

If I'm not wrong, the code written by @Fisherman166 avoid the creation of the log file when running on emu. This is an issue in this case, as we need to create the log file regardless of our running platform. I might be wrong on this... I'll check tomorrow.

Luca1991 avatar Mar 07 '18 00:03 Luca1991

It is true that only the real hardware creates a logfile. Attempting to run the same code that works on the Xbox hardware on Cxbx-Reloaded will cause errors, likely because the virtual file system in the Cxbx kernel is not fully implemented. Until the Cxbx file system is more robust, I don't think there is a better way to get output from the kernel test suite than redirecting Cxbx kernel output to a krnldebug.txt file.

Fisherman166 avatar Mar 07 '18 05:03 Fisherman166

Using xqemu or any emulators that require kernel bios will require self-hosted CI. Any cloud service to run the test with required kernel bios are likely not permitted. But I'm not a lawyer. 🤷‍♂️

RadWolfie avatar May 18 '23 15:05 RadWolfie