XPMCK icon indicating copy to clipboard operation
XPMCK copied to clipboard

Add a Testing Framework

Open bazz1tv opened this issue 8 years ago • 0 comments

Currently, this is "just a dream" and should be considered a more long-term goal.

A testing framework can ensure that new contributions do not somehow mangle proper functionality in the program as a whole. And yes, sometimes modifying part A to a program can somehow unexpectedly mess up part B!! A testing framework takes lots of work to create, but can potentially save lots of debug time and is an official way of ensuring complete and proper functionality throughout the product, and throughout development cycles.

Ideally the functionality of XPMCK can completely (or at least to a very large degree) be tested programmatically. Here are some examples of test cases (off the dome):

  • ensure proper VGM is exported for all VGM-export supported systems
    • This requires answering the question "What constitutes a properly exported VGM?" -- likely a "target VGM" from a particular test MML file is considered the desired result and is used as a comparison file.
  • ensure MML commands are functioning properly.

I'm thinking of ways to programmatically test some of the "you-need-to-hear-this-to-verify" cases. I'm not really sure at all. Perhaps, If simple test MML files are crafted with their pre-verified proper output recorded to WAV as the basis for a comparison test to newly compiled ones. This is something that might be a solution for many MML effects and such. This solution would only "break" if some contribution intentionally modifies the "regular audio output" or intentionally modifies an effect.. in which case the test case(s) must be updated.

bazz1tv avatar Apr 09 '16 07:04 bazz1tv