Should outputs of gerenated code also need to be added in stable?

[cd155]

Drasil generates code. The generated code also generated some outputs. I just wonder if it is a good idea to have outputs that are generated by generated code also need to be included in the stable. In other words, how to verify whether generated code will produce consistent results.

Right now, we are manually checking whether outputs of generated code are consistent, but is it a good idea to automate this part?

[JacquesCarette]

This generalizes to: how do we insure that the code we generate is "as expected", not just textually, but that it actually works too. And absolutely, it should be automated.

However, we haven't done this yet because we don't want to force all developers in Drasil to install development environments for all the languages. We should likely do so for some CI versions though. There are details to be worked out, as installing everything takes a long time. So maybe not for all check-ins, but as a pass before a PR is accepted, yes.

The other question is: do we really want to do this using our very crude method (textual output in stable)? Probably not.

[smiths]

We don't currently generate test cases for our case studies. This would be a good thing to add in the future.