sim_if: continue after compilation failure if '--keep-compiling' is True

[umarcor]

Currently, option --keep-compiling will raise an exception/error after all files are compiled. That is, it allows to continue while compilation failures are produced, but it does not allow to run the tests (those whose sources were successfully compiled, at least). This PR changes it, so that execution continues after compilation when -k is used.

[LarsAsplund]

I haven't reviewed this yet but a gitter discussion reminded me that the compile error must never be forgotten. It should be seen in the test summary and the return code should not be zero

[umarcor]

@LarsAsplund the main purpose of this PR is to remove https://github.com/VHDL/Compliance-Tests/blob/master/run.py#L30-L69 by using --keep-compiling in order to run the tests which could be compiled. If we force the return code to be non-zero, we need some additional feature to cover the capability required in Compliance-Tests.

[LarsAsplund]

I think that use case is perfectly valid but do you really want the default behavior for a failing compliance test to be return code = 0?

[umarcor]

I think that use case is perfectly valid but do you really want the default behavior for a failing compliance test to be return code = 0?

Only in the cases when the user has explicitly set --keep-compiling. To my understanding, --keep-compiling invalidates the meaning of the exit code. The user is forced to read the log (in the terminal or the XML report) in order to know what happened. It is known that the exit code is a failure, because using --keep-compiling would not be required otherwise.

I would expect the exit code to be meaningful if we had some xfail behaviour, such as the one in pytest. So that the user would say "I want to keep compiling and I expect it to fail (exit code 0) but let me know if nothing fails (exit code 1)". That is, an inversion of the logic/meaning of the exit code when --keep-compiling is used.