fuzion icon indicating copy to clipboard operation
fuzion copied to clipboard

replace all "negative" tests with "simple and negative" tests, one by one

Open simonvonhackewitz opened this issue 1 year ago • 1 comments

There are unintended errors in negative tests which are ignored because they don't happen in a line which is supposed to produce an error, see #3684.

I tried to check the total error count, but that does not work, because sometimes multiple lines which are marked to produce an error are reported in one error (e.g. in tests/abstractfeatures_negative 12 lines are commented to produce errors in the source code but the error count when running it is four).

With "simple and negative" tests, there would still be the possibility to accidentally rerecord new errors, but at least there would be a chance to notice them. And it would still protect against accidental rerecording of missing expected errors

simonvonhackewitz avatar Sep 04 '24 14:09 simonvonhackewitz

probably yes. But we should do this maybe one test at a time and verify carefully if the errors are really what was intended by the test. I would like to avoid having errors in simple-and-negative tests that are actually wrong, better disable some tests and create issues if that is the case.

fridis avatar Sep 05 '24 10:09 fridis