Skript icon indicating copy to clipboard operation
Skript copied to clipboard

Tests to return their own results

Open TheLimeGlass opened this issue 2 years ago • 3 comments

Suggestion

Have Skript unit tests return a custom results so that in the case that they do error, we can provide a full detailed list of how and why the tests failed individually.

Also add support for delays in the Skript unit tests.

Also collect test Throwables to report at the end of the test with a failure.

These are also sorta labeled in #3767

TheLimeGlass avatar Jul 26 '22 18:07 TheLimeGlass

Currently extra information is usually given in the assert error message, for example https://github.com/SkriptLang/Skript/blob/e9d04e7604c13d0de8d210f88ea0f021cdace009/src/test/skript/tests/regressions/4672-looping-expression-list-single-variables.sk#L10

Though this context is definitely not added in every message, it could be. Does this not give enough opportunity to give information, and if so, what do you suggest?

TPGamesNL avatar Jul 27 '22 09:07 TPGamesNL

My plans where to implement something into the structure of the test, collect results when it finishes the section node (This allows for delay) and then return that as a collection of results for TestResults.class. Things like time taken, errors, parse errors, extra string details, variable information, etc can all be added to the individual test results.

Probably something for after the structure API.

TheLimeGlass avatar Jul 27 '22 09:07 TheLimeGlass

Sounds good 👍

TPGamesNL avatar Jul 27 '22 09:07 TPGamesNL