codewars
Time critical code times out with test framework showing no details of partially completed tests by default
When running "performance" Katas, code typically times out repeatedly before arriving at a solution. Now most test frameworks don't print any information on the (partial) completed testcases at all. This is crucial to determine how well you're doing against the goals.
In e.g. Python, you have to force the framework to generate output by sparingly print ('.',flush=True), which runs both the risk of overflowing the output buffer and the risk of needing additional time to execute.
Why not change the test frameworks so that after each test case, output is by default flushed in the CW "terminal".
Since this is not necessarily a language-specific complaint:
can we please have runtimes in Haskell tests? At all?
Why not change the test frameworks so that after each test case,
PR welcome. For Python, see https://github.com/codewars/python-test-framework . I guess simply adding flush=True will work.
can we please have runtimes in Haskell tests? At all?
If I remember correctly, I had to remove it because it required extra nesting for some reason. I'll try to look for the issue. The source of the formatter is at https://github.com/codewars/hspec-formatters-codewars
For Haskell, I misremembered the few changes I made for https://github.com/codewars/codewars-runner-cli/issues/655 so forget what I wrote above about the nesting.
The completion time for each test case/group is not there because it's not provided by Hspec (at least it didn't back then). https://www.codewars.com/kumite/5c703ea1c447bd4b75bafebf?sel=5c703ea1c447bd4b75bafebf is a hack to add it, but I don't remember the details anymore.
EDIT: Found https://github.com/codewars/codewars-runner-cli/issues/699#issuecomment-466498112
It looks like a newer version of Hspec have some functions to get time since the test started, so maybe it's possible once we update. Not sure if there's a Hspec version compatible with the current one though. We might need to wait until we add a new Haskell version.
This mostly has to do with stdout flushing, and is universal across languages. If you manually flush stdout whenever control enters user function, it works fine again.
