test-suite icon indicating copy to clipboard operation
test-suite copied to clipboard

Published 20 hours ago • verified publisher icon solid-contrib

number of tests reported varies

Open michielbdejong opened this issue 5 years ago • 4 comments

If you look at https://github.com/solid/test-suite/pull/64 you see that the number of tests for which an earl:outcome triple is produced, is sometimes 45, sometimes 27, and sometimes 49. Why is that?

michielbdejong avatar Jan 08 '20 14:01 michielbdejong

When i re-ran, it reported 48 instead of 49 for NSS, so it even varies across runs.

michielbdejong avatar Jan 08 '20 15:01 michielbdejong

Huh, that's pretty strange, I haven't seen that. Do you see it without the EARL formatter too?

kjetilk avatar Jan 08 '20 16:01 kjetilk

no, then all of them always seemed to report 54 tests.

michielbdejong avatar Jan 08 '20 16:01 michielbdejong

OK, good, so something with the EARL reporter. I can look into that.

kjetilk avatar Jan 08 '20 22:01 kjetilk