l3build
l3build copied to clipboard
l3build check reports misleading if no tests defined
I'm not sure if this is a bug. Maybe it is a request for clearer reporting.
If I run l3build check
with a test file which fails to define any tests and an expectation file which also fails to define any, then l3build
reports as follows:
Running checks on
mini-test (1/1)
All checks passed
It is true that all checks passed. It is equally true that all checks failed. So the final claim is technically correct, albeit misleading. The (1/1)
is also confusing, I think, at least if you've seen this kind of output from make
, where similar tallies typically indicate the number of tests rather than the number of files.
testfiles/mini-test.lvt
:
\input regression-test.tex\relax
\typeout{hylo}
testfiles/mini-test.lve
:
\input regression-test.tex\relax
\typeout{hello}
Could you clarify the output somehow? Some reminder that the numbers mean files rather than tests, maybe, and/or No checks failed
rather than All checks passed
. Insofar as I understand the code, isn't the former rather than the latter what's actually being reported here?
Failing that, you could remind people that l3build
allows an empty domain, but while this is entirely logical, it is not very intuitive. There is something quite odd about the fact it makes claims such as 'All flying pigs speak French' and 'Every unicorn loves a Womble' true.