l3build icon indicating copy to clipboard operation
l3build copied to clipboard

Check fails, stdout and stderr

Open projetmbc opened this issue 1 year ago • 6 comments

When check fails, you don't use the stdout stream. It should be better to indicate the failure "Check failed with <<NAM-OF-THE-TEST-FILE>>" inside the stderr stream. Doing this allow extra tools to catch only checking that gives an error without having to parse all the stdout stream.

projetmbc avatar Oct 28 '24 11:10 projetmbc

This is not an error from the l3build point of view: it's information for the user.

josephwright avatar Oct 28 '24 12:10 josephwright

This makes sense. But I see more that as a workflow error than a computation one. This makes me write weird lines of code : ugly tests .

projetmbc avatar Oct 28 '24 12:10 projetmbc

I suspect you are doing something ... odd from our point of view: the entire idea with l3build is you run it yourself then take appropriate action if there is a failure. Could you outline your use case?

josephwright avatar Oct 28 '24 12:10 josephwright

Using parallelism, my script speeds up the search for regression when I modify my code. If I have an unexpected failure, I then work on each failing test on a case-by-case basis.

PS: in the end, once everything is fixed, I use l3build ctan, and therefore a classic l3build check.

projetmbc avatar Oct 28 '24 12:10 projetmbc

Using parallelism, my script speeds up the search for regression when I modify my code. If I have an unexpected failure, I then work on each failing test on a case-by-case basis.

We do that too :)

josephwright avatar Oct 28 '24 12:10 josephwright

@zauguin You are the expert in this area - what's your take?

josephwright avatar Oct 28 '24 12:10 josephwright