alcotest

alcotest copied to clipboard

I want to use CircleCI's statistical features, which can tell me what tests are most flaky, etc. However, this requires a junit.xml formatted output, so it would be great if alcotest supported one.

It would be useful, when/if this is implemented, if we could still get console output while writing junit.xml output to a file.

You will probably be interested by https://github.com/Khady/ocaml-junit/blob/master/alcotest/junit_alcotest.mli

To use it you can try:

$ opam pin add junit_alcotest https://github.com/Khady/ocaml-junit.git

I haven't tried it but it seems that @Khady is already using alcotest+circleCI with some success.

I didn't take time to make a release on opam. But if there is some interest, I can do this during the week.

I don't know about the CircleCI's statistical features. But I can confirm that the repo contains a setup working with circleci and that the junit file generated is correctly interpreted by circleci.

Thanks @Khady, that would be great!

(PS: the "Insights" button should show you frequently failed tests and slowest tests for your repo).

@pbiggar it has been merged in opam https://github.com/ocaml/opam-repository/pull/11887

It would be useful, when/if this is implemented, if we could still get console output while writing junit.xml output to a file.

This is the current behavior of my implementation. See https://github.com/Khady/ocaml-junit/blob/master/alcotest/test/alcotest_report.ml for an example of usage. And https://circleci.com/gh/Khady/ocaml-junit/27 for the result in circleci.

Thanks @Khady! It's working perfectly :)

Re-opening this issue as there's been some interest from Tezos developers in having an integrated solution for Junit reporters in Alcotest. We have @Khady's excellent junit_alcotest to work from, so it should "just" be a case of having a mechanism to register this kind of reporter (or, failing that, a similar first-class feature).

CC @Sir4ur0n.