Some tests are not run when the corresponding file opens other modules

[benjub]

Alcotest has a strange (buggy?) behavior when a test file opens a module of the library of the same project. In the example below, test/dune indicates that there are two tests, a and b corresponding to the two files test/a.ml and test/b.ml, but when running dune runtest, the test a is run twice and the test b is not run. This is apparently due to the fact that test/b.ml uses a function defined in lib/a.ml by using open Alcobug_lib [...] A.fa, and somehow this tricks Alcotest into running a a second time instead of running b. Note that when not using open but the full qualified name Alcobug_lib.A.fa, the bug disappears and both tests are run as expected.

Here is the full example:

$ tree -a
.
├── bin
│   ├── dune
│   └── main.ml
├── dune-project
├── lib
│   ├── a.ml
│   ├── b.ml
│   └── dune
└── test
    ├── a.ml
    ├── b.ml
    └── dune

3 directories, 9 files
$ cat dune-project; echo; for i in $(ls **/*); do echo "(*** $i ***)"; cat $i; echo; done
(lang dune 3.15)

(package
 (name alcobug_name)
 (depends
  ocaml
  dune
  (alcotest :with-test)))

(*** bin/dune ***)
(executable
 (public_name alcobug_public)
 (name main)
 (libraries alcobug_lib))

(*** bin/main.ml ***)
let () = print_endline "Hello"

(*** lib/a.ml ***)
let fa x = x

(*** lib/b.ml ***)
let fb x = x

(*** lib/dune ***)
(library
 (name alcobug_lib))

(*** test/a.ml ***)
let () =
  let _ = Alcobug_lib.A.fa 0 in
  Alcotest.run "aze" [ ("azer", [ Alcotest.test_case "azert" `Quick (fun () -> ()) ]) ]

(*** test/b.ml ***)
open Alcobug_lib
let () =
  let _ = A.fa 0 in
  Alcotest.run "baze"
    [ ("bazer", [ Alcotest.test_case "bazert" `Quick (fun () -> ()) ]) ]

(*** test/dune ***)
(tests
 (names a b)
 (libraries alcotest alcobug_lib))

$ dune runtest
Testing `aze'.                      
This run has ID `A8AJ7II1'.

  [OK]          azer          0   azert.

Full test results in `xxx/_build/default/test/_build/_tests/aze'.
Test Successful in 0.000s. 1 test run.
Testing `aze'.                     
This run has ID `1BQPG8KQ'.

  [OK]          azer          0   azert.

Full test results in `xxx/_build/default/test/_build/_tests/aze'.
Test Successful in 0.000s. 1 test run.

[kentookura]

I am experiencing this as well.

[kentookura]

I think this has to do with the fact that we are reusing the same name for the corresponding test module. I managed to work around this by prefixing all test modules with Test_.

[benjub]

I think this has to do with the fact that we are reusing the same name for the corresponding test module. I managed to work around this by prefixing all test modules with Test_.

Yes, that's the reason. Maybe the term "corresponding file" in the title was not clear, but that's what was meant: often, when I have a file in the library lib/a.ml, the corresponding test file is test/a.ml. I also had to rename some of my files like test/a_.ml, which is not very satisfying.