com-lihaoyi
mixing in multiple testFramework silently drops others.
I want to share one other finding while migration my sbt project to mill. The testFramework config in a TestModule is silently overridden when multiple TestModules are mixed in. Digging in the issues having a module per testframework is a design choice but the migration path (at least for me) was a bit confusing and required some additions steps in separating tests by framework into separate modules. If you forget to do it then just the tests for only one framework are executed and the other tests are ignored, no warnings.
The current mill init just picks one I think (could not find the related code). But perhaps when doing a mill init on an sbt project it would be helpful to report back when multiple testFramework are found, but not all migrated?
Having to split the test sources based on the testFramework is probably best solved by filtering source files by name or content for existing projects? Otherwise one would also have to change SBT setups during migration.. Perhaps it would also be possible to create multiple modules on the migration with a test{framework-name}-style naming and sources filtered by their content?
Interesting finding, thanks for reporting! I think until we improve the migrator, we should at least warn the user if we detect multiple test frameworks. But IIRC, sbt is auto-detecting test frameworks, so we might hit projects where it is hard or impossible to properly detect all frameworks without building and analyzing the build results.
Technically, it should be easy, to add additional test frameworks to an existing test module. It can be as lightweight as adding another object test2 extends TestModule.XyzFramework (e.g. TestModule.Scalatest) and override the testClasspath by pointing it to the primary test module's compile output.
Mill does have the ability to separate the TestModule from the JavaModule/ScalaModule, so in theory we could generate one object test extends ScalaModule for compilation and multiple testScalaTests extends TestModule, testSpecs2 extends TestModule, etc. modules for discovering the tests. This should give us the ability to separately run the tests discovered by each test framework without needing to manually fiddle with filtering source files and such
