Support multi-modules project

[flniu]

For example, my project (a git repo) has several sub folders, each contains a go.mod. The project root folder may or may not contain a go.mod.

Is that possible to run gotestsum to execute all tests in each modules?

[dnephin]

gotestsum runs go test, so it has the same limitations. You can run it on each module individually, but there's no way to run it on all of the modules from a single command.

Are you looking to use gotestsum in CI, or to run tests for local development?

[flniu]

I'd like to use gotestsum for both local development and CI. It make sense to run a single command that executes all tests and prints a test coverage summary, like run pytest with plugin pytest-cov. Do you consider to support this feature or similar plugin mechanism?

For example: add -R option which means run tests recursively.

Total test coverage need further calculation, so there also should be an option for it.

Plugin mechanism may be too complex and unnecessary since Go and Python are different.

[dnephin]

I think this is an interesting idea! It might be good to first see how easy it is to do this with a bash script. That should give us some idea of what logic is necessary to find all the modules. Does this do what you want?

for mod in $(find . -name go.mod | xargs -n 1 dirname); do (cd "$mod"; go test ./...); done

I think, at least initially, it would be good to make this a separate sub-command (similar to gotestsum tool slowest). I think the advantage to a sub-command is that if go test ever gets some way to run tests across multiple modules we won't be in a situation where gotestsum behaves differently than go test.

Maybe gotestsum testall or gotestsum allmodules. As long as the sub-command accepts all the same command line flags, I hope that should work for your use case.

[flniu]

Agree. Keep gotestsum behaves consistent with go test will be better. That's a good point.

I have another concern. We may want to ensure the total test coverage of a repo. So the CI will check total coverage and fail if it's lower than a specified value, like pytest-cov can do.

However, if a sub module contains no _test.go files, go test will generate an empty coverage profile. But what we want is a profile with 0 coverage.

So the bash script may look like this:

TOTAL_COVERAGE_FILE=$(pwd)/total_coverage.out
for mod in $(find . -name go.mod | xargs -n 1 dirname); do
    cd "$mod"
    touch dummy_test.go
    go test -cover ./... -coverpkg=./... -coverprofile=coverage.out
    tail -n +1 coverage.out >>"$TOTAL_COVERAGE_FILE"
    rm dummy_test.go
done
calc_total_coverage "$TOTAL_COVERAGE_FILE"

Sub-command allmodules and option --total-coverage should be orthogonal, of cause. People may use only one of them.

[dnephin]

Not exactly the same request as this issue, but I thought I would mention that #276 adds a new --watch-chdir flag which makes it possible to use --watch with multi-module projects. Without this flag go test refuses to run tests for any package outside the main Go module. With the flag, gotestsum changes to the directory where a file is modified, so go test . works and --watch is able to run the tests for any directory, even if it's outside of the main Go module.