boyter
refactor: convert Bash integration tests to Go tests
How It Works
Golang compiles all test cases and their related code into a single binary file (usually named as <package-name>.test). The binary will call func TestMain(m *testing.M) before any test case running. So we can call our main function in the TestMain if special flag was set. This will include all SCC code in the generated binary file, transforming it into an SCC command-line program with test cases. Calling this binary file is almost equivalent to directly invoking SCC. Therefore, we can use exec.Command in our integration test cases to invoke this binary file and capture all its output for our testing.
The TestMain looks like this:
func TestMain(m *testing.M) {
idx := slices.Index(os.Args, sccTestFlag)
// search for the special flag, if it was set, that means we call the binary from `exec.Command`
// so we call main.main and skip all test cases
if idx != -1 {
os.Args = slices.Delete(os.Args, idx, idx+1)
main()
return
}
// if the special flag was not set, that means this binary is called by `go test`
// so we do not run main function but run all the test cases
os.Exit(m.Run())
}
We can use a helper function to run SCC and add special flags:
func runSCC(args ...string) (string, error) {
args = slices.Insert(args, 0, sccTestFlag) // set the special flag
cmd := exec.Command(sccBinPath, args...)
res, err := cmd.CombinedOutput()
return string(res), err
}
This makes the test code much simple than what #633 does. Similar strategies are widely used in projects like Kubernetes, Docker, the Go compiler, Prometheus and so on.
Migrating all integration tests is a big project, so I will complete it in several steps:
- First, I will migrate all regression tests.
- Second, I will migrate all tests that create temporary files. Go test can create temporary files more safely and will clean them up automatically.
- Then, I'd like to migrate languages test.
- Finally, migrate other test cases to Go test.
This PR is the first two step.
Disadvantages:
The runtime of go test -race has significantly increased. For comparison, the runtime on my laptop before migration was 10s, while after migration it is 40s. Although t.Parallel() is used to run tests in parallel, the total runtime is expected to exceed one minute once all tests are migrated.
Yeah neat. This looks like a good way to start breaking away from that test script into pure Go. Somewhat related to what I started over https://github.com/boyter/scc/pull/633 although this does the main thing I had been putting off.
Yeah neat. This looks like a good way to start breaking away from that test script into pure Go. Somewhat related to what I started over #633 although this does the main thing I had been putting off.
This PR is ready for review. You can take a look at the converted test cases to check if I missed something.