Exclude tests from coverage detection

[LinqLover]

In my solution, I have different kinds of tests, including unit tests and acceptance tests. For the coverage detection, I would like to take the unit tests into consideration only. That is, I would expect to change the .smalltalk.ston config like this:

SmalltalkCISpec {
	...
	#testing : {
		...
		#coverage : {
			#packages : [ 'MyProject-Core' ],
			#tests : {
				#exclude : {
					#classes : [ #MyProjectAcceptanceTest ],
					#packages : [ 'MyProject-Tests-Acceptance' ]
				}
			}
		}
	}
}

In this example, coverage.tests would define a selection of tests that should still be run but not be used for coverage detection.

Is there any possible way to configure this at the moment or would this indeed be a new feature request?

A current workaround would be to run smalltalkci twice with two different configurations, but I think this is really suboptimal, mainly in terms of efficiency.

[fniephaus]

If you split your code base into different packages, then you should be able to only run coverage against specific packages. There are also some magic literals (e.g. #ignoreForCoverage) and methods (e.g. #classNamesNotUnderTest) you can use to control coverage (see here).

Note that smalltalkCI aims to be close to the coverage feature provided by the test runner. If you can determine code coverage with the test runner in the way you want, you should be able to do the same with smalltalkCI.

[LinqLover]

My usage scenario would be something like this:

Package MyProject-Core:
  MyClass
    #foo
    #bar
Package MyProject-Tests:
  MyUnitTest
  MyAcceptanceTest

Both tests access MyClass, but only MyAcceptanceTest accesses MyClass >> #bar. In this example, I would expect a code coverage of 50%.

In the Squeak Test Runner, I would need to make two runs of which in the first run, I would run tests with coverage for MyUnitTest and in the second run, I would run tests without coverage for MyAcceptanceTest. So this is not yet supported in smalltalkCI? :-)

[fniephaus]

No, this is not really supported. However, you could run smalltalkCI a second time in a pre/postTesting script. The API is quite simple:

https://github.com/hpi-swa/smalltalkCI/blob/ba92e0ca58e5c189289ed19ae49a8b6bd6544c83/squeak/run.sh#L341