smalltalkCI icon indicating copy to clipboard operation
smalltalkCI copied to clipboard

Coveralls testing is skipping class side & extension methods

Open peteruhnak opened this issue 6 years ago • 2 comments

Apparently class-side and extension methods are both being skipped during coverage collection.

Here's a sample coveralls_results.json

here's the build https://travis-ci.org/peteruhnak/pharo-travis-pipeline/jobs/360667859

Testing itself works, fine, but not collection.

E.g. PTItem class>>testedClassSideMethod is correctly tested (Line 735 #testClassSideMethod)

but there's no mention of testedClassSideMethod in the JSON (instead there's just null)

Dtto for extension methods https://github.com/peteruhnak/pharo-travis-pipeline/tree/master/repository/PharoTravisPipelineExtensions.package

peteruhnak avatar Mar 31 '18 17:03 peteruhnak

I could reproduce this issue with multiple Squeak repositories for each of which the Squeak Test Runner displays the coverage results correctly:

image image

The following kinds of methods are not written correctly into the JSON file:

  • class-side methods
  • extension methods
  • trait methods (neither registered in the trait behavior nor in its users)
  • special selectors such as #~~

The defect method is SCICodeCoverage class >> #filePathFor:in:, which uses a probably too simple heuristic for creating the FileTree paths for all methods:

https://github.com/hpi-swa/smalltalkCI/blob/b30166bda3aabf8c74266f6fbaf85b7592891cdd/repository/SmalltalkCI-Coverage-Core.package/SCICodeCoverage.class/class/filePathFor.in..st#L1-L16

Fixing class-side methods is easy, just exchange class name with class theNonMetaClassName.
For extension methods, #packageNameForClass: needs to be replaced with something that takes the extension category of the method into consideration.
For traits, things get more complicated because afaik all compiled methods from a trait are copied into the method dictionary of all of its users.
For special selectors, the official escape sequences can be found in MCFileTreeStCypressWriter class >> #initializeSpecials. May I ask why SCICodeCoverage does not reuse the methods from MCFileTreeStCypressWriter (e.g. #fileNameForSelector:), is this a dependency issue? :-)

LinqLover avatar Dec 31 '21 02:12 LinqLover

If you could provide fixes, that would be great. Contributions are welcome! If there's potential for reuse, feel free to give it a try. In theory, smalltalkCI should be able to exist without Metacello. Also, we try to keep the number of external dependencies as low as possible to avoid breakages. Maybe copy the relevant methods over and add a comment to document where they are from?

fniephaus avatar Dec 31 '21 12:12 fniephaus