coverlet
coverlet copied to clipboard
coverlet-coverage
Unable to specify output directory when using VSTest integration?
In the MSBuild integration of Coverlet, you can specify the output directory (and/or file) using -p:CoverletOutput='./result/'.
Am I correct that this feature is not available in the VSTest integration? And if so, are there any technical reasons why this can't be implemented into the VSTest integration?
@MarcoRossignoli You can specify the results directory by using dotnet test --results-directory:"e:\resDir". The coverage file will be by default generated in the results directory specified. This can be even given in the runsettings.
https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test?tabs=netcore21
https://docs.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019
@MarcoRossignoli that doesn't quite replicate the behavior of the MSBuild integration. The value I found in the original behavior is that you can specify the directory and that is the root directory where the coverage report will be output (this way it would just overwrite the same file, instead of creating a bunch of coverage reports)
In this case, using ... --results-directory:"./Path/" it still creates a folder in the specified directory with a GUID, meaning you won't be able to overwrite coverage each time it is run, instead you get a new file each time.
@vagisha-nidhi I think that GUID it's by design and it's not removable...right?
@Stefanie899 I'm going to close after Vagisha's confirmation, this is non something that coverlet can solve, maybe a try on vstest repo.
Could you explain why that's not "removable?" You have a constructor that takes that directory as a parameter, and in your default ctor, you use Guid.NewGuid().ToString().
While I understand you can't perfectly replicate the behavior of the .MSBuild, why not check to see if a property has been set in the .runsettings for the outputSubDirectory, and let it be something static. Not having any idea what the output directory is going to be makes finding it a little difficult as part of a build process.
Even if it's just the ability to make it "/coverage" so that I know exactly where the .xml file will be would fix my issue.
@DaleyKD I think that the right place to ask for this is vstest repo https://github.com/microsoft/vstest/issues
@MarcoRossignoli : Please forgive me, I'm not trying to be obtuse.
Why can't you just use the other constructor for AttachmentManager here in CoverletCoverageCollector and pass in a setting (you'll have to create it) that you get from RunSettings?
My proposed solution has nothing (that I can see) to do with VSTest.
EDIT: Using this ctor:
public AttachmentManager(DataCollectionSink dataSink, DataCollectionContext dataCollectionContext, TestPlatformLogger logger, TestPlatformEqtTrace eqtTrace, string reportDirectoryName, IFileHelper fileHelper, IDirectoryHelper directoryHelper, ICountDownEvent countDownEvent)
and pass in reportDirectoryName.
That guid is added by vstest platform, that class is only a wrapper that send file to vstest plat, is vstest platform that write file and log to console file path. We have to follow vstest collector "protocol" otherwise future update on vstest plat could conflict with our custom behavior. The same for "console" output, we cannot write to console with simple Console.Writeline, vstest plat orchestrate all workflow here.
and pass in reportDirectoryName.
That class doesn't persist file...send only report stream to vstest plat through sink. This is the specs to implement a correct collector. This collector code was written as by MS vstest team not by us.
Why not simply update CI and move file where you want at the end of coverage test?
@DaleyKD this is the place inside vstest where that guid is added to final path
https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.Common/DataCollection/DataCollectionAttachmentManager.cs#L101-L125
That guid is always added at the end SessionOutputDirectory .
Coverlet CI is using properties in Directory.Build.props for test results output directory.
https://github.com/coverlet-coverage/coverlet/blob/cf40840873588e3b596eebc25362732728b3d651/Directory.Build.props#L36C1-L44C19
