coverlet
coverlet copied to clipboard
coverlet-coverage
Running coverlet.msbuild inside docker with --no-build
I am trying to run coverlet.msbuild with --no-build flag inside the docker container. And I am never able to generate the code coverage. I cannot run restore in the container. Hence, container does not have an access coverlet.msbuild nuget.
Logs: Command: dotnet test $startDir/$testPath/$testFile /p:CollectCoverage=true /p:CoverletOutputFormat=json /p:CoverletOutput=$outputFile /p:MergeWith=$mergeFile --verbosity diag --no-build
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Passed LogDataConverterMapsLogLevelsToLogSeverities [15 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (None,Unknown) [9 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Trace,Verbose) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Debug,Verbose) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Information,Information) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Warning,Warning) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Error,Error) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Critical,Error) [< 1 ms]
Passed LogDataConverterDetectsUnknownLogLevelMappings [< 1 ms]
Passed LogStoreProviderEnumDefinedTest [148 ms]
Passed LogStoreProviderEnumNotDefinedTest [< 1 ms]
Test Run Successful.
Total tests: 10
Passed: 10
Total time: 1.1038 Seconds
Done executing task "Microsoft.TestPlatform.Build.Tasks.VSTestTask". (TaskId:21)
21:17:54.857 1>Done building target "VSTest" in project "COM.iX.Logging.Provider.Test.csproj".: (TargetId:2)
21:17:54.858 1>Done Building Project "/github/workspace/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj" (VSTest target(s)).
Deferred Messages
21:17:54.861 0>
Detailed Build Summary
======================
(TaskId:0)
============================== Build Hierarchy (IDs represent configurations) =====================================================
Id : Exclusive Time Total Time Path (Targets)
----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
0 : 1.768s 1.768s /github/workspace/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj (VSTest) (TaskId:0)
============================== Node Utilization (IDs represent configurations) ====================================================
Timestamp: 1 Duration Cumulative
----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
637509214730885310: 0 1.772s 1.772s ................................... (TaskId:0)
-----------------------------------------------------------------------------------------------------------------------------------
Utilization: 100.0 Average Utilization: 100.0 (TaskId:0)
Project Evaluation Performance Summary:
290 ms /github/workspace/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj 1 calls
Project Performance Summary:
1428 ms /github/workspace/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj 1 calls
1428 ms VSTest 1 calls
Target Performance Summary:
9 ms ShowCallOfVSTestTaskWithParameter 1 calls
1423 ms VSTest 1 calls
Task Performance Summary:
8 ms Message 18 calls
20 ms CallTarget 1 calls
1386 ms Microsoft.TestPlatform.Build.Tasks.VSTestTask 1 calls
Build succeeded.
0 Warning(s)
0 Error(s)
Expected Behaviour
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Passed LogDataConverterMapsLogLevelsToLogSeverities [22 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (None,Unknown) [12 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Trace,Verbose) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Debug,Verbose) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Information,Information) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Warning,Warning) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Error,Error) [< 1 ms]
Passed LogDataConverterMapsLogLevelsToLogSeverities (Critical,Error) [< 1 ms]
Passed LogDataConverterDetectsUnknownLogLevelMappings [< 1 ms]
Passed LogStoreProviderEnumDefinedTest [196 ms]
Passed LogStoreProviderEnumNotDefinedTest [< 1 ms]
Test Run Successful.
Total tests: 10
Passed: 10
Total time: 1.1466 Seconds
Done executing task "Microsoft.TestPlatform.Build.Tasks.VSTestTask". (TaskId:22)
12:58:07.009 1>Done building target "VSTest" in project "COM.iX.Logging.Provider.Test.csproj".: (TargetId:5)
12:58:07.009 1>Target "GenerateCoverageResult: (TargetId:7)" in file "/home/admin1/.nuget/packages/coverlet.msbuild/3.0.3/build/coverlet.msbuild.targets" from project "/home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj" (target "GenerateCoverageResultAfterTest" depends on it):
Using "Coverlet.MSbuild.Tasks.CoverageResultTask" task from assembly "/home/admin1/.nuget/packages/coverlet.msbuild/3.0.3/build/coverlet.msbuild.tasks.dll".
Task "Coverlet.MSbuild.Tasks.CoverageResultTask" (TaskId:23)
Task Parameter:ThresholdStat=minimum (TaskId:23)
Task Parameter:ThresholdType=line,branch,method (TaskId:23)
Task Parameter:InstrumenterState=/tmp/tmp0vqEQf.tmp (TaskId:23)
Task Parameter:OutputFormat=json (TaskId:23)
Task Parameter:Threshold=0 (TaskId:23)
Task Parameter:Output=/home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/coverage.json (TaskId:23)
Calculating coverage result...
[coverlet] Hit file '/tmp/COM.iX.Logging.Provider_6c1ec4a4-7989-466c-b501-b83278e7731f' deleted (TaskId:23)
Generating report '/home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/coverage.json'
+-----------------------------+------+--------+--------+
| Module | Line | Branch | Method |
+-----------------------------+------+--------+--------+
| COM.iX.Logging.Provider | 80% | 62.5% | 66.66% |
+-----------------------------+------+--------+--------+
+---------+------+--------+--------+
| | Line | Branch | Method |
+---------+------+--------+--------+
| Total | 80% | 62.5% | 66.66% |
+---------+------+--------+--------+
| Average | 80% | 62.5% | 66.66% |
+---------+------+--------+--------+
Output Item(s):
CoverletReport=
/home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/coverage.json
Format=json (TaskId:23)
Done executing task "Coverlet.MSbuild.Tasks.CoverageResultTask". (TaskId:23)
12:58:07.261 1>Done building target "GenerateCoverageResult" in project "COM.iX.Logging.Provider.Test.csproj".: (TargetId:7)
12:58:07.261 1>Target "GenerateCoverageResultAfterTest: (TargetId:8)" in file "/home/admin1/.nuget/packages/coverlet.msbuild/3.0.3/build/coverlet.msbuild.targets" from project "/home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj" (entry point):
12:58:07.261 1>Done building target "GenerateCoverageResultAfterTest" in project "COM.iX.Logging.Provider.Test.csproj".: (TargetId:8)
12:58:07.262 1>Done Building Project "/home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj" (VSTest target(s)).
Deferred Messages
12:58:07.266 0>
Detailed Build Summary
======================
(TaskId:0)
============================== Build Hierarchy (IDs represent configurations) =====================================================
Id : Exclusive Time Total Time Path (Targets)
----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
0 : 2.658s 2.658s /home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj (VSTest) (TaskId:0)
============================== Node Utilization (IDs represent configurations) ====================================================
Timestamp: 1 Duration Cumulative
----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
637508230846046671: 0 2.661s 2.661s ..................................................... (TaskId:0)
-----------------------------------------------------------------------------------------------------------------------------------
Utilization: 100.0 Average Utilization: 100.0 (TaskId:0)
Project Evaluation Performance Summary:
350 ms /home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj 1 calls
Project Performance Summary:
2256 ms /home/admin1/actions-runner/ix-logging/_work/iX-logging/iX-logging/COM.iX.Logging.Provider.Test/COM.iX.Logging.Provider.Test.csproj 1 calls
2256 ms VSTest 1 calls
Target Performance Summary:
0 ms InstrumentModulesNoBuild 1 calls
0 ms GenerateCoverageResultAfterTest 1 calls
3 ms ShowCallOfVSTestTaskWithParameter 1 calls
12 ms GenerateProgramFile 1 calls
252 ms GenerateCoverageResult 1 calls
572 ms InstrumentModules 1 calls
1414 ms VSTest 1 calls
Task Performance Summary:
2 ms Message 18 calls
8 ms CallTarget 1 calls
251 ms Coverlet.MSbuild.Tasks.CoverageResultTask 1 calls
567 ms Coverlet.MSbuild.Tasks.InstrumentationTask 1 calls
1399 ms Microsoft.TestPlatform.Build.Tasks.VSTestTask 1 calls
Build succeeded.
0 Warning(s)
0 Error(s)
hi @pankti11 sorry for the delay, very busy days, outside container does it work?
Yes. but the only difference is the outside container I do a build. However, inside the container, I don't do-build. However, the bin folder (dlls are present from the build outside the container) is present in the container. So when I do dotnet test with --nobuild I expect it to generate the code coverage. Maybe I am missing something.
Do you copy the source code into the docker container in addition to the binaries? Coverlet skips projects with PDBs but no local source files as a heuristic to filter out libraries that shouldn't be part of the coverage. (See https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Instrumentation/Instrumenter.cs#L88)
It may also be an issue that the path name of the directory containing the code in the docker image is different to the path name outside it, which means that the source files cannot be found even if they are copied into the image. You can try to enable verbose logging (see https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/Troubleshooting.md) and see if it logs messages about not finding the source files.
So inside the container there is source file as well as the binaries. From the logs I could decipher that the Build Task which is responsible to generate the CodeCoverage is never runs. Hence, I concluded that when I do dotnet test --no-build, dotnet is not aware of the coverlet because I did not run dotnet build. When I do dotnet build inside container and then do dotnet test --no-build then the coverage is created by coverlet without any issue.
@petli We are running into similar issues. Actually we want to build and pack the binaries in one job and consume the nuget packages in multiple test jobs. Would be optimal if we wouldn't have to map the sources in the test job. Is that a requirement?
Additionally we tried as a workaround to map the sources along with the test run, but even that doesn't work. The path to the sources is identically, but of course we didn't do a build in the test job. Is there some output expected relative to the sources which we need to transfer from our build job to the test jobs? (maybe some files in the obj folder? )
Can I somehow configure to instrument binaries with embedded pdbs, even if no sources for them are available? We have the sources embdeded, along with the embedded pdb. Are embedded sources not supported?
btw. we are using coverlet.collector
This issue is stale because it has been open for 3 months with no activity.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}