Coverlet, Multiple TargetFramework, SourceLink, Deterministic Build and Pain

[TommasoBelluzzo]

Hi all. I'm ready to publish a new release of my FastHashes library.

I'm running the following configurations for my solution:

• .NET Core 3.1 for my test project, requiring .NET Standard 2.1 library.
• .NET Core 2.1 for my test project, requiring .NET Standard 2.0 library.
• Deterministic build as per DeterministicBuild.targets.

FastHashes.csproj:

<TargetFrameworks>netstandard2.0;netstandard2.1</TargetFrameworks>`

FastHashes.Tests.csproj:

<ItemGroup>
    <ProjectReference Include="..\FastHashes\FastHashes.csproj" Condition="'$(TargetFramework)' == 'netcoreapp2.1'">
      <SetTargetFramework>TargetFramework=netstandard2.0</SetTargetFramework>
    </ProjectReference>
    <ProjectReference Include="..\FastHashes\FastHashes.csproj" Condition="'$(TargetFramework)' == 'netcoreapp3.1'">
      <SetTargetFramework>TargetFramework=netstandard2.1</SetTargetFramework>
    </ProjectReference>
</ItemGroup>

I'm running the following GitHub Actions for tests and coverage collection:

name: "Continuous Integration"

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  continuous_integration:
    name: Continuous Integration
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ ubuntu-latest, macos-latest, windows-latest ]
    steps:
    - name: Repository Checkout
      uses: actions/checkout@v2
    - name: .NET Setup
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: |
          2.1.x
          3.1.x
          5.0.x
    - name: Restore
      working-directory: Solution/
      run: dotnet restore --force-evaluate
    - name: Build
      working-directory: Solution/
      run: dotnet build FastHashes.sln --verbosity normal --configuration Release --no-restore --no-incremental
    - name: Test
      working-directory: Solution/
      run: dotnet test FastHashes.sln --verbosity normal --configuration Release --no-restore --no-build /p:CollectCoverage=true /p:CoverletOutputFormat="lcov" /p:CoverletOutput="../Coverage/" /p:MergeWith="../Coverage/coverage.info"
    - name: Coverage
      uses: coverallsapp/github-action@master
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        path-to-lcov: Solution/Coverage/coverage.info

On my local machine, when I execute the test project, tests are executed and coverage reports are properly produced. This is probably due to the fact that Deterministic Build and SourceLink are disabled. The only problem is that they are not merged into a single file as per MergeWith directive: local_console.txt

On the GitHub Actions machine, tests are executed but coverage reports are empty. Of course, no merge file is produced. I would be pleased to attach the log file but "something went really wrong, and we can't process that file". So let's just insert a workflow run reference: https://github.com/TommasoBelluzzo/FastHashes/actions/runs/2876788194

2022-08-17T16:17:25.7698646Z ./Solution/Coverage:
2022-08-17T16:17:25.7698891Z total 0
2022-08-17T16:17:25.7699367Z -rw-r--r-- 1 runneradmin 197121 0 Aug 17 16:17 coverage.netcoreapp3.1.info
2022-08-17T16:17:25.7699986Z -rw-r--r-- 1 runneradmin 197121 0 Aug 17 16:16 coverage.netcoreapp2.1.info

I think this has something to do with previous issues I already checked. I tried to follow the guidelines for properly setting up the project but I really can't find a way to make my CI work.

[TommasoBelluzzo]

@MarcoRossignoli gentle ping, my release is stuck if I can't solve this problem

[MarcoRossignoli]

Merge is a bit tricky you can follow the guide here https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/Examples/MSBuild/MergeWith/HowTo.md Also it's possible that your empty report is due to a known issue with msbuild/dotnet tool driver https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#vstest-stops-process-execution-early in that case we don't have a lot of solution you should use the collectors integration https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/VSTestIntegration.md and let your report generator to merge files.

My advice is to mote to data collector integration https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/VSTestIntegration.md and use a tool like report generator(https://github.com/danielpalme/ReportGenerator#usage--command-line-parameters) to merge the files at the end before the upload(at the moment the merge is not supported by data collector integration).

[TommasoBelluzzo]

@MarcoRossignoli, thanks for your prompt reply! I switched to collectors, but the issue is still present and after messing a bit with my project files I can definately say it's related to DeterministicSourcePaths. When DeterministicSourcePaths is turned off, everything works. On both my local PC and GitHub Actions servers, if it's set to true, the tested library is not being instrumented and is completely ignored.

Workflow run with deterministic source paths disabled (coverage reports are filled): https://github.com/TommasoBelluzzo/FastHashes/actions/runs/2941021454

Workflow run with ddeterministic source paths enabled (coverage reports are empty): https://github.com/TommasoBelluzzo/FastHashes/actions/runs/2941039391

Might you kindly check my solution to see if I'm doing something wrong? Maybe there are obvious errors I can't spot.

Thanks!

[github-actions[bot]]

This issue is stale because it has been open for 3 months with no activity.