Contribution guide?

[ffMathy]

I would love to contribute to this library, but I don't know where to start, or how to "debug" my changes. I tried cloning the repo and running tests, but many of the integration tests fail, and I don't know why.

An explainer in the README.md would be good.

[MarcoRossignoli]

I would love to contribute to this library

Hi @ffMathy! At the moment we don't have a great documentation on how to debug stuff so I try to help here with some guidelines(It was on our roadmap):

We have like every other project a lot of unit testing for every part of the application, btw sometimes to understand why some code is not reported correctly we need to run a sort of "integration" test to debug where is the problem. To do that we have implemented some "helpers" to simulate a real coverage session. The nature of coverage is pretty invasive because we need to inject new IL instructions to allow the hit accounting. Due to that we have to copy/paste+rename test dll that we'll use for instrumentation of a specific part/pattern of C# code. We do that out of process using a package called Tmds.ExecFunction. In this way all instrumentation and accounting will run out of process without any concurrent file access issue.

Clearly we need also to "debug" this tests and to allow that we need to instrument and load "in process" because visual studio debugger need access to file+pdb stuff. In that case we added a simple extension method that will instrument copied dll inprocess during debugging session so we can check step-by-step all phases.

Take a look at this first explained example of a complete instrumentation:

https://github.com/coverlet-coverage/coverlet/blob/master/test/coverlet.core.tests/Coverage/CoverageTests.SelectionStatements.cs#L14

        public void SelectionStatements_If()
        {
            // We need to pass file name to remote process where it save instrumentation result
            // Similar to msbuild input/output
            string path = Path.GetTempFileName();
            try
            {
                // Lambda will run in a custom process to avoid issue with statics and file locking
                FunctionExecutor.Run(async (string[] pathSerialize) =>
                {
                    // Run load and call a delegate passing class as dynamic to simplify method call
                    CoveragePrepareResult coveragePrepareResult = await TestInstrumentationHelper.Run<SelectionStatements>(instance =>
                    {
                        // We call method to trigger coverage hits
                        instance.If(true);

                        // For now we have only async Run helper
                        return Task.CompletedTask;
                    }, persistPrepareResultToFile: pathSerialize[0]);

                    // we return 0 if we return something different assert fail
                    return 0;
                }, new string[] { path });

                // We retrive and load CoveragePrepareResult and run coverage calculation
                // Similar to msbuild coverage result task
                CoverageResult result = TestInstrumentationHelper.GetCoverageResult(path);

                // Generate html report to check
                // TestInstrumentationHelper.GenerateHtmlReport(result);

                // Asserts on doc/lines/branches
                result.Document("Instrumentation.SelectionStatements.cs")
                      // (line, hits)
                      .AssertLinesCovered((11, 1), (15, 0))
                      // (line,ordinal,hits)
                      .AssertBranchesCovered((9, 0, 1), (9, 1, 0));
            }
            finally
            {
                // Cleanup tmp file
                File.Delete(path);
            }

As you can see in this code we use the helper TestInstrumentationHelper to instrument particular dll(where the T class resides) and in a lamba we can call every method/props belonging to that class because instance is a dynamic object type. To avoid issue with concurrent access we're using FunctionExecutor.Run that will run lambda in new process. At the end of run we can get coverage result and do assertions on real coverage. Also you can generate web reports in automatic way thanks to ReportGenerator integration https://github.com/coverlet-coverage/coverlet/blob/master/test/coverlet.core.tests/Coverage/InstrumenterHelper.Assertions.cs#L23 (pass true to show).

The above code will run test in a new process and will work only running by console dotnet test you cannot run inside visual studio. To debug step-by-step we added some extension method to run all instrumentation/coverage inprocess with visual studio debugged attached

https://github.com/coverlet-coverage/coverlet/blob/master/test/coverlet.core.tests/Coverage/InstrumenterHelper.cs#L305

So if you simply change the line

  FunctionExecutor.Run(async (string[] pathSerialize) =>

with

 FunctionExecutor.RunInProcess(async (string[] pathSerialize) =>

and add a breakpoint here you can step into engine and debug all from instrumentation to accounting.

If you need further help feel free to DM me on twitter https://twitter.com/MarcoRossignoli (until a good guide will be available, could be a great good start PR 😄 write a debugging guide )

[github-actions[bot]]

This issue is stale because it has been open for 3 months with no activity.