Create a code coverage report while running Run-TestsInNavContainer

[Sven-Niehus]

Hi,

it would be helpful to have the ability to create a code coverage report while running the tests with Run-TestsInNavContainer. The easiest way would be to add a option to the Run-TestsInNavContainer commandlet (-codeCoverage for example) that automatically generates the code coverage report alongside the XUnit test result file.

Having the code coverage for our tests would be really helpful for our build pipelines, because it would enable us to check for the code coverage in pull requests and thus automatically refuse bad pull requests that do not meet our quality expectations and still need to be worked on.

Azure DevOps supports code coverage files in Cobertura and JaCoCo format out of the box (Source [1] [2]), so those file formats would be preferred.

[Sven-Niehus]

An example for a Cobertura coverage file can be found here

[freddydk]

I think that is something the moderndev team is working on - and as soon as that exists, I will integrate it into the run-tests function. @atoader - do you have any info about when we will have something like this?

[XVII]

Something we're keeping an eye out for too. Will help us ensure we comply with Microsoft AppSource code coverage requirements.

[atoader]

We are working on this, but I cannot provide an exact ETA. We should have it in place for Wave 2 2020 at the latest

[pmatsconsulting]

Hi, Can we still expect have it in place for Wave 2 2020 at the latest?

[atoader]

Hi! The most likely timeline is CU2 of Wave 2 2020.

[ninten]

Hi! The most likely timeline is CU2 of Wave 2 2020.

Hi! Wave 2 2020 is already out, but what does CU2 mean?

[freddydk]

2020 wave 2 is 17.0 17.2 is what is referred to as cu2 here - december / january likely

[TheNCuber]

Now that BC 17.2 is out, are there any news concerning this functionality in navcontainerhelper?

[freddydk]

I will check up with the team to see if there is something we can use

[steinarb-ls]

Any updates on this?

[TheNCuber]

Any news on this @freddydk ? Code coverage in AL is available now through page 9990 I think, so I guess it should be possible to add it to BCContainerHelper. 🤔

[freddydk]

I haven't looked, but I guess it is just a matter of getting the time then... Hopefully later in March

[elieassi]

Hello, any updates on this please?

[danielgorski-bynd365]

Hello again, any updates regarding this feature?

[Spycoclown]

@freddydk how is the status?

[TheNCuber]

As I could not wait any longer, I basically wrote a PowerShell script (~600 lines) that parses the output of xmlports 9990 and 9991 (code coverage summary & code coverage detailed) as inspired by test-runner-service by @jimmymcp. It then generates an html report with an overview of the code coverage of all objects (screenshot 1) as well as a detailed html file for every object with line-by-line color coding and hit count, link to all procedures/triggers... (screenshot 2). The html reports are based on templates from the report generator that unfortunately doesn't support AL. The script also generates a very simple cobertura xml file with the global coverage percentage for Azure DevOps. I use the script in our Azure DevOps pipeline to publish the reports and the xml file for every build directly to Azure DevOps so the global percentage as well as the html reports are visible there. At the moment the script is just a quick and dirty hack, but if I find time, I will try to open source a clean version of it, so we can maybe start from there.

Screenshot 1 (summary): Screenshot 2 (detailed):

[Spycoclown]

@TheNCuber is it possible to get beta access? it looks great and I am looking for something like this since DevOps Pipeline can be used :)

[freddydk]

Very cool. It was my understanding that the code coverage didn't support the AL Test Runner (only the C/AL Test Runner, which will be removed any time). I will ask Nikola for his comments on this?

[christianbraeunlich]

Hi, have now also lost my way here ;-) Any news on this? I would assume that it will be provided in the PowerShell module in the foreseeable future. Would be quite nice if we don't have to reinvent the wheel. Thanks in advance!

[Djordje-Arsic]

@TheNCuber Looking exactly for what you have created. Any chance you can share your PS script?

Anyhow, it would be nice if somebody could at least provide some news on this topic, I'm not seeing any new info anywhere... @freddydk @atoader

[jonaswre]

@freddydk hinted it would be available soon. Since the ~~ALC~~ Test Runner now does create the coverage but it still needs to be extracted from the container

[freddydk]

not the ALC, the AL Test Runner does. Yes - assuming that we can add this in november or december

[bszentesi]

Hi @freddydk , You didn't mention the year... was it on purpose? :)

[freddydk]

:-( - sorry - will see what I can do

[hvandenborn007]

Any news yet? we are waiting with expectation :-D we really like this and are waiting for over some years now...

[MaxFalcone]

Hi, I think it might be related to the topic. Is ProduceCodeCoverageMap parameter of PsTestFunctions.ps1 not used intentionally in Run-testsInBcContainer? Or am I missing some plot?

[MaxFalcone]

VSCode now natively supports code coverage decarations. It even has an option to load coverage data from files (small extension vould be required though, if I got it correctly). Ability to export code coverage afterr running tests becomes more valuable.

[Antebur]

Are there any news on this one ? i would like to publish the Code Coverage Report to Devops Pipeline Tab. That would be a nice feature