Code coverage report ?

[noomerzx]

Any way to see code coverage percentage for tested directory ?

[RuiffCardoso]

This with included integration when editing file would be great.

[sayonsur]

I would like to help although i am new to VScode extensions and typescript. Looking at jacoco, can we do this ?

• code coverage can be enabled from json config.
• prerequisite can be jacoco dependency.
• whenever a UT runs we trigger jacoco command to generate report.
• the output is xml , it can be shown in UI.

[jdneo]

Hi @sayonsur,

Thank you for your enthusiasm for this feature. Yes, all kind of contribution is welcome! Let me spend some time to do some investigation for this issue. I'll get back to you when I get some ideas.

Thanks.

[jdneo]

Hi @sayonsur, I have some questions:

prerequisite can be jacoco dependency.

Do you mean that we will embedded a jacoco jar in the extension?

the output is xml , it can be shown in UI.

Does that mean that we will parse the xml and show it in the VS Code webview?

[sayonsur]

Hi @jdneo,

I believe it can be achieved in both ways. You can embed the Jacobo jar and programmatically generate code coverage whenever a test is run. Or we can rely on the jar being present in system and have a configurable path to use it.

It generates a xml . So it can be easily parsed for a web view report. In fact there are many vscode plugins which already parses xml and show report or color code the lines also directly.

[jdneo]

The embedded solution looks better since it could avoid breaking changes of the jacoco. But one thing we need to take care is about its license.

BTW, would you like to contribute for this feature?

[sayonsur]

Hi @jdneo, I don't have expertise on TS. Although I would love to give it a shot. Is there a timeline ? I might need more time , to understand both the code base and embed jacoco to achieve this.

[jdneo]

Hi @sayonsur, there is no timeline. Just give it a shot whenever you want. If you meet any problem, just feel free to let me know.

[sayonsur]

Sure thing. I will give it a shot and reach out to you in case i need any guidance on the Test project.

[sayonsur]

@jdneo,

Is there any documentation for the Java test runner ? I don't see any associated test project. What parameters to pass to debug and check the flow. Could you please help me out.

Thanks, Sayon

[jdneo]

Hi @sayonsur,

Please refer to this page: https://github.com/microsoft/vscode-java-test/blob/master/CONTRIBUTING.md

If you find anything missing, please let me know.

Thanks.

[Yashwin12]

Hi @jdneo, do we have any updates on the line-by-line code-coverage feature? Thanks in advance.

[jdneo]

Hey @Yashwin12,

Sorry not yet so far. But feel free to contribute if you want. 😃

[jdneo]

Just FYI, VS Code team is now working on a test API proposal, which might also include the test coverage support. Let's wait and see what we can do there: https://github.com/microsoft/vscode/issues/107467

[jdneo]

For everyone who wants to have this feature, could you please answer the following question to help us better understand the use case?

• What's the coverage tool you are using? (JaCoCo, Clover, Cobertura, JCov, Code Cover, PIT, etc...)
• How do you use the coverage tool? (Maven/Gradle/Eclipse/IntelliJ/CI integration, etc...)

[JonathanTrEs]

For everyone who wants to have this feature, could you please answer the following question to help us better understand the use case?

• What's the coverage tool you are using? (JaCoCo, Clover, Cobertura, JCov, Code Cover, PIT, etc...)
• How do you use the coverage tool? (Maven/Gradle/Eclipse/IntelliJ/CI integration, etc...)

• I use JaCoCo
• With the Eclipse/IntelliJ IDE

[RussellSanders1]

For everyone who wants to have this feature, could you please answer the following question to help us better understand the use case?

• What's the coverage tool you are using? (JaCoCo, Clover, Cobertura, JCov, Code Cover, PIT, etc...)
• How do you use the coverage tool? (Maven/Gradle/Eclipse/IntelliJ/CI integration, etc...)

• using Intellij IDEA code coverage runner
• from Intellij IDEA (naturally lol)

[baincd]

For everyone who wants to have this feature, could you please answer the following question to help us better understand the use case?

* What's the coverage tool you are using? (JaCoCo, Clover, Cobertura, JCov, Code Cover, PIT, etc...)

* How do you use the coverage tool? (Maven/Gradle/Eclipse/IntelliJ/CI integration, etc...)

• Jacoco
• Eclipse and Maven

[Tazaf]

For everyone who wants to have this feature, could you please answer the following question to help us better understand the use case?

• What's the coverage tool you are using? (JaCoCo, Clover, Cobertura, JCov, Code Cover, PIT, etc...)
• How do you use the coverage tool? (Maven/Gradle/Eclipse/IntelliJ/CI integration, etc...)

• JaCoCo
• Maven (mvn test)

[vinerich]

For everyone who wants to have this feature, could you please answer the following question to help us better understand the use case?

* What's the coverage tool you are using? (JaCoCo, Clover, Cobertura, JCov, Code Cover, PIT, etc...)

* How do you use the coverage tool? (Maven/Gradle/Eclipse/IntelliJ/CI integration, etc...)

• JaCoCo
• Gradle/Eclipse/IntelliJ (all generate the coverage through a gradle call and then parse it accordingly)

[baincd]

It is possible to get code coverage reports from tests run in VS Code, however it does require some manual configuration, manual steps, and additional extensions:

1. Create a Java Test Configuration that configures the Jacoco agent to generate the execution data file (jacoco.exec)

"java.test.config": [
    {
        "name": "jacoco",
        "vmargs": ["-javaagent:/home/chris/.vscode/org.jacoco.agent-0.8.8-runtime.jar=destfile=target/jacoco.exec,append=false"]
    }
],

2. Use either the Jacoco CLI report command or the Maven org.jacoco:jacoco-maven-plugin:report goal generate an XML report (jacoco.xml) and HTML report from jacoco.exec. (This script can be used to do this "automatically" by creating the reports whenever jacoco.exec changes)

3. The overall code coverage can be viewed via the HTML report (by default, the Jacoco Maven plugin saves this to target/site/jacoco/index.html). This can be viewed in a web browser, or in VS Code using an extension to preview HTML files (example screenshot uses the Live Preview extension)

4. The VS Code Coverage Gutters Extension can be installed to display code coverage from the XML report on each Java file within the editor (example screenshot has "Coverage-gutters: Show Line Coverage" setting enabled

The overall flow looks something like this:

flowchart
    s1["1. Run Tests in VS Code<br/>(vmargs configured for jacoco agent)"] --> jacoco.exec[/jacoco.exec/]
    jacoco.exec --> s2[2. Jacoco CLI report command OR <br/>mvn org.jacoco:jacoco-maven-plugin:report]
    s2 --> index.html[/jacoco/index.html/]
    s2 --> jacoco.xml[/jacoco.xml/]
    index.html --> s3[3. View code coverage in browser <br/>or with a VS Code extension to preview HTML]
    jacoco.xml --> s4[4. Code coverage displayed in Java editor<br/>with Coverage Gutters extension]

Suggestions to improve the Code Coverage UX in VS Code

To improve the code coverage UX in VS code (and make it similar to Eclipse and IntelliJ), I offer the following suggestions

• Add a postLaunchTask option for Java Test Configurations so step 2 (generation of code coverage reports) could be configured to automatically happen after tests are run
  • It would be a big win to be able to have the code coverage reports generated automatically when tests are run
• Natively display code coverage reports (with links that directly open Java files)
  • There are workarounds (browser, extensions to preview HTML). But having this natively in VS Code would be nice
• Automatic setup of steps 1 and 2
  • Assuming this is the most complex, I think it'd be better to focus on the other options first. And good documentation on how to setup VS Code to generate code coverage reports may be more feasible then automating the setup

[jdneo]

@baincd Thank you for the detailed suggestions.

The current state is that we are waiting the VS Code to publish the test coverage API: https://github.com/microsoft/vscode/issues/123713

About adding postLaunchTask, I would like to see more use cases, otherwise, once the API get released, this field would become useless.

But anyway, the steps you provided would be a great workaround for the current moment.

[baincd]

I'm excited to hear that there is a test coverage API (https://github.com/microsoft/vscode/issues/123713) in the works. Knowing this is on deck, I think manually generating the Jacoco report is fine for now.

Regarding a postLaunchTask option, after experimenting a bit more I realized that there is not a way to pass the Java project being tested to the task. This means it could not be used to generate the Jacoco report for Java projects not in the root folder, (including multi-module projects). Since it would only work for some projects, I don't think it is justified add for now.

[baincd]

I put together this script that runs continuously, creating the Jacoco reports (XML, HTML) when jacoco.exec changes. This can be used to do step 2 above automatically.

[viswatejan]

I usually use maven for my Java projects and so I have created a maven favorite entry in the Settings.JSON file for the jacoco code coverage report.

"maven.terminal.favorites": [
    {
      "alias": "Java Code Coverage",
      "command": "clean -X org.jacoco:jacoco-maven-plugin:prepare-agent verify org.jacoco:jacoco-maven-plugin:report"
    }
  ],

In addition, I have installed coverage-gutters extension.

So, when ever I want code coverage, I will just run my Maven Favorite to run the Java Code Coverage like shown below:

And I use Live Preview extension to preview the index.html file in the target/site/jacoco directory.

I switch multiple java workspaces and so I don't want to create the same task in all my projects. This way my maven favorite will be available for all my workspaces.