pytype icon indicating copy to clipboard operation
pytype copied to clipboard

Add junit xml output format

Open UfoRobot opened this issue 6 years ago • 2 comments

This could allow to easily integrate pytype checks in many continuous integration environments.

I found a similar issue ( https://github.com/google/pytype/issues/92) but it was resolved with CSV formatting

UfoRobot avatar Jul 22 '19 11:07 UfoRobot

How is the junit xml suppsed to look like ?

The closest thing I could find was mypy's junit output eg :

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="1" name="mypy" skips="0" tests="1" time="0.937">
  <testcase classname="mypy" file="mypy" line="1" name="mypy-py3_8-linux" time="0.937">
    <failure message="mypy produced messages">
vulnerabilities/tests/conftest.py:26: error: Skipping analyzing 'pytest': found module but no type hints or library stubs
vulnerabilities/tests/test_import_cmd.py:25: error: Skipping analyzing 'django.core.management': found module but no type hints or library stubs
</failure>
  </testcase>
</testsuite>

So inferring from that, I guess pytype's junit ouput would roughly look like

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="1" name="pytype" skips="0" tests="1" time="0.937">
  <testcase classname="pytype" file="pytype" line="1" name="pytype-py3_8-linux" time="0.937">
    <failure message="pytype produced messages">
File "/home/shivam/coding/opensource/vulnerablecode/vulnerabilities/import_runner.py", line 152, in _process_added_advisories: Function _collect_package_urls was called with the wrong arguments [wrong-arg-types]
         Expected: (batch: Set[vulnerabilities.data_source.Advisory])
  Actually passed: (batch: vulnerabilities.data_source.Advisory)
</failure>
  </testcase>
</testsuite>

for the stdout error produced

File "/home/shivam/coding/opensource/vulnerablecode/vulnerabilities/import_runner.py", line 152, in _process_added_advisories: Function _collect_package_urls was called with the wrong arguments [wrong-arg-types]
         Expected: (batch: Set[vulnerabilities.data_source.Advisory])
  Actually passed: (batch: vulnerabilities.data_source.Advisory)

Basically wrapping the error's found by pytype in a <failure> tag .

Is this the expected junit output ? I haven't used junit so pardon me if this comes off silly ;)

sbs2001 avatar Jul 20 '20 14:07 sbs2001

More or less, yes. There's attributes in the schema which would be useful to fill in, such as severity levels and whatnot.

The main use case I'd have for this would be integration with jenkins dashboards.

adam-azarchs avatar Nov 04 '20 22:11 adam-azarchs