flake8-formatter-junit-xml icon indicating copy to clipboard operation
flake8-formatter-junit-xml copied to clipboard

Pass line and file as text rather than xml attribute

Open dmgening opened this issue 6 years ago • 3 comments

Hi! I have trouble displaying resulting XML with Gitlab test reports feature. Currently, it only shows the source that generated the error and no information for exact place where the error has occurred.

As I understand Gitlab closely follows JUnit XML specification by IBM (or at least they bother to put a link to it from their documentation) and it does not specify file and line attributes for TestCase element. https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_14.1.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html

Could we have an option to write them as plain text output? Either to failure.message or failure.text would be great.

dmgening avatar Jan 09 '19 15:01 dmgening

Thank you for reporting. I didn't know IBM's specification!

This formatter's failure.message already includes filePath and line. ref: https://github.com/astj/flake8-formatter-junit-xml/blob/master/flake8_formatter_junit_xml/formatter.py#L28 example: https://github.com/astj/flake8-formatter-junit-xml/blob/master/tests/expected.xml#L5 (which means in file some/filename.py at line 2 and column 1).

Isn't it enough?

astj avatar Jan 09 '19 15:01 astj

Weird, it might be multiple thing overlapping then. I'll try to gather some details how xml is processed.

To be more specific here is what i see in report attached to merge request:

screenshot 2019-01-09 at 20 53 42 screenshot 2019-01-09 at 20 54 02

And here is actual XML output:

	<testsuite disabled="0" errors="0" failures="1" name="flake8._/errors_py" skipped="0" tests="1" time="0">
		<testcase file="./errors.py" line="138" name="F811, redefinition of unused 'CardLostError' from line 120">
			<failure message="./errors.py:138:1: F811 redefinition of unused 'CardLostError' from line 120" type="failure">class CardLostError(ProcessingError):
^</failure>
		</testcase>
	</testsuite>

dmgening avatar Jan 09 '19 17:01 dmgening

So I've taken some time and found actual parser Gitlab CE uses (https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/parsers/test/junit.rb), model (https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/reports/test_case.rb) and list of exposed fields (https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/serializers/test_case_entity.rb)

At this point, I think I've misled you about JUnit spec since they are gathering data really close to what you are generating :) Sorry for that. Althrough seems like JUnit team publishing XSD: https://github.com/junit-team/junit5/blob/master/platform-tests/src/test/resources/jenkins-junit.xsd

As for my issue, I think it might be related to missing classname property. They have an opened issue for that, but I didn't had time to check if it would change the output. https://gitlab.com/gitlab-org/gitlab-ce/issues/50964

dmgening avatar Jan 10 '19 00:01 dmgening