xcresulttool
xcresulttool copied to clipboard
kishikawakatsumi
Code coverage report is not showing up
I had 255 test cases and 77 cases failed I had these errors, is it because of that?
Error: The 'summary' will be truncated because the character limit (65535) exceeded.
Error: The 'text' will be truncated because the character limit (65535) exceeded.
Error: Annotations that exceed the limit (50) will be truncated.
No, that message is irrelevant. That message simply informs you that the report was partially truncated because it exceeded the internal limits of the GitHub Checks API. Strictly speaking, it is a warning message, not an error.
I think the reason why the code coverage is not shown is that the XCResult bundle does not contain coverage. Can you confirm that the test bundle contains the coverage data? Or can you share the test bundle for me to investigate the problem?
It does contain it
When I run xcrun xccov view --report --json Example.xcresult I get a json result of code coverage in the terminal
It seems the report generated is cut off I believe when I run massive amounts of test cases only part of them show up in the final detailed output It shows up in the table but gets truncated in the details view As you can see after a certain number of test classes it comes like this and then moves to annotations
Oh, that's probably why. GitHub Checks has a character limit, and I think it exceeded that limit just by reporting lots of test failures. That's probably why the test coverage was truncated.
I see, Is there any workaround for this? Like can we disable annotations/detailed view and get code coverage instead?
If you set the show-passed-tests: false parameter, only failed tests will be reported, so the display will be slightly increased.
However, the case where so many tests are failing is currently outside the scope of the use case. What kind of behavior are you expecting? Do you only want to see the test coverage?
Yes tried with that flag, it improved a little bit, but still not covered all test classes and got cut off in details view Yes an option to disable details section or the annotations section so there is enough space for code coverage results. Ideally we can customize what sections need to be displayed in the end result.
I see. Thanks.
Ideally we can customize what sections need to be displayed in the end result.
Yes, I agreed with you. I plan to do that eventually, but it will take some time because I need to rewrite parsing xcresult into a flexible code.
@tonyarnold do you mean you are not getting the test results at all or is it like me just cut off in the middle?
My results did show up (although I misunderstood that it overwrites the results if you try to capture more than one xcresult bundle), however they were truncated like yours @denil-ct.
Is there anyway the tool can generate an output file that we can upload as an artifact to get around the git word limit?
@Dharna We decided to go with this tool as it gave a file output that we could then ingest into our systems. This allowed us to overcome the truncation issue. Although, one thing to note is, that tool was not designed for Xcode or iOS development in general, more oriented towards windows, so you might need to install a couple of dependencies as needed.
@denil-ct I see. Yeah I am mainly looking for iOS and macOS report generation. Thank you for the tool! Will take a look.