rules_xcodeproj icon indicating copy to clipboard operation
rules_xcodeproj copied to clipboard

Bug: XCTest traces are missing in failure context

Open luispadron opened this issue 1 year ago • 6 comments

Description

In Xcode 12+ a test case which calls into a function that then calls something like XCTFail/XCTAssert* has proper traces back up the stack to the originating caller. This helps diagnose where the test case failure is called. This happens automatically in a new Xcode 12+ project, for example:

Screenshot 2024-06-18 at 11 07 52 AM

The same code in a BwB rules_xcodeproj project only shows the trace for the XCTAssert:

Screenshot 2024-06-18 at 11 14 28 AM

Reproduction steps

Write a test which calls into some other function that then fails. Or see the attached zip with a vanilla Xcode and rules_xcodeproj example to reproduce the issue.

TestTracingExample.zip

Expected behavior

The test failures look like they do in vanilla Xcode

rules_xcodeproj version

2.4.0

Xcode version

15.4

Bazel version

7.2.0

rules_apple version

3.5.1

rules_swift version

1.18.0

Additional information

No response

luispadron avatar Jun 18 '24 15:06 luispadron

This bug is extremely annoying, as it drastically complicates daily work with XCTest. Could it be prioritized ?

shuguenot avatar Sep 03 '24 10:09 shuguenot

I expect this is yet another "relative paths break Xcode" issue (e.g. #1119, #1284, #1336). Try launching Xcode from the Bazel execution root (I believe described in one or more of those issues) to see if that fixes it.

This might be fixable the same way that we fixed test issues to begin with, by swizzling something: https://github.com/bazelbuild/apple_support/pull/128.

brentleyjones avatar Sep 17 '24 13:09 brentleyjones

Same issue with Swift Testing, we only see test failures inline at the top-level test function level, not at the actual point of failure in the test sources. Would really love to see this issue fixed as-well 🙏

dask8 avatar Nov 19 '24 10:11 dask8

I tried the opening project in execution root work around but that didn't seem to work for this case.

This seems to effect other error message tracing as well, like a swift macro compiling with an error does not trace back to the call site

luispadron avatar Mar 04 '25 05:03 luispadron

Some more testing here:

Image
Image

In the first image (Vanilla Xcode) the failure has the context where as second image (Bazel) the information seems to be lost.

luispadron avatar Apr 05 '25 17:04 luispadron

Some more debugging:

Looks like XCTSourceCodeFrame contains the address and symbol info, when i print the XCTSourceCodeFrame.symbolicationError instance variable I see:

Symbolication Error: Error Domain=XCTSymbolicationServiceErrorDomain Code=6 "Unable to get source info for address 4382236627 in image 'NonEmpty-Unit-UnitTests'" UserInfo={NSLocalizedDescription=Unable to get source info for address 4382236627 in image 'NonEmpty-Unit-UnitTests'}

luispadron avatar Apr 05 '25 19:04 luispadron