`swift test` output does not differentiate tests with the same name from different suites

[jmschonfeld]

Description

Given I have two test suites that look like the following:

struct Day01Tests {
    @Test func part1() { /* ... */ }
    @Test func part2() { /* ... */ }
}

struct Day02Tests {
    @Test func part1() { /* ... */ }
    @Test func part2() { /* ... */ }
}

The output of swift test does not allow the reader to distinguish between the two part1 tests in separate test suites. I encountered this when working on my AdventOfCode project, but I think there's a variety of projects that use generic function names for tests (think testEquatable/testHashable in suites for each publicly defined type) that would need to differentiate between these tests.

Is there a way the output could differentiate? IIRC XCTest fully qualified the test names with the suite names, but I might be wrong on that

Expected behavior

I expect that the produced output of swift test would differentiate (potentially via fully qualifying) tests with the same name in different suites

Actual behavior

Tests are not differentiated:

􀟈 Test run started.
􀄵 Testing Library Version: 102 (arm64e-apple-macos13.0)
􀟈 Suite Day01Tests started.
􀟈 Suite Day02Tests started.
􀟈 Test part1() started.
􀟈 Test part2() started.
􀟈 Test part1() started.
􀟈 Test part2() started.
􁁛 Test part2() passed after 0.001 seconds.
􁁛 Test part1() passed after 0.001 seconds.
􁁛 Test part1() passed after 0.001 seconds.
􁁛 Test part2() passed after 0.001 seconds.
􁁛 Suite Day02Tests passed after 0.011 seconds.
􁁛 Suite Day01Tests passed after 0.011 seconds.
􁁛 Test run with 8 tests passed after 0.011 seconds.

Steps to reproduce

1. Add the sample code above to a swift package
2. Run tests in Xcode (or presumably with swift test as well)

swift-testing version/commit hash

Testing Library Version: 102 (arm64e-apple-macos13.0)

Swift & OS version (output of swift --version ; uname -a)

Xcode 16.1 (16B40) on macOS 15.1.1 (24B91)

[grynspan]

Workarounds:

• Name your tests distinctly :D
• Use the custom display name feature to provide more verbose names for these tests
• Use --verbose

But yes, I agree we can probably do something here to improve what you're seeing.

[jmschonfeld]

Yeah the workarounds can help to locally distinguish, but I don't want to actually check any of those in to our sources to display them in CI when triaging failures is critical. I know for example in Foundation we have a variety of suites like LocaleTests/DateTests/etc. and all of them have testEquality/testHashable/testCodable etc. and it'd be unfortunate to need to name them (either via function name or custom display name) things like testDateEquality when they're already grouped inside DateTests. Hopefully improving what we see here isn't too much of a headache.

[grynspan]

It's not hard to make a change here, we just need to figure out exactly what to write. In verbose mode, we can put the full ID of the test, for instance, while in non-verbose mode we can put something like "test foo in suite bar".

[grynspan]

Tracked internally as rdar://140834848.

[KentLottis-jwn]

I'm leading adoption for my organization and ran into this yesterday and +1 to the problem of having many different components with a common set of sub-suite and test function names. It seems all we need is to use the fully-qualified test (possibly in addition to the friendly name):

"Test WidgetEngine.ErrorHandling.invalidName() ("Verify invalid name handling") started"