Provide an API for custom expectations with diagnostics
Description
Spawned from the discussion in https://forums.swift.org/t/brainstorming-customizing-matchers/67456/11:
When a custom comparison is needed, it was recommended that users could just write whatever comparison they need as a Bool-returning function and pass that into #expect. That's fine from the point of view of keeping the #expect API simple, but expectations will often want to provide better diagnostics than what can be extracted from the arguments.
For example, pretend we didn't have isSuperset(of:) and we wrote our own terrible predicate:
extension Collection where Element: Hashable {
func isThisASuperset(of smaller: some Collection<Element>) -> Bool {
let diff = Set(smaller).subtracting(Set(self))
return diff.isEmpty
}
}
@Test func foo() {
let x = [1, 3]
let y = [1, 2, 4, 5]
#expect(x.isThisASuperset(of: y))
}
When running this test, the runner produces the following diagnostic:
Expectation failed: (x → [1, 3]).isItASuperset(of: y → [1, 2, 4, 5])
which is helpful! But what I'd really like is something like:
Expectation failed: argument contained [2, 4, 5] which were not in the receiver -- (x → [1, 3]).isItASuperset(of: y → [1, 2, 4, 5])
One way to achieve this would be to create an #expect overload that takes, say, an ExpectationResult instead. That could look like the following:
enum ExpectationResult {
case success
case failure(reason: String)
}
and then the predicate becomes:
extension Collection where Element: Hashable {
func isThisASuperset(of smaller: some Collection<Element>) -> ExpectationResult {
let diff = Set(smaller).subtracting(Set(self))
return diff.isEmpty
? .success
: .failure(reason: "argument contained \(diff) which were not in the receiver")
}
}
This would be a bare minimum API for these kinds of diagnostics, and simple enough for most purposes. It wouldn't be advanced enough to factor in the named of the arguments though; if we wanted the output to be something like this, we'd need something more complex:
Expectation failed: 'y' contained [2, 4, 5] which were not in 'x' -- (x → [1, 3]).isItASuperset(of: y → [1, 2, 4, 5])
Expected behavior
No response
Actual behavior
No response
Steps to reproduce
No response
swift-testing version/commit hash
0.0.0-initial
Swift & OS version (output of swift --version && uname -a)
No response
Tracked internally with rdar://106832903. Using this OSS issue to track going forward.