swift-testing
swift-testing copied to clipboard
swiftlang
[WIP] Rethink how we capture expectation conditions and their subexpressions.
This PR completely rewrites how we capture expectation conditions.
Explanation
For example, given the following expectation:
#expect(x.f() == 123)
We currently detect that there is a binary operation and emit code that calls the binary operator as a closure and passes the left-hand value and right-hand value, then checks that the result of the operation is true.
This is sufficient for simpler expressions like that one, but more complex ones (including any that involve try or await keywords) cannot be expanded correctly. With this PR, such expressions can generally be expanded correctly.
The change involves rewriting the macro condition as a closure to which is passed a local, mutable "context" value. Subexpressions of the condition expression are then rewritten by walking the syntax tree of the expression (using typical swift-syntax API) and replacing them with calls into the context value that pass in the value and related state.
If the expectation ultimately fails, the collected data is transformed into an instance of the SPI type Expression that contains the source code of the expression and interesting subexpressions as well as the runtime values of those subexpressions.
Nodes in the syntax tree are identified by a unique ID which is composed of the swift-syntax ID for that node as well as all its parent nodes in a compact bitmask format. These IDs can be transformed into graph/trie key paths when expression/subexpression relationships need to be reconstructed on failure, meaning that a single rewritten node doesn't otherwise need to know its "place" in the overall expression.
Examples
As an example, this expectation…
#expect(g() > 500)
… previously expanded to…
Testing.__checkBinaryOperation(
g(),
{ $0 > $1() },
500,
expression: .__fromBinaryOperation(
.__fromSyntaxNode("g()"),
">",
.__fromSyntaxNode("500")
),
comments: [],
isRequired: false,
sourceLocation: Testing.SourceLocation.__here()
).__expected()
… but will now expand to:
Testing.__checkCondition(
{ (__ec: inout Testing.__ExpectationContext) -> Swift.Bool in
__ec(__ec(g(),0x2) > 500,0x0)
},
sourceCode: [
0x0: "g() > 500",
0x2: "g()"
],
comments: [],
isRequired: false,
sourceLocation: Testing.SourceLocation.__here()
).__expected()
More interestingly, an expression with side effects or complex nested operations can also be translated. For example, this throwing expression…
#expect((try g() > 500) && true)
… was…
Testing.__checkValue(
(try g() > 500) && true,
expression: .__fromSyntaxNode("(try g() > 500) && true"),
comments: [],
isRequired: false,
sourceLocation: Testing.SourceLocation.__here()
).__expected()
… but now becomes:
try Testing.__checkCondition(
{ (__ec: inout Testing.__ExpectationContext) -> Swift.Bool in
try Testing.__requiringTry(__ec(__ec((try __ec(__ec(g(),0x1ba) > 500,0xba)),0x2) && __ec(true,0x400000),0x0))
},
sourceCode: [
0x0: "(try g() > 500) && true",
0x2: "(try g() > 500)",
0xba: "g() > 500",
0x1ba: "g()",
0x400000: "true"
],
comments: [],
isRequired: false,
sourceLocation: Testing.SourceLocation.__here()
).__expected()
Caveats
There remain a few caveats (that also generally affect the current implementation):
- Mutating member functions are syntactically indistinguishable from non-mutating ones and miscompile when rewritten;
- Expressions involving move-only types are also indistinguishable, but need lifetime management to be rewritten correctly; and
- ~~Expressions where the
tryorawaitkeyword is outside the#expectmacro cannot be expanded correctly because the macro cannot see those keywords during expansion.~~ This is now resolved. - Operations that cause us to capture move-only values (which is disallowed in most contexts) are syntactically indistinguishable from "normal" operations, which causes compilation of such code to fail obscurely.
The first issue might be resolvable in the future using pointer tricks, although I don't hold a lot of hope for it. The second issue is probably resolved by non-escaping types. The third issue is an area of active exploration for us and the macros/swift-syntax team.
Resolved Issues
Resolves #162. Resolves rdar://135437448.
Checklist:
- [x] Code and documentation should follow the style of the Style Guide.
- [x] If public symbols are renamed or modified, DocC references should be updated.
This PR is blocked by https://github.com/swiftlang/swift-syntax/pull/2724 (or some alternate solution) because the following expressions would have previously compiled while they will fail with this change:
try #expect(await f())
try #expect(x[123])