defradb
defradb copied to clipboard
A bug in testing framework if error is raised and assertion of the error passes, it omits other assertions
Describe the problem
Our testing framework seems to be not asserting the other result values once an error is encountered and assertion for that ExpectedError
passes.
To Reproduce
Both the below tests pass (Results
field in each is different):
func TestStopsAssertionOnceRaisedErrorIsAsserted1(t *testing.T) {
test := testUtils.RequestTestCase{
Description: "Delete multiple documents that exist sub selection, should give error.",
Request: `mutation { delete_user(ids: ["bae-xyz"]) }`,
Results: []map[string]any{{"_key": "Doesn't assert the result if raised expected error."}},
ExpectedError: "Field \"delete_user\" of type \"[user]\" must have a sub selection.",
}
executeTestCase(t, test)
}
func TestStopsAssertionOnceRaisedErrorIsAsserted1(t *testing.T) {
test := testUtils.RequestTestCase{
Description: "Delete multiple documents that exist sub selection, should give error.",
Request: `mutation { delete_user(ids: ["bae-xyz"]) }`,
Results: []map[string]any{},
ExpectedError: "Field \"delete_user\" of type \"[user]\" must have a sub selection.",
}
executeTestCase(t, test)
}
Expected behavior
The test should assert other fields (like Results
in this case) in this case, and ensure tests only pass if all testing args are what we expect them to be, which doesn't make sense so preferably not allow this at all in the testing framework.