SpringBoot-Learning Removing flakiness using .json() to check JSON string responses

Removing flakiness using .json() to check JSON string responses

Open akshathphillips opened this issue 1 year ago • 4 comments

What is the purpose of this PR

This PR fixes the error resulting from a flaky test: com.didispace.chapter21.Chapter21ApplicationTests
The mentioned test may fail or pass without changes made to the source code when it is run depending on the JSON's non-deterministic deserialization.

Why the tests fail

This test fails because when the JSON's response is deserialized to a String.class, the order of the fields inside the object may change to be in random positions. For example, {"id":1, "age":20}, or {"age":20, "id":1}

Reproduce the test failure

Run the tests with NonDex maven plugin. The commands to recreate the flaky test failures are:
- Run the test with the NonDex maven plugin
  - mvn -pl 2.x/chapter2-1 test edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=com.didispace.chapter21.Chapter21ApplicationTests#testUserController -DnondexRuns=10
Fixing the flaky test now may prevent flaky test failures for users checking out this project.

Expected results

The test should run successfully when run with NonDex.

Actual Result

We get the following failure for test com.didispace.chapter21.Chapter21ApplicationTests

[ERROR] Failures: 
[ERROR]   Chapter21ApplicationTests.testUserController:52 Response content
Expected: "[{\"id\":1,\"name\":\"测试大师\",\"age\":20}]"
     but: was "[{\"age\":20,\"id\":1,\"name\":\"测试大师\"}]"

Fix

For test com.didispace.chapter21.Chapter21ApplicationTests - Parse the expected and actual strings as JSON and assert the two are "similar" - i.e. they contain the same attribute-value pairs regardless of formatting with a lenient checking. The .json(), does a lenient checking on the fields, which solves this issue.