Fix nondeterministic failures in HttpOperationsTest
Description
Type of change
- [ ] New feature
- [ ] Bug fix for existing feature
- [x] Code quality improvement
- [x] Addition or Improvement of tests
- [ ] Addition/Improvement of documentation
Summary
The flaky behavior was detected in org.zalando.riptide.compatibility.HttpOperationsTest#shouldPut when running under NonDex randomization, which revealed that the test relied on deterministic field ordering in the serialized JSON request body.
Example of the original test assertion:
verifyRequestBody(recordedRequest, "{\"name\":\"D. Fault\",\"birthday\":\"1984-09-13\"}");
However, the JSON serialization of request bodies can vary in key ordering depending on the underlying Map or serializer implementation used within Jackson. Since JSON field order is not guaranteed by the specification, different JVM runs or environments may produce:
Run 1: {"name":"D. Fault","birthday":"1984-09-13"}
Run 2: {"birthday":"1984-09-13","name":"D. Fault"}
The helper method verifyRequestBody(...) was asserting equality on raw JSON strings:
assertEquals(expectedBody, recordedRequest.getBody().readString(UTF_8));
This made the test order-sensitive to the serialization order of JSON fields, even though the semantic data was identical.
Failure Message running NonDex:
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] HttpOperationsTest.shouldExchange:303->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[ERROR] HttpOperationsTest.shouldExecute:288->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[ERROR] HttpOperationsTest.shouldExecute:288->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[ERROR] HttpOperationsTest.shouldExecute:288->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[ERROR] HttpOperationsTest.shouldPostForLocation:143->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[ERROR] HttpOperationsTest.shouldPostForLocation:143->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[ERROR] HttpOperationsTest.shouldPostForObject:168->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[ERROR] HttpOperationsTest.shouldPut:190->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[ERROR] HttpOperationsTest.shouldPut:190->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
[INFO]
Fix
The test verification was updated to perform semantic comparison instead of string equality: • Both the expected and actual JSON strings are parsed into maps using Jackson’s ObjectMapper.readValue(). • The test now asserts equality on these parsed Map<String, String> objects, which ignores field order differences. • The content type check (application/json) remains intact to preserve original behavior.
Updated code excerpt:
Map<String, String> expected = MAPPER.readValue(expectedBody, new TypeReference<>() {});
Map<String, String> actual = MAPPER.readValue(requestBody, new TypeReference<>() {});
assertEquals(expected, actual);
This ensures deterministic, logically equivalent test validation across environments and JVMs, eliminating the order-dependent flaky behavior.
Related Tests
org.zalando.riptide.compatibility.HttpOperationsTest#shouldExchange
org.zalando.riptide.compatibility.HttpOperationsTest#shouldExecute
org.zalando.riptide.compatibility.HttpOperationsTest#shouldPatchForObject
org.zalando.riptide.compatibility.HttpOperationsTest#shouldPostForLocation
org.zalando.riptide.compatibility.HttpOperationsTest#shouldPostForObject
org.zalando.riptide.compatibility.HttpOperationsTest#shouldPut
How to Reproduce
Reproduced the failure using NonDex, a tool from the University of Illinois designed to detect ID tests (Iteration-Dependent tests).
Build module and Run a single test (no shuffling) All tests pass.
Run with NonDex (shuffling)
cd riptide-compatibility
mvn -T1 edu.illinois:nondex-maven-plugin:2.2.1:nondex \
-Denforcer.skip=true \
-Dtest=org.zalando.riptide.compatibility.HttpOperationsTest \
-DnondexRuns=10 \
-DnondexMode=FULL \
-DfailIfNoTests=false \
-Dmaven.test.failure.ignore=true
Expected Example
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] HttpOperationsTest.shouldExchange:303->verifyRequestBody:347 expected: <{"name":"D. Fault","birthday":"1984-09-13"}> but was: <{"birthday":"1984-09-13","name":"D. Fault"}>
Verification
- ✅ test passes.
- ✅ Multiple NonDex runs (
-DnondexRuns=100) pass with no flakes. - ✅ No behavior change.