swagger-core icon indicating copy to clipboard operation
swagger-core copied to clipboard

Fixed 35 different flaky tests with ID category

Open xinying-7 opened this issue 11 months ago • 1 comments

These tests are failing NonDex since they're directly comparing JSON Strings which can have different key order when run multiple times.

Click on to see more details on the error message
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   NumericFormatTest.testFormatOfDecimal:48 expected [{
  "ModelWithDecimalFields" : {
    "type" : "object",
    "properties" : {
      "id" : {
        "minimum" : 3.3,
        "exclusiveMinimum" : false,
        "type" : "number",
        "format" : "double"
      }
    }
  }
}] but found [{
  "ModelWithDecimalFields" : {
    "properties" : {
      "id" : {
        "minimum" : 3.3,
        "format" : "double",
        "exclusiveMinimum" : false,
        "type" : "number"
      }
    },
    "type" : "object"
  }
}]

All the tests I fixed have similar issues, so reproducing them requires similar steps. For example, for the testFormatOfDecimal function located inio/swagger/v3/core/converting/NumericFormatTest.java , Here's the steps to reproduce:

Run this at the root directory: mvn -pl modules/swagger-core edu.illinois:nondex-maven-plugin:2.1.7:nondex -Dtest=io.swagger.v3.core.converting.NumericFormatTest#testFormatOfDecimal -DnondexRuns=1

(Note: The failing test might not be seen if every test happens to have the JSON string in the correct order. Try running it several times or increase the the number of runs with -DnondexRuns= to reproduce the issue.)

To fix this, I converted the actual and expected JSON String into JSON objects and compare with JSONAssert.assertEquals, which ensures correct equality checks regardless of key order.

xinying-7 avatar Nov 15 '24 23:11 xinying-7