json-sanitizer icon indicating copy to clipboard operation
json-sanitizer copied to clipboard

Fix flaky EvalMinifierTest

Open 219sansim opened this issue 2 years ago • 1 comments

Description

There are some assertions in the test testMinify which rely on the fact that iterating over the pool map returns elements in a consistent order. However, a HashMap may return different orders of the tokens. Instead a better Map to use is LinkedHashMap which ensures that the token keys are returned in a consistent order. https://github.com/OWASP/json-sanitizer/blob/fc612ab374de73d03864d56fb87b6a103b234489/src/test/java/com/google/json/EvalMinifierTest.java#L50-L55

Motivation and Context

There is a flaky test detected in EvalMinifierTest.java - testMinify. This test may sometimes fail since it assumes that iterating over the Map keys will be the same as their inserting order.

Another, way to resolve this error could be to change the test in such a way that the assertions don't assume a particular order of the keys.

How Has This Been Tested?

Reproduction of error Run maven tests with NonDex tool

mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=com.google.json.EvalMinifierTest

The test passes after changing the HashMap to LinkedHashMap Error Output

Running com.google.json.EvalMinifierTest
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec <<< FAILURE!
testMinify(com.google.json.EvalMinifierTest)  Time elapsed: 0.002 sec  <<< FAILURE!
junit.framework.ComparisonFailure: ['foo','bar','foo','bar','foo','bar','foo','bar','foo','bar','foo','bar','foo','bar','foo','bar','foo','bar','foo','bar','foo','z'] expected:<...unction(a,b){return[[a,b,a,b,a,b,a,b,a,b,a,b,a,b,a,b,a,b,a,b,a,"z"]}("foo","bar]"))> but was:<...unction(a,b){return[[b,a,b,a,b,a,b,a,b,a,b,a,b,a,b,a,b,a,b,a,b,"z"]}("bar","foo]"))>

219sansim avatar Nov 22 '23 18:11 219sansim

@mikesamuel can you review this PR and eventually merge?

dbelyaev avatar Oct 21 '24 11:10 dbelyaev