plantuml-gradle-plugin icon indicating copy to clipboard operation
plantuml-gradle-plugin copied to clipboard

Fix flaky test

Open ThugJudy opened this issue 7 months ago • 0 comments

This PR aims to fix the following 3 flaky test cases:

com.github.roroche.plantuml.BuildClassDiagramTaskTest.produceTests com.github.roroche.plantuml.diagrams.ClassDiagramTest.produceTests com.github.roroche.plantuml.diagrams.PrintDiagramTest.produceTests

I found and confirmed the flaky behavior using an open-source research tool NonDex, which shuffles implementations of nondeterminism operations.

Problem:

The order of the attributes of class that is obtained by ClassDiagramBuilder's build function is prone to changes and causes the flakiness

Solution:

Since we do not have access to the internal library, I have a added custom function that verifies that the strings are similar

Steps to reproduce The following command can be used to reproduce the assertion errors and verify the fix.

Integrate NonDex:

Add the following snippet to the top of the build.gradle in $PROJ_DIR:

plugins { id 'edu.illinois.nondex' version '2.1.1-1' } Append to build.gradle in $PROJ_DIR:

apply plugin: 'edu.illinois.nondex' Execute Test with Gradle:

./gradlew --info test --tests com.github.roroche.plantuml.diagrams.PrintDiagramTest.produceTests ./gradlew --info test --tests com.github.roroche.plantuml.diagrams.ClassDiagramTest.produceTestsorg.apereo.portal.events.tincan.json.LrsDataModelSerializeTest.testLrsObjectSerialize ./gradlew --info test --tests com.github.roroche.plantuml.BuildClassDiagramTaskTest.produceTestsorg.apereo.portal.events.AnalyticsIncorporationComponentEventSerializationTest.testEventFiltering

Run NonDex:

./gradlew --info nondexTest --tests=com.github.roroche.plantuml.diagrams.PrintDiagramTest.produceTests --nondexRuns=50 ./gradlew --info nondexTest --tests=com.github.roroche.plantuml.diagrams.ClassDiagramTest.produceTests --nondexRuns=50 ./gradlew --info nondexTest --tests=com.github.roroche.plantuml.BuildClassDiagramTaskTest.produceTests --nondexRuns=50 Test Environment:

Java version "1.8.0_381" macOS Venture Version 13.4.1 (22F82) Please let me know if you have any concerns or questions.

ThugJudy avatar Nov 17 '23 00:11 ThugJudy