Investigate flaky Flyway test

[gastaldi]

There's something fishy with Flyway lately:

2022-10-14T23:15:00.4215101Z Caused by: java.lang.RuntimeException: Failed to start quarkus
2022-10-14T23:15:00.4215480Z 	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
2022-10-14T23:15:00.4215875Z 	at io.quarkus.runtime.Application.start(Application.java:101)
2022-10-14T23:15:00.4216406Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2022-10-14T23:15:00.4216956Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2022-10-14T23:15:00.4217570Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2022-10-14T23:15:00.4218056Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2022-10-14T23:15:00.4218493Z 	at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:237)
2022-10-14T23:15:00.4218960Z 	at io.quarkus.test.QuarkusUnitTest.beforeAll(QuarkusUnitTest.java:642)
2022-10-14T23:15:00.4219274Z 	... 43 more
2022-10-14T23:15:00.4219665Z Caused by: org.flywaydb.core.api.exception.FlywayValidateException: Validate failed: Migrations have failed validation
2022-10-14T23:15:00.4220118Z Migration checksum mismatch for migration version 1.0.0
2022-10-14T23:15:00.4220667Z -> Applied to database : 156834024
2022-10-14T23:15:00.4220964Z -> Resolved locally    : -986920762
2022-10-14T23:15:00.4221294Z Either revert the changes to the migration, or run repair to update the schema history.
2022-10-14T23:15:00.4221716Z Need more flexibility with validation rules? Learn more: https://rd.gt/3AbJUZE
2022-10-14T23:15:00.4222091Z 	at org.flywaydb.core.Flyway.lambda$migrate$0(Flyway.java:134)
2022-10-14T23:15:00.4222714Z 	at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:204)
2022-10-14T23:15:00.4223102Z 	at org.flywaydb.core.Flyway.migrate(Flyway.java:128)
2022-10-14T23:15:00.4223538Z 	at io.quarkus.flyway.runtime.FlywayRecorder.doStartActions(FlywayRecorder.java:79)
2022-10-14T23:15:00.4224040Z 	at io.quarkus.deployment.steps.FlywayProcessor$startActions1770701860.deploy_0(Unknown Source)
2022-10-14T23:15:00.4224543Z 	at io.quarkus.deployment.steps.FlywayProcessor$startActions1770701860.deploy(Unknown Source)
2022-10-14T23:15:00.4224880Z 	... 51 more

/cc @gastaldi if you have an idea.

Originally posted by @gsmet in https://github.com/quarkusio/quarkus/issues/28613#issuecomment-1279727919

[quarkus-bot[bot]]

/cc @cristhiank, @geoand, @gsmet

[gastaldi]

Those error messages are expected and are part of the io.quarkus.flyway.test.FlywayExtensionRepairAtStartTest test

[geoand]

But they shouldn't be making the whole test fail, right?

[gastaldi]

Exactly, I'm still investigating what's causing the test failures. Weird is that I see these error messages but the build doesn't fail for me locally

[gsmet]

Could it be because of the order in which the tests are executed? Might be worth to order them alphabetically then in reverse order and see how things go.

[gastaldi]

I don't see a way to test in reverse order, so I ran with these settings a couple of times and saw some failures. This PR seems to have fixed them:

mvn test -Djunit.jupiter.testmethod.order.default=org.junit.jupiter.api.MethodOrderer\$Random -Djunit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer\$Random