FHIR
FHIR copied to clipboard
intermittent audit test failure in CI
Describe the bug We were seeing intermittent audit test failures a while back. Those seemed to stabalize for a while, but recently we saw another failure for a test that usually passes.
From https://github.com/LinuxForHealth/FHIR/actions/runs/3819990229/jobs/6497976982
Error: org.linuxforhealth.fhir.server.test.SearchRevIncludeTest.testCreateNutritionOrders Time elapsed: 10.019 s <<< FAILURE!
java.util.concurrent.ExecutionException: java.lang.AssertionError: expected [SUCCESSFUL] but found [SERVER_ERROR]
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest.testCreateNutritionOrders(SearchRevIncludeTest.java:495)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:677)
at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:962)
at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.testng.TestRunner.privateRun(TestRunner.java:806)
at org.testng.TestRunner.run(TestRunner.java:601)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
at org.testng.SuiteRunner.run(SuiteRunner.java:330)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
at org.testng.TestNG.runSuites(TestNG.java:1099)
at org.testng.TestNG.run(TestNG.java:1067)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:324)
at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:74)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:123)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
Caused by: java.lang.AssertionError: expected [SUCCESSFUL] but found [SERVER_ERROR]
at org.testng.Assert.fail(Assert.java:110)
at org.testng.Assert.failNotEquals(Assert.java:1413)
at org.testng.Assert.assertEqualsImpl(Assert.java:149)
at org.testng.Assert.assertEquals(Assert.java:131)
at org.testng.Assert.assertEquals(Assert.java:643)
at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest$NutritionOrderCallable.call(SearchRevIncludeTest.java:533)
at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest$NutritionOrderCallable.call(SearchRevIncludeTest.java:519)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Error: org.linuxforhealth.fhir.server.test.SearchRevIncludeTest.testDeleteNutritionOrders Time elapsed: 1.016 s <<< FAILURE!
java.util.concurrent.ExecutionException: java.lang.AssertionError: expected [false] but found [true]
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest.testDeleteNutritionOrders(SearchRevIncludeTest.java:701)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:69)
at org.testng.internal.invokers.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:361)
at org.testng.internal.invokers.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:296)
at org.testng.internal.invokers.TestMethodWorker.invokeAfterClassMethods(TestMethodWorker.java:222)
at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:131)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.testng.TestRunner.privateRun(TestRunner.java:806)
at org.testng.TestRunner.run(TestRunner.java:601)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
at org.testng.SuiteRunner.run(SuiteRunner.java:330)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
at org.testng.TestNG.runSuites(TestNG.java:1099)
at org.testng.TestNG.run(TestNG.java:1067)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:324)
at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:74)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:123)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
Caused by: java.lang.AssertionError: expected [false] but found [true]
at org.testng.Assert.fail(Assert.java:110)
at org.testng.Assert.failNotEquals(Assert.java:1413)
at org.testng.Assert.assertFalse(Assert.java:78)
at org.testng.Assert.assertFalse(Assert.java:88)
at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest$NutritionOrderCallableDelete.call(SearchRevIncludeTest.java:735)
at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest$NutritionOrderCallableDelete.call(SearchRevIncludeTest.java:718)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
...
Error: Failures:
Error: SearchRevIncludeTest.testCreateNutritionOrders:495 » Execution java.lang.AssertionError: expected [SUCCESSFUL] but found [SERVER_ERROR]
Error: SearchRevIncludeTest.testDeleteNutritionOrders:701 » Execution java.lang.AssertionError: expected [false] but found [true]
[INFO]
Error: Tests run: 1337, Failures: 2, Errors: 0, Skipped: 113
Environment Which version of LinuxForHealth FHIR Server?
To Reproduce ?
Expected behavior tests should consistently pass or fail
Additional context Add any other context about the problem here.
There were a couple of NullPointerExceptions in the audit build logs(https://github.com/LinuxForHealth/FHIR/actions/runs/3819990229/jobs/6497976982) which has been fixed as part of pull request: https://github.com/LinuxForHealth/FHIR/pull/4130 .