quarkus
quarkus copied to clipboard
quarkusio
`SQLException: Unable to enlist connection to existing transaction` when accessing multiple persistence units in the same transaction since 3.8.2
Describe the bug
After updating from 3.8.1 to 3.8.2, some of our tests that insert data in multiple PUs within a single transaction now fail and throw an exception.
That exception is thrown whenever we attempt to access two or more persistence units within a single @Transactional method. This worked fine in previous releases.
We suspect that the bug is due to Agroal 2.3, since we encountered the same problem weeks ago while attempting to force the 2.3 version on older Quarkus releases.
Expected behavior
The transaction commits successfully without any error.
Actual behavior
The transaction is rolled back and this exception is thrown:
io.quarkus.arc.ArcUndeclaredThrowableException: Error invoking subclass method
at io.test.agroal.bug.reproducer.ReproducerApp_Subclass.run(Unknown Source)
at io.test.agroal.bug.reproducer.ReproducerApp_ClientProxy.run(Unknown Source)
at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:132)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
at io.quarkus.runner.GeneratedMain.main(Unknown Source)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:113)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: jakarta.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1283)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:104)
at io.quarkus.narayana.jta.runtime.NotifyingTransactionManager.commit(NotifyingTransactionManager.java:70)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.endTransaction(TransactionalInterceptorBase.java:406)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:171)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:107)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:38)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:61)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:32)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
... 10 more
Caused by: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection [Exception in association of connection to existing transaction] [n/a]
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:63)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:108)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:94)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:116)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:143)
at org.hibernate.engine.jdbc.internal.MutationStatementPreparerImpl.connection(MutationStatementPreparerImpl.java:137)
at org.hibernate.engine.jdbc.internal.MutationStatementPreparerImpl$1.doPrepare(MutationStatementPreparerImpl.java:48)
at org.hibernate.engine.jdbc.internal.MutationStatementPreparerImpl$StatementPreparationTemplate.prepareStatement(MutationStatementPreparerImpl.java:106)
at org.hibernate.engine.jdbc.internal.MutationStatementPreparerImpl.prepareStatement(MutationStatementPreparerImpl.java:38)
at org.hibernate.engine.jdbc.mutation.internal.ModelMutationHelper.standardStatementPreparation(ModelMutationHelper.java:145)
at org.hibernate.engine.jdbc.mutation.internal.ModelMutationHelper.lambda$standardPreparation$0(ModelMutationHelper.java:118)
at org.hibernate.engine.jdbc.mutation.internal.PreparedStatementDetailsStandard.resolveStatement(PreparedStatementDetailsStandard.java:87)
at org.hibernate.engine.jdbc.mutation.internal.JdbcValueBindingsImpl.lambda$beforeStatement$0(JdbcValueBindingsImpl.java:88)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.hibernate.engine.jdbc.mutation.spi.BindingGroup.forEachBinding(BindingGroup.java:51)
at org.hibernate.engine.jdbc.mutation.internal.JdbcValueBindingsImpl.beforeStatement(JdbcValueBindingsImpl.java:85)
at org.hibernate.engine.jdbc.mutation.internal.AbstractMutationExecutor.performNonBatchedMutation(AbstractMutationExecutor.java:104)
at org.hibernate.engine.jdbc.mutation.internal.MutationExecutorSingleNonBatched.performNonBatchedOperations(MutationExecutorSingleNonBatched.java:40)
at org.hibernate.engine.jdbc.mutation.internal.AbstractMutationExecutor.execute(AbstractMutationExecutor.java:52)
at org.hibernate.persister.entity.mutation.InsertCoordinator.doStaticInserts(InsertCoordinator.java:175)
at org.hibernate.persister.entity.mutation.InsertCoordinator.coordinateInsert(InsertCoordinator.java:113)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2873)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:104)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:632)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:499)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:363)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:41)
at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1403)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:484)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2319)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:1976)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
at org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorImpl.beforeCompletion(JtaTransactionCoordinatorImpl.java:336)
at org.hibernate.resource.transaction.backend.jta.internal.synchronization.SynchronizationCallbackCoordinatorNonTrackingImpl.beforeCompletion(SynchronizationCallbackCoordinatorNonTrackingImpl.java:47)
at org.hibernate.resource.transaction.backend.jta.internal.synchronization.RegisteredSynchronization.beforeCompletion(RegisteredSynchronization.java:37)
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:52)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:351)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:69)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:138)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1271)
... 22 more
Caused by: java.sql.SQLException: Exception in association of connection to existing transaction
at io.agroal.narayana.NarayanaTransactionIntegration.associate(NarayanaTransactionIntegration.java:130)
at io.agroal.pool.ConnectionPool.getConnection(ConnectionPool.java:257)
at io.agroal.pool.DataSource.getConnection(DataSource.java:86)
at io.quarkus.hibernate.orm.runtime.customized.QuarkusConnectionProvider.getConnection(QuarkusConnectionProvider.java:23)
at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:46)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnecagedImpl.java:113)
... 59 more
Caused by: java.sql.SQLException: Unable to enlist connection to existing transaction
at io.agroal.narayana.NarayanaTransactionIntegration.associate(NarayanaTransactionIntegration.java:121)
... 64 more
How to Reproduce?
Reproducer: https://github.com/jacopo-cavallarin/agroal-bug-reproducer
Clone the linked repo and follow the instructions in the README
Output of uname -a or ver
Darwin M0-055116363 23.2.0 Darwin Kernel Version 23.2.0: Wed Nov 15 21:55:06 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6020 arm64
Output of java -version
openjdk version "21.0.1" 2023-10-17 LTS OpenJDK Runtime Environment Temurin-21.0.1+12 (build 21.0.1+12-LTS) OpenJDK 64-Bit Server VM Temurin-21.0.1+12 (build 21.0.1+12-LTS, mixed mode)
Quarkus version or git rev
3.8.2
Build tool (ie. output of mvnw --version or gradlew --version)
Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae) Maven home: /Users/jacopocavallarin/.m2/wrapper/dists/apache-maven-3.9.6-bin/3311e1d4/apache-maven-3.9.6 Java version: 21.0.1, vendor: Eclipse Adoptium, runtime: /Users/jacopocavallarin/.sdkman/candidates/java/21.0.1-tem Default locale: en_IT, platform encoding: UTF-8 OS name: "mac os x", version: "14.2.1", arch: "aarch64", family: "mac"
Additional information
No response
Hello! I have the same situation where I'm trying to perform select operations on two datasources in single transaction.
After updating Quarkus from version 3.8.1 to version 3.8.2 I get the following error:
java.sql.SQLException: Enlisted connection used without active transaction
Error stacktrace:
at io.agroal.narayana.XAExceptionUtils.xaException(XAExceptionUtils.java:20)
at io.agroal.narayana.XAExceptionUtils.xaException(XAExceptionUtils.java:8)
at io.agroal.narayana.LocalXAResource.rollback(LocalXAResource.java:89)
at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:338)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:644)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:398)
at io.agroal.narayana.NarayanaTransactionIntegration.associate(NarayanaTransactionIntegration.java:120)
at io.agroal.pool.ConnectionPool.getConnection(ConnectionPool.java:257)
at io.agroal.pool.DataSource.getConnection(DataSource.java:86)
at com.company.cloud.core.quarkus.db.datasource.testcontainers.TransactionsTest.lambda$testConnectionIsNotSharedWithinTransaction$3(TransactionsTest.java:146)
at io.quarkus.narayana.jta.TransactionRunnerImpl.lambda$run$0(TransactionRunnerImpl.java:27)
at io.quarkus.narayana.jta.QuarkusTransactionImpl.callInOurTx(QuarkusTransactionImpl.java:136)
at io.quarkus.narayana.jta.QuarkusTransactionImpl.callRequireNew(QuarkusTransactionImpl.java:106)
at io.quarkus.narayana.jta.QuarkusTransactionImpl.call(QuarkusTransactionImpl.java:29)
at io.quarkus.narayana.jta.TransactionRunnerImpl.run(TransactionRunnerImpl.java:26)
at com.company.cloud.core.quarkus.db.datasource.testcontainers.TransactionsTest.testConnectionIsNotSharedWithinTransaction(TransactionsTest.java:139)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at io.quarkus.test.junit.QuarkusTestExtension.runExtensionMethod(QuarkusTestExtension.java:1013)
at io.quarkus.test.junit.QuarkusTestExtension.interceptTestMethod(QuarkusTestExtension.java:827)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: java.sql.SQLException: Enlisted connection used without active transaction
at io.agroal.pool.ConnectionHandler.verifyEnlistment(ConnectionHandler.java:381)
at io.agroal.pool.ConnectionHandler.transactionRollback(ConnectionHandler.java:352)
at io.agroal.narayana.LocalXAResource.rollback(LocalXAResource.java:86)
... 85 more
This error is reproduced when trying to run the next test:
@Test
void testConnectionIsNotSharedWithinTransaction() throws SQLException {
//Datasources creating and runing flyway scripts ...
Assertions.assertNotEquals(firstDatasource, secondDatasource);
firstDatasource.getConnection().createStatement().execute("INSERT into persons(first_name, last_name) VALUES ('first', 'person')");
secondDatasource.getConnection().createStatement().execute("INSERT into persons(first_name, last_name) VALUES ('second', 'person')");
QuarkusTransaction.requiringNew().run(() -> {
try {
ResultSet firstResultSet = firstDatasource.getConnection().createStatement().executeQuery("SELECT first_name from persons where last_name = 'person'");
Assertions.assertTrue(firstResultSet.next());
Assertions.assertEquals("first", firstResultSet.getString(1));
Assertions.assertFalse(firstResultSet.next());
ResultSet secondResultSet = secondDatasource.getConnection().createStatement().executeQuery("SELECT first_name from persons where last_name = 'person'");
Assertions.assertTrue(secondResultSet.next());
Assertions.assertEquals("second", secondResultSet.getString(1));
Assertions.assertFalse(secondResultSet.next());
} catch (SQLException e) {
Assertions.fail(e);
}
});
}
}
This seems caused by https://github.com/quarkusio/quarkus/pull/39072.
~However~ setting quarkus.datasource.XYZ.jdbc.transactions=xa on the datasources involved as per the discussion there seems to ~cause other issues~ solve the issue.
Is it safe to downgrade agroal to 2.2 in pom.xml or better return to 3.8.1 and waiting for a fix?
I think this is an improvement, but this is a pretty big change, meaning that if you have not paid attention to the transaction handling when using multiple data sources you might need several code changes to fix this. And that's why I think it might be a good idea to revert the Agroal upgrade and introduce it in the next minor version upgrade. Because otherwise this might block some users from upgrading to patches containing security fixes.
Also this might be worth explaining in https://quarkus.io/guides/transaction as you might come across this issue the moment you add the second datasource in the application.
In my case I have several data sources in use and they are in separate modules and I consider them being independent and that's why I have tried to keep the transaction handling separated - event though I have noticed earlier that Quarkus has allowed me to do queries in separate datasources using the same transaction. This change brought up couple of places where my transaction handling was still allowing the same transaction. In these places I was able to fix it either using the annotation:
@Transactional(Transactional.TxType.REQUIRES_NEW)
or if I needed to have separate transactions inside the same method I could use the programmatic approach:
return QuarkusTransaction.requiringNew().call(() -> {
// fetch stuff
});
I hope these tips could be helpful to someone.
At the very least this needs an entry in the migration guide, both 3.9 and 3.8... looks like we collectively skipped that, sorry @gastaldi .
To be honest, I think the change should be reverted for 3.8.x. This is a breaking change and very unexpected. Maybe someone could point out exactly what changed, and why?
There is still the open question whether we can set the agroal version back to 2.2 in our projects: https://github.com/quarkusio/quarkus/issues/39283#issuecomment-1997220206
~~I may have a reproducer involving camel-quarkus. It shows the exception mentioned in https://github.com/quarkusio/quarkus/issues/39283#issuecomment-1991484325.~~
~~https://github.com/turing85/quarkus-camel-transactions~~
~~I am still awaiting confirmation from the camel team that the application does (as per the camel specification) what I think it should do.~~
~~Side note: camel-quarkus was not updated from 3.8.1 to 3.8.2; it remained at version 3.8.0.~~
It seems that https://github.com/agroal/agroal/commit/342ee87e372c324b7356542499dc4e2610eb5afb is the commit that caused the issue to appear. However, this alone does not seem to be the root cause. The root cause seems to be https://github.com/agroal/agroal/commit/ced9e8b285bf7e56b07a93ff019a405fe92380ad. It feels like this throws is missing some additional checks.
The example I gave in my previous comment (https://github.com/quarkusio/quarkus/issues/39283#issuecomment-2016514410) does not use XA at all. Thus, I do not understand why createXaResource(...) should even be called.
Thanks @turing85 for the reproducer!
I'm sorry that I have to say these codes were working before but not right. In your case, if you want to do the db clean in source database and db write in target database in a transation which means do them all or nothing, it definely needs XA. I highly recommend to config all the datasource with ….jdbc.transactions = xa and also enable quarkus.transaction-manager.enable-recovery=true .Otherwise, it would be risk for the inconsistent data in two datasources.
The root case in agroal is to add LastResource interface in LocalXAResource. IIRC, narayana allows only one LastResource to enlist in a transaction @mmusgrov ? From the transaction perspective, this is right to make sure the non-XA resource could be involved in a XA transaction.
So I think the change in agroal makes sense but unfortuntaly, it breaks the applications which involve two or more non-XA resource in a XA transaction.
@maxandersen I think we definily need a document to describe these changes and impacts.
@zhfeng comment is spot on. Agroal added the LastResource to LocalXAResource to ensure that a single database is enlisted. Also, there was a change to throw an exception when the enlistment fails because that is a necessary step to manage the connection.
Unfortunately, the current exception does not describe the issue and is not very meaningful. I'll try to improve that.
The question still is: why was this change made? And should we really include it in a patch-level update of an LTS? Was something broken (as in "transactional properties were violated", not in "actual behaviour was different from documented behaviour") before?
@zhfeng can you clarify that last question ?
If we need to break Lts behaviour we need to have a reason.
Trying to grok what was actually happening in previous versions ? Sounds like commits was potentially happening outside a tx. For some users that might be tolerable (bad; I know) so is there a way for those users to get that old behaviour back or are you saying this is literally broken behaviour in all cases?
can you clarify that last question ?
Yeah, in previous version, if it involves multi non-XA datasources in a transaction, Narayana Transaction Manager CAN NOT guarantees that DO THEM ALL OR NOTHING. I think it could violate the Atomic property of Transaction. Also this does not work in the crash recovery secenario. And I understand that in some cases, users don't want such a Strong Consistent transaction behavior.
is there a way for those users to get that old behaviour back?
Yeah, I think there is a propery we can set in Narayana to allow multi LastResources like
arjPropertyManager.getCoreEnvironmentBean().setAllowMultipleLastResources(true);
but it should be set before creating the instance of TransactionManager. So I think we need to introduce a ConfigItem in quarkus-narayana-jta just like quarkus.transaction-manager.allow-multiple-last-resources=true. Do we have any plan to release 3.8.4 and I can try to add it?
@mmusgrov What do you think if we can introudce such a propery in quarkus-narayana-jta?
@zhfeng we could add that property but it is transactionally unsafe so we I doubt we'd add it.
@zhfeng we could add that property but it is transactionally unsafe so we I doubt we'd add it.
I am for adding this property. For quarkus 3.9.0 and onwards, the property can default to false, so users have to opt-in in order to use it. For 3.8.x, however, I think it should default to true to not break existing behaviour.
Even allowing a single one-phase aware resource to join an XA transaction containing two-phase aware resources is transactionally unsafe. Beyond that, allowing two such resources is asking for trouble and we will get many users asking us why the integrity of their data is compromised. I can anticipate that it will end badly and give Quarkus a poor reputation.
I can anticipate that it will end badly and give Quarkus a poor reputation.
Worse than a patch version of an LTS containing breaking changes...?
Adding it to the management api/property config implies that Quarkus will support users who fall foul of the consequences of using this behaviour, I would even go so far as saying they'd be better of disabling transactions and winging it instead since that would be better than having naive users thinking that transactions are giving them protection - we already give them the option of disabling recovery, which is a bit odd, and allowing multiple last resources would only compound the problem of allowing transactionally unsafe usage of the extension.
I'd be agreeable to telling them that quarkus supports system properties that they can use to enable this behaviour but not for adding it directly to the extension config, ie it would become a workaround for a known defect.
Finally as I mentioned above LRCO is unsafe but there is a safe alternative, called Commit Markable Resource, but that only allows a single 1-phase aware resource to join an XA transaction.
@maxandersen , @mmusgrov So... how do we proceed now? What is the process?
Can't I just tell you the system property and then then someone updates the docs?
Can't I just tell you the system property and then then someone updates the docs?
If I undestand the comment from @zhfeng correctly, this is not sufficient.
Can't I just tell you the system property and then then someone updates the docs?
If I undestand the comment from @zhfeng correctly, this is not sufficient.
There is no sufficient fix: LRCO is transactionally unsafe.
Can't I just tell you the system property and then then someone updates the docs?
If I undestand the comment from @zhfeng correctly, this is not sufficient.
There is no sufficient fix: LRCO is transactionally unsafe.
Since there is no fix or property that can be exposed on Quarkus side to mitigate the problem, I think that at least the reproducer from @jacopo-cavallarin should be used as a starting point to write a Quarkus blog post about this issue, in order to illustrate how to restructure the code and fix the problematic transactions.
The application should be using persistence units that are backed by XA datasources so I would argue that this is an education issue - the docs should highlight the requirement when adding multiple such units and they should indicate that even using a single non-XA datasource with XA datasources isn't recommended.