shardingsphere Transition from Jakarta EE 8 to Jakarta EE 9

Feature Request

For English only, other languages will not be accepted.

Please pay attention on issues you submitted, because we maybe need more details. If no response anymore and we cannot make decision by current information, we will close it.

Please answer these questions before submitting your issue. Thanks!

Is your feature request related to a problem?

No

Describe the feature you would like.

After Jakarta EE 9 has been released, The group id javax-*** is no longer maintained, it would be a good idea to migrate javax to jakarta namespace.

The following GAVs are mentioned in shardingsphere:

<dependency>
    <groupId>javax.transaction</groupId>
    <artifactId>jta</artifactId>
</dependency>
<dependency>
    <groupId>javax.annotation</groupId>
    <artifactId>javax.annotation-api</artifactId>
</dependency>
<dependency>
    <groupId>javax.xml.bind</groupId>
    <artifactId>jaxb-api</artifactId>
</dependency>
<dependency>
    <groupId>javax.activation</groupId>
    <artifactId>javax.activation-api</artifactId>
</dependency>

Here is the latest version until May 30th, 2023:

<dependency>
    <groupId>jakarta.transaction</groupId>
    <artifactId>jakarta.transaction-api</artifactId>
    <version>2.0.1</version>
</dependency>
<dependency>
    <groupId>jakarta.annotation</groupId>
    <artifactId>jakarta.annotation-api</artifactId>
    <version>2.1.1</version>
</dependency>
<dependency>
    <groupId>jakarta.xml.bind</groupId>
    <artifactId>jakarta.xml.bind-api</artifactId>
    <version>4.0.0</version>
</dependency>
<dependency>
    <groupId>jakarta.activation</groupId>
    <artifactId>jakarta.activation-api</artifactId>
    <version>2.1.2</version>
</dependency>

Refs: Jakarta EE Maven Coordinates Transition from Java EE to Jakarta EE Javax to Jakarta Namespace Ecosystem Progress

Jun 04 '23 07:06 laingke

~~@laingke Hey, this issue is blocked at https://github.com/atomikos/transactions-essentials/issues/171. If we need to remove all of the Atomikos-related unit tests because of a namespace change, this is an issue that has to be put on the mailing list.~~
~~I know you have noticed that the https://www.atomikos.com/Blog/TransactionsEssentials6dot0. But we consider that their Release Notes mention that they currently provide only limited support for JMS 2.0, which is a red flag.~~
update: This issue is only related to whether Bitronix related support needs to be retained.

Jun 04 '23 16:06 linghengqian

@FlyingZC Hi, I noticed that Bitronix's repository has been archived, refer to https://github.com/bitronix/btm and https://shardingsphere.apache.org/document/5.3.2/en/dev-manual/transaction/ .
Engineers at Software AG abandoned Bitronix. Should we remove Bitronix-related code for this issue? Refer to https://github.com/bitronix/btm/issues/107 .

Jun 05 '23 17:06 linghengqian

@linghengqian I personally agree with the removal, atomikos and naryana are enough, and bitronix has not been maintained for a long time.

Jun 07 '23 07:06 FlyingZC

@linghengqian I recently refactored the BTM project(with the latest dependies) https://github.com/laingke/btm , and an unofficial version may be released in the future. Is it possible for shardingsphere to integrate an unofficial artifact?

Jun 07 '23 10:06 laingke

@linghengqian I recently refactored the BTM project(with the latest dependies) https://github.com/laingke/btm , and an unofficial version may be released in the future. Is it possible for shardingsphere to integrate an unofficial artifact?

I think there is no problem, because Bitronix is only one of the implementations of a specific SPI in ShardingSphere. For other implementations, as long as the LICENSE is allowed by ASF, they can be included directly.

Jun 07 '23 12:06 linghengqian

@linghengqian I recently refactored the BTM project(with the latest dependies) https://github.com/laingke/btm , and an unofficial version may be released in the future. Is it possible for shardingsphere to integrate an unofficial artifact?

I think there is no problem, because Bitronix is only one of the implementations of a specific SPI in ShardingSphere. For other implementations, as long as the LICENSE is allowed by ASF, they can be included directly.

That is OK, since 3.0.0-SNAPSHOT, Bitronix licensed with APACHE LICENSE 2.0.

Jun 08 '23 04:06 laingke

@laingke Hi, I would like to inquire if I can remove the btm related code from ShardingSphere now to resolve the issue https://github.com/apache/shardingsphere/issues/26896 ?
It seems unlikely that laingke/btm will release its first version before the release of ShardingSphere 5.4.1.

Jul 11 '23 17:07 linghengqian

@laingke Hi, I would like to inquire if I can remove the btm related code from ShardingSphere now to resolve the issue https://github.com/apache/shardingsphere/issues/26896 ?

It seems unlikely that laingke/btm will release its first version before the release of ShardingSphere 5.4.1.

That is OK, my version still need 1 or 2 month to develop.

Jul 11 '23 23:07 laingke

@laingke @KouShenhai @xiaogang-github The migration to Jakarta EE 9 is already in https://github.com/apache/shardingsphere/pull/27976 .
Assuming you are using Spring Boot OSS 3/Quarkus 3. Jakarta EE 9.1/10 users can test this PR-related branch.

Aug 07 '23 22:08 linghengqian

On https://github.com/linghengqian/shardingsphere-v540-readwrite-splitting-test corresponding to commit https://github.com/linghengqian/shardingsphere-v540-readwrite-splitting-test/commit/a1203203db1d421452b8b465b58003224a936836 , I created some tests for Spring Boot OSS 3.1.2. Except for XA distributed transactions, ordinary functions can be used normally.
The required dependencies may not be obvious, maybe I should create a documentation PR to clarify up front.

        <dependency> 
             <groupId>org.apache.shardingsphere</groupId> 
             <artifactId>shardingsphere-jdbc-core</artifactId> 
             <version>5.4.0</version> 
         </dependency> 
         <dependency> 
             <groupId>org.yaml</groupId> 
             <artifactId>snakeyaml</artifactId> 
             <version>1.33</version> 
         </dependency> 
         <dependency> 
             <groupId>org.glassfish.jaxb</groupId> 
             <artifactId>jaxb-runtime</artifactId> 
             <version>2.3.8</version> 
         </dependency>

I'm still not sure if I should merge https://github.com/apache/shardingsphere/pull/27976 before Spring Boot OSS 2.7.x ends community support (2023-11-18). On the ShardingSphere side, issues from Spring Boot 2.x have increased recently.

Aug 28 '23 17:08 linghengqian

There hasn't been any activity on this issue recently, and in order to prioritize active issues, it will be marked as stale.

Sep 27 '23 20:09 github-actions[bot]

I test with quarkus-shardingsphere-jdbc and it fails to start the application

Caused by: java.lang.RuntimeException: java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException
	at io.quarkus.hibernate.orm.runtime.JPAConfig.startAll(JPAConfig.java:78)
	at io.quarkus.hibernate.orm.runtime.HibernateOrmRecorder.startAllPersistenceUnits(HibernateOrmRecorder.java:101)
	at io.quarkus.deployment.steps.HibernateOrmProcessor$startPersistenceUnits1868654632.deploy_0(Unknown Source)
	at io.quarkus.deployment.steps.HibernateOrmProcessor$startPersistenceUnits1868654632.deploy(Unknown Source)
	... 13 more
Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException
	at org.apache.shardingsphere.mode.repository.standalone.jdbc.JDBCRepository.init(JDBCRepository.java:59)
	at org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader.findService(TypedSPILoader.java:61)
	at org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader.getService(TypedSPILoader.java:110)
	at org.apache.shardingsphere.mode.manager.standalone.NewStandaloneContextManagerBuilder.build(NewStandaloneContextManagerBuilder.java:48)
	at org.apache.shardingsphere.driver.jdbc.core.datasource.ShardingSphereDataSource.createContextManager(ShardingSphereDataSource.java:78)
	at org.apache.shardingsphere.driver.jdbc.core.datasource.ShardingSphereDataSource.<init>(ShardingSphereDataSource.java:66)
	at org.apache.shardingsphere.driver.api.ShardingSphereDataSourceFactory.createDataSource(ShardingSphereDataSourceFactory.java:95)
	at org.apache.shardingsphere.driver.api.yaml.YamlShardingSphereDataSourceFactory.createDataSource(YamlShardingSphereDataSourceFactory.java:167)
	at org.apache.shardingsphere.driver.api.yaml.YamlShardingSphereDataSourceFactory.createDataSource(YamlShardingSphereDataSourceFactory.java:102)
	at org.apache.shardingsphere.driver.jdbc.core.driver.DriverDataSourceCache.createDataSource(DriverDataSourceCache.java:52)
	at org.apache.shardingsphere.driver.jdbc.core.driver.DriverDataSourceCache.lambda$get$0(DriverDataSourceCache.java:46)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
	at org.apache.shardingsphere.driver.jdbc.core.driver.DriverDataSourceCache.get(DriverDataSourceCache.java:46)
	at org.apache.shardingsphere.driver.ShardingSphereDriver.connect(ShardingSphereDriver.java:53)
	at io.agroal.pool.ConnectionFactory.createConnection(ConnectionFactory.java:226)
	at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:536)
	at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:517)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at io.agroal.pool.util.PriorityScheduledExecutor.beforeExecute(PriorityScheduledExecutor.java:75)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

Nov 17 '23 07:11 zhfeng

It seems that when quarkus build the application, it will exclude the old Java EE artifacts like javax.xml.bind:jaxb-api. Is there any plan to bump to Jakarata EE 10? It's a uncompatitable change from the package name javax. -> jakarta.

Nov 17 '23 07:11 zhfeng

Or is it possible to implement a memory only repostitory? like MemoryRepository which does not depend on any other external persistent storage.

Nov 17 '23 07:11 zhfeng

Or is it possible to implement a memory only repostitory? like MemoryRepository which does not depend on any other external persistent storage.

This may be a bit troublesome. There are many modules that need to be parsed through JAXB API. Please refer to https://github.com/apache/shardingsphere/pull/29013 .
Of course shardingsphere can be migrated to Jakarta EE 9 right away to fix this issue, it just requires me to fix #27976 .
Merging #27976 will mean that in the existing shardingsphere example module, the spring boot example module needs to use JDK17+ to run. We also need to declare that support for XA distributed transactions will be dropped on spring boot 2.x. Considering that Spring Boot 2.x will end OSS support this month, and shardingsphere will release 5.4.2 in December this year, should I merge #27976 on the current version milestone? Or merge #27976 on shardingsphere 5.4.3 or shardingsphere 5.5.0?

Nov 17 '23 11:11 linghengqian

According to the dependencies of org.glassfish.jaxb:jaxb-runtime:2.3.9, is it feasible to only do the following? The following dependencies are still in the Java EE 8 namespace, but the GAV of the corresponding dependencies is migrated to Eclipse.
javax.xml.bind:jaxb-api:2.3.1 to jakarta.xml.bind:jakarta.xml.bind-api:2.3.3
javax.activation:javax.activation-api:1.2.0 to jakarta.activation:jakarta.activation-api:1.2.2

Nov 17 '23 12:11 linghengqian

Well, these javax -> jakarta changes are uncompatitable and ~~I think it needs a majar bump version to shardingsphere like 6.0.0~~. If it only impact the internal parsing xml config file, it could not bump the majar verion.

Nov 17 '23 12:11 zhfeng

There is only shardingsphere-standalone-mode-repository-jdbc uses jaxb. All of others are test. And it seems it's only used to parse the configuration files like H2.xml internally. So it might not impact users?

Nov 17 '23 12:11 zhfeng

So @linghengqian are you working on this migration?

Nov 17 '23 12:11 zhfeng

Just FYI, narayana 7.0.0.Final has been moved to support Jakara EE 10 and change the license to Apache License V2.0.

Nov 17 '23 12:11 zhfeng

So @linghengqian are you working on this migration?

I will rebase #27976 in the next few days and ask for review.

Nov 17 '23 12:11 linghengqian

Or is it possible to implement a memory only repostitory? like MemoryRepository which does not depend on any other external persistent storage.

#27976 remains external controversy and #27976 is proposed to delay the merger by me.
I noticed the uselessness of the JAXB API while investigating #29209, and I'm now opening #29384.
#29384 removed all use of the JAXB API, retaining use only in test modules. I think this will provide an out-of-the-box experience for Spring Boot OSS 3/Quarkus 3/Micronaut Framework 4/Helidon 3 users.
XA distributed transactions are still not ready on Spring Boot OSS 3/Quarkus 3/Micronaut Framework 4/Helidon 3.

Dec 12 '23 17:12 linghengqian

There hasn't been any activity on this issue recently, and in order to prioritize active issues, it will be marked as stale.

Mar 01 '24 20:03 github-actions[bot]

For anyone following the migration movement of the javax.annotation.** packages in com.google.code.findbugs:jsr305, this package will logically move to https://github.com/jspecify/jspecify to follow Spring Boot's lead. See https://spring.io/blog/2024/10/01/from-spring-framework-6-2-to-7-0 and https://github.com/spring-projects/spring-framework/issues/28797 .
In fact, ShardingSphere does not have any transaction integration test for Spring Boot at all, and it is a bit doubtful whether #27976 has the impact of classpath conflict.
Having said that, if someone really want to test Narayana's distributed transactions, they actually need to start a k3s container through testcontainers-java, that is, start a local kubernetes cluster through docker engine, which may bring greater trouble. Because Narayana is a typical stateful component, its scalability needs to be achieved through WildFly's Kubernetes Operator. See https://github.com/wildfly/wildfly-operator/blob/1.1.0/doc/user-guide.adoc#scaledown-transaction-recovery .

Oct 02 '24 00:10 linghengqian

For anyone following the migration movement of the javax.annotation.** packages in com.google.code.findbugs:jsr305, this package will logically move to https://github.com/jspecify/jspecify to follow Spring Boot's lead. See https://spring.io/blog/2024/10/01/from-spring-framework-6-2-to-7-0 and Migrate to JSpecify annotations for nullability constraints spring-projects/spring-framework#28797 .

com.google.code.findbugs:jsr305 has been removed from Guava side. It is necessary to use JSpecify instead of com.google.code.findbugs:jsr305 on shardingsphere side. See https://github.com/google/guava/pull/7571 .

Feb 28 '25 14:02 linghengqian

I'm currently assessing ShardingSphere for my Spring Boot 3 application, but I see on the documentation and also here:

XA distributed transactions are still not ready on Spring Boot OSS 3/Quarkus 3/Micronaut Framework 4/Helidon 3.

Does it still hold true? What could we do to fix it or work around it?

I'd love to help. Thanks in advance!

Mar 27 '25 20:03 rickygodoy

I'm currently assessing ShardingSphere for my Spring Boot 3 application, but I see on the documentation and also here:

XA distributed transactions are still not ready on Spring Boot OSS 3/Quarkus 3/Micronaut Framework 4/Helidon 3.

Does it still hold true? What could we do to fix it or work around it?

I'd love to help. Thanks in advance!

@rickygodoy Considering #27976 is due to external disputes involving Spring Boot OSS 4 and Jackson 3, there is only one path left to move the current issue forward. That is to create a new Jakarta EE 9 namespace optional module for the following Maven modules via eclipse-transformer/transformer.

org.apache.shardingsphere:shardingsphere-transaction-core
org.apache.shardingsphere:shardingsphere-transaction-xa-atomikos
org.apache.shardingsphere:shardingsphere-transaction-xa-narayana

There are also some slack discussions not recorded on github.com trying to allow ShardingSphere to continue to provide XA integration support for Seata's XA mode and Bitronix Transaction Manager (BTM) through eclipse-transformer/transformer.
The discussed changes around JSpecify did break the compatibility of the new Guava version with ShardingSphere's master branch. Also see https://github.com/immutables/immutables/issues/1535 .
A possible example is https://github.com/jbosstm/narayana/pull/1824 . But I personally don't have time to write and test a similar PR.

Mar 28 '25 13:03 linghengqian

Considering the underlying premise of investigating https://github.com/apache/shardingsphere/issues/34973, most of the third-party dependencies of the XA implementation of Java EE 8 are no longer updated. Of course, there are only 4 projects that actually implement XA. In the current issue, I tend to bypass the existing modules and directly create several new modules, although this will introduce a lot of duplicate code. Starting from atomikos is the easiest, because the new version of Narayana requires JDK11.

Jun 25 '25 03:06 linghengqian

Sorry @linghengqian - I lost this context for a long time. What are you meaning that creating sererval new modules? they are the shardingsphere modules?

Jun 25 '25 03:06 zhfeng

Sorry @linghengqian - I lost this context for a long time. What are you meaning that creating sererval new modules? they are the shardingsphere modules?

There is no doubt that the new Maven module will be created in the master branch of the current project. I expect that I will propose a PR in the 5.5.3 milestone. Of course, if someone comes up with a better idea or PR, I can also help review it.

Jun 25 '25 04:06 linghengqian