2.0: Jakarta EE 9+ migration

[bhagyashriSawkar]

Updated a few files with changes required for upgrade to Jakarta EE 9+. The jar is getting generated but test cases are not getting executed, will work on that.

Please let me know if there is something else that also needs to be checked. Also I did override a few required abstract method with empty/default value, would appreciate if you take a look at those.

Thanks.

[chadlwilson]

This would be a breaking change probably requiring a 1.3 or 2.0 release.

Right now 1.2.x off master is not very stable (needs #262 at the very least to get fixes that were already there on 1.1.x but never merged into master), and master still lacks validated testing with modern Rails versions for which we probably need help with to get stable.

If you know Rails and jruby-rack better than me (especially the session stuff) I could do with some help trying to get the tests working with modern Rails and JRuby versions. I'll probably create a separate draft PR for that shortly.

[chadlwilson]

The branch/PR at #262 does get the tests/specs all running/passing reliably however :-)

[kares]

:+1: looks like a good start but the PR feels incomplete.

my idea about supporting jakarta/javax.servlet at the same time would have been converging towards moving all .rb bits that script the javax.servlet API into .java and than potentially use some kind of a weaving approach or the "migration" tool than Tomcat has to release 2 artifacts from the same build... :shrug:

[headius]

Exciting! This will obviously need to be rebased on master due to recent updates there, but I'm glad you've attempted this work.

This would be a breaking change probably requiring a 1.3 or 2.0 release.

use some kind of a weaving approach or the "migration" tool than Tomcat has

I would tend to lean toward a hard version move as a 2.0 release myself.

The old javax.servlet users have to be dwindling pretty fast these days. If we continued to maintain a 1.x line for a while that supports javax.servlet and provided 2.x for jakarta, we'd avoid any complex build situation and everyone would still be able to keep working with what they have. We might start dropping some deprecation warnings into the 1.x line at some point.

@kares Are you opposed to this clean break, branching off 1.2.x and releasing a 2.x that uses only jakarta packages going forward?

[kares]

@kares Are you opposed to this clean break, branching off 1.2.x and releasing a 2.x that uses only jakarta packages going forward?

Not at all, just shared my personal opinions, think some folks might still be stuck with older versions here and there due "legacy" (enterprise) reasons. Thus was thinking about ways to support both wout having extra work (such as backports).

I am very happy to see any effort such as this one (a full API rewrite) to keep the jruby-rack project going.

Regarding this PR my only concern is that some methods feel like stubs and should be revisited implementation wise according to the servlet specification.

[chadlwilson]

FWIW for GoCD i'm one of those stuck on javax namespace at the moment due to needing breaking Jetty 12.x upgrades + multiple Spring FW version upgrades which in turn need major Hibernate and Spring Security upgrades - and really only having myself to do it. 😅 But accept that is probably not most people's situation and being on all this EOL stuff is bad for many other reasons.

My main concern about this if this makes it harder to get to a supported Rails version (i.e if jruby-rack code change is needed to fix Rails 7 compat and no backport is possible) as currently I believe an issue with jruby-rack is preventing me getting to Rails 7+ and 6.1 is also EOL now :(

[kares]

My main concern about this if this makes it harder to get to a supported Rails version

I believe your concern, despite having an "unusual" app, is one shared by many legacy users -> simply hard enough for them to upgrade to a recent JRuby/Rails version while on top of it having to worry about a server upgrade as well.

believe an issue with jruby-rack is preventing me getting to Rails 7+ and 6.2 is also EOL now :(

Maybe you can open a discussion about what is your specific use-case and why you can not upgrade.

[chadlwilson]

@kares about all I know is still at https://github.com/jruby/jruby-rack/issues/244

I've subsequently seen similar looking session errors with the appraisals on Rails 6.x but yeah, separate to this issue so should discuss elsewhere.

[headius]

Another option would be a new named gem that is jakarta-rack or something, but there may be transitive dependency issues getting RubyGems/Bundler to pick that up.

[bhagyashriSawkar]

Thanks for all the comments. The branch is now synced with the latest master code.

Regarding this PR my only concern is that some methods feel like stubs and should be revisited implementation wise according to the servlet specification.

I'm not an expert in java, so I don't know what the implementation should be. In most places I've just updated javax with jakarta and added those stubs as per the errors I've encountered. Anyone wanting to update those methods is welcome.

[chadlwilson]

We're still some way away from being able to merge/release something like this given we haven't yet released 1.3.x and this would be another breaking change, however I have merged from master and fixed some quick win test problems with the PR.

~~Right now the remaining (unit) test failures are missing class name jakarta.servlet.http.HttpUtils which is due to reliance on a deprecated utility from the old Servlet APIs which has finally been removed in 5.x onwards.~~ After that I suspect there may be some other issues to resolve with the functional tests with Rails.

[chadlwilson]

Temporarily propagated the removed javax.servlet.HttpUtils#parseQueryString into jruby-rack's code and at least the tests are all passing now.

To get this over the line, we probably mainly need to

• be somewhat comfortable 1.2.4 is stable in various folks' setups so we don't have to actively maintain 3 branches.
• get 1.3.0 out (pretty close I think)
• decide what 2.0.0 looks like, other than this change

Ideally we'd also have some stronger Rails regression tests reinstated to bring more confidence in these bigger releases, but that's probably a nice-to-have.

[headius]

Is there a reason to require Java 17 minimum? See my comment in #234, but basically I know of at least one large jruby-rack user that still has to support Java 8 for some time.

[ajuckel]

For compatibility reasons, I'd argue for sticking tracking the underlying Jakarta EE compat. I can't find a quick compatibility matrix online to reference, but I'm pretty sure that means:

• Jakarta EE 9.0 -> Java 8
• Jakarta EE 9.1-10 -> Java 11
• Jakarta EE 11 -> Java 17

This PR says Jakarta EE 9+, but looking at the pom.xml, it seems that we're targeting the Jakarta EE 11 packages. If that's the case, if this is meant to jump directly to Jakarta EE 11, then I believe Java 17 is required.

[chadlwilson]

Require, no, but because the library is tested with Spring framework they made a decision to require Java 17 to get JEE 9 support thus it cant be tested on lower Java versions anyway. It's not deliberately targeting JEE 11 packages.

(As a side note, the Jakarta namespace move is one of the dumbest,.most painful wastes of time in the Java ecosystem for a decade)

[ajuckel]

@headius For your "large jruby-rack user that still has to support Java 8 for some time", are they large enough to warrant an attempt at creating a separate Java 8/Jakarta EE 9/Rack ?? version? Or are their needs met with the current release?

[ajuckel]

I did some more poking locally, and am caught up with what @chadlwilson was saying. Even if I lower the packages to their EE 9 variants and update the main code to restore Java 8 compatibility, the spring-test dependency is problematic. Spring Test 5.3.39 is using the old javax packages, and 6.0.23 is compiled with Java 17.

There's a non-zero chance that I might poke further at getting the testing side of EE 9+SE 8 resolved but I wouldn't hold your breath. I'm content to agree it would be a non-trivial effort to support EE 9 on Java 8 in jruby-rack.

If I wanted to be more immediately useful, where would folks recommend I direct some effort? I see this comment from a few days ago, but I'm not sure where the 1.2.4 or 1.3.0 effort is occurring or what the status is.

[chadlwilson]

Thanks for your interest @ajuckel .

To begin with, I'll dig back and see why I bumped the servlet API further than 5.0. Possibly I thought it didn't do any 'harm' to support the newer API definitions as nothing had been removed, only added.

But the spring test usage is more restrictive. Perhaps we can relook at that. I haven't really dug into its importance outside using it for mock servlet requests etc, but it might be possible to go back to what was done before and copy their mock code into the library, even though I was trying to avoid that maintenance overhead/cruft.

[chadlwilson]

Right now

• 1.2.4 is now released and should be stable for almost all users previously on 1.1.x (up to and incl. Rails 7.2)
• @kares is stabilizing logger changes on 1.3.0
• @kares reintroduced proper rails integration tests for 7.2 on 1.3.x/master. I'll see if I can extend to other rails versions. I also had a local branch attempting this earlier.
• ~~we might need to try and address this spring test stuff on 1.3.x (master) as I believe it also caused a Java 11 minimum change there too due to Spring 5.3.x which I hear we'd like to drop back to Java 8~~ Edit: Not an issue, see https://github.com/jruby/jruby-rack/pull/315

[kares]

Interesting, have considered Spring to be conservative - that feels like quite the jump they made with JakartaEE 9.0.

Spring was brought in simply because of having the ServletContext mock implementation, that's pretty much the main thing needed (for tests), in that sense an alternate or in-house MockServletContext implementation should be acceptable.

On the other hand, if we bump to jruby-rack it's perfectly reasonable to bump the minimum Java as well, to an up-to-date version. If someone migrates from javax.servler -> jakarta they might as well bump Java from 8. :shrug:

[kares]

and before I forget due jruby-rack being both a Java library as well as a Ruby gem,

assuming the 1.4.0 release is not javax.servlet retrofitted but "only" a full jakarta.servlet rewrite (current PR state) than I believe the jruby-rack version number should be 2.0 since it's not server "forward" compatible like the 1.x line...

[chadlwilson]

I've pushed an experiment here to copy the Spring Test mock objects from 5.3 back across, move them to jakarta.servlet and see if I can get things working easily enough to retain Java 8 compatibility.

I couldn't find an alternate Mock implementation that is maintained and looks decent. :-(

[chadlwilson]

I suppose it doesn't matter too much which order we do things, but probably wait since we're probably not too far away on 1.3.0?

For this PR, I'll

• [x] do another sweep for unused mock code
• [x] fix the import ordering churn
• [x] find dead servlet < 5.0 compat code (which definitely won't work after this PR since servlet 5.0 is obviously forward only due to the namespace change, whereas earlier 1.2.x and 1.3.x changes might still work with Servlet 2.5 since they didn't tend to remove anything from the impls.)

[headius]

jruby-rack version number should be 2.0

Came here to say this too. 👍

I think the leap forward should be a major version change so we can go on maintaining the 1.x series with Java 8 and earlier javax.servlet API support. Then we don't have to discuss how to support both cases in one branch.

[chadlwilson]

jruby-rack version number should be 2.0

Came here to say this too. 👍

I think the leap forward should be a major version change so we can go on maintaining the 1.x series with Java 8 and earlier javax.servlet API support. Then we don't have to discuss how to support both cases in one branch.

Yup, already changed to 2.0.0, and also re-introduced Java 8 compatibility for both 1.3.x (master) and 2.0.x (this PR) - thanks for the input!

I've tried to summarise the current known + planned compatibility in the README to the best of my knowledge, which might help review the plan implied by current discussions on this PR and elsewhere.

https://github.com/jruby/jruby-rack/blob/master/README.md

The big open question is how/when to introduce Rack 3.x support. I am somewhat fearful about how big a change this might be and whether it might also need yet another jruby-rack major version change to avoid having to support alongside 2.2.

Kind of want to avoid ending up with an impossible matrix of things that'd need to be upgraded at the same time, but nor lots of branches to maintain in parallel for an extended period :-(

[headius]

might also need yet another jruby-rack major version change

I guess more interesting to me than the level of effort required is the compatibility impact to existing applications. Can old apps "just work" on Rack 3 if they have up to now been based on earlier versions? What's the real-world impact if we go ahead and do Rack 3 at the same time as Jakarta?

[chadlwilson]

Yeah, fair question. I'm no expert, just a wee bit burned by the level of coupling between jruby-rack and rack internals when trying to debug various session issues in later rails versions and having to shuttle between the 3 codebases to figure out why something that used to work no longer does 😅

Given rails itself has been able to support both rack 2.2 and rack 3.0 side-by-side for some time now, it presumably isn't impossible, and aside from splitting rack-session, it doesn't look too bad based on https://github.com/rack/rack/blob/main/UPGRADE-GUIDE.md

Anyway, sorry for the partially OT comment - can move discussion to https://github.com/jruby/jruby-rack/issues/304 when we get further into it.