unit icon indicating copy to clipboard operation
unit copied to clipboard

Published 20 hours ago • verified publisher icon nginx

Java: SpringBoot3 - Unable to load Context

Open tippexs opened this issue 2 years ago • 2 comments

On Unit 1.29.1 - Java17 - Docker eclipse-temurin:17-jdk-jammy

It is not possible to load a SpringBoot application version >=3.0. Unit did not crash. It simply isn't loading the Spring Application context. The log initially shows:

2023/05/06 01:34:53 [info] 1#1 unit 1.30.0 started
2023/05/06 01:34:53 [info] 76#76 discovery started
2023/05/06 01:34:53 [notice] 76#76 module: java 17.0.7 "/usr/lib/unit/modules/java.unit.so"
2023/05/06 01:34:53 [info] 1#1 controller started
2023/05/06 01:34:53 [notice] 1#1 process 76 exited with code 0
2023/05/06 01:34:53 [info] 78#78 router started
2023/05/06 01:34:53 [info] 78#78 OpenSSL 3.0.2 15 Mar 2022, 30000020
2023/05/06 01:34:53 [info] 79#79 "java" prototype started
2023/05/06 01:34:53 [info] 80#80 "java" application started

Sending request to the HelloWorldController which should answer with a nice Hello from Spring on Unit! is displaying a directory index like

t.stark@C02FN35FMD6R ~/w/u/p/docker (master)> curl -v localhost:8090
*   Trying 127.0.0.1:8090...
* Connected to localhost (127.0.0.1) port 8090 (#0)
> GET / HTTP/1.1
> Host: localhost:8090
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Last-Modified: Sat, 06 May 2023 01:34:53 GMT
< Server: Unit/1.30.0
< Date: Sat, 06 May 2023 02:59:28 GMT
< Transfer-Encoding: chunked
<
<a href="WEB-INF">WEB-INF</a><br>
<a href="org">org</a><br>
<a href="META-INF">META-INF</a><br>
* Connection #0 to host localhost left intact

Starting unitd in debug mode shows the reason:

2023/05/06 02:45:12.831 [debug] 0#573 [unit] Context.wsSession.test: Endpoint instance registration failed
2023/05/06 02:45:13.379 [debug] 0#573 [unit] Context.loadInitializer: initializer: nginx.unit.websocket.server.WsSci
2023/05/06 02:45:13.384 [debug] 0#573 [unit] Context.loadInitializer: find handles: javax.websocket.server.ServerEndpoint
2023/05/06 02:45:13.385 [debug] 0#573 [unit] Context.loadInitializer: find handles: javax.websocket.server.ServerApplicationConfig
2023/05/06 02:45:13.385 [debug] 0#573 [unit] Context.loadInitializer: find handles: javax.websocket.Endpoint
2023/05/06 02:45:13.385 [debug] 0#573 [unit] Context.loadInitializer: no handles implementations
2023/05/06 02:45:13.386 [debug] 0#573 [unit] Context.load initializer(s)
2023/05/06 02:45:13.424 [debug] 0#573 [unit] Context.loadInitializer: initializer: org.apache.jasper.servlet.JasperInitializer
2023/05/06 02:45:13.424 [debug] 0#573 [unit] Context.loadInitializer: no HandlesTypes annotation
2023/05/06 02:45:13.428 [debug] 0#573 [unit] Context.URLPattern: '*.jsp' is a suffix pattern
2023/05/06 02:45:13.429 [debug] 0#573 [unit] Context.URLPattern: '*.jspx' is a suffix pattern

As Spring >= requires Tomcat10 with Servlet 5 it is not longer compatible with Units Servlet Implementation. Looking at the Changes from Apache Tomcat 8 to 10 shows that they moved from javax to jakarta.

It would be necessary to implement a new Servlet Spec to be able to load the Context of the newer generation of Spring Boot applications for example.

Please consider this issue as documentation only - we are not in a hurry to fix this for now. I just wanted to find a place to document my findings.

An application to reproduce is attached.

spring3-0.0.1-SNAPSHOT.war.zip

The unit configuration

{
  "listeners": {
    "*:80": {
      "pass": "applications/java"
    }
  },

  "applications": {
    "java": {
      "type": "java",
      "webapp": "spring3-0.0.1-SNAPSHOT.war",
      "working_directory": "/var/www/"
    }
  }
}

tippexs avatar May 06 '23 03:05 tippexs

Hi there,

I've done some tests and investigations on that topic. I'm now able to start a basic spring boot 3.3.4 application over java 17 with nginx unit.

Changing all references from javax.servlet to Jakarta.servlet is not the major problem, even if some deprecated methods does not exist anymore. ClassGraph library encounter some problem to list classes in the extracted war because of java module system. I had to create a module-info.java that references dependencies in unit java jar. My spring java application also now have to care about module requirements.

Some minor changes were also needed in makefile / autoconf to split classpath and modulepath, more jars needed at compile time because of "requires" declaration on unit jar.

The major issue I encountered is that sometimes, when I call PUT rest endpoint to load my application war, the process to load the application freeze, sometimes before extracting the war, sometimes just after extracting it... no clue why.

I have to kill the "prototype" and "application" process, kill the main unit process, rm the config.json in state dir, rm all references to module-info classes and extracted webapp directory in $TMPDIR, then try again ...

Changes made : https://github.com/nginx/unit/compare/master...gdufrene:unit:fix/spring3

Many TODOs to resolve around WebSocket and Multipart

best regards.

gdufrene avatar Oct 19 '24 12:10 gdufrene

@gdufrene that branch looks fantastic; are you planning to continue working on it and propose a pull request here? We'd gladly review it or try to assist in debugging.

callahad avatar Oct 21 '24 08:10 callahad

Hello @callahad , thanks for your feedback.

Sure I can help around this issue and continue working on it, time to time.

There is some questions around nginx-unit support for java EE vs jakarta EE. Do you want to support both ? Or only jakarta EE ? And over that API-bound question, the Java supported version will apply : java 8 / 11 / 17 ?

Maybe 2 different module will be a better fit :

  • java EE with java 8
  • jakarta EE with java 17

I think it will fit support for major frameworks such as spring and sprint-boot. Spring boot 2.7 requires java 8 at minimum and is based on java EE spec. Spring boot >3 requires java 17 at minimum and is based on jakarta spec.

So, on the other hand, spring boot 2.7 could be considered as a "legacy" stack, but still supported with commercial support. Just providing an open-source support with jakarta EE / java 17 could be an option.

Let me know what you think.

A last few words about the "native" approach. More and more enterprises deliver their applications as container. Start time and memory consumption is a big topic on java with that kind of application delivery. Native compilation seems to be a way to cope with that problem (with compromise on throughput and debug complexity) In which way nginx-unit can integrate that kind of thing ?

gdufrene avatar Oct 27 '24 11:10 gdufrene

@gdufrene Right now, the Java language support in Unit is largely community driven; the folks we have working on Unit itself aren't especially familiar with Java in the wild.

My gut is that supporting Spring Boot 3 (and thus Jakarta EE / Java 17) is the right way to go for an open source project. However, we suspect that there are users of Unit relying on Java EE...

We'd welcome your recommendations for what would be most useful as a consumer of the Java language support.

callahad avatar Oct 28 '24 09:10 callahad

Hi @gdufrene

The normal way we handle support for different versions of language stack in the language modules is via a bunch of #ifdefs... see src/nxt_php_sapi.c and src/python for examples.

I see from your current changes you are really just changing the code under src/java/nginx (as well as the java build infra), the C code under src/java seems isolated from these changes.

My gut feeling is while #ifdefs may not be the way to go here, also I feel an entirely separate module may be overkill (at the source level).

It seems like we really need to re-factor the current java module (and this is entirely off the top of my head after 5 minutes looking at it) the we should split the module up into three components

  1. Core. The C code (src/java) at the least
  2. javax target, the current java code. Perhaps this should all move to under src/java/javax
  3. jakarta target, say under src/java/jakarta

Then when configuring the java module you could specify a --target= (or whatever name) for either javax or jakarta... You would be able to do this for each target to get two actual modules if you needed to support both... Probably some details to figure out like module naming, e.g java.javax.unit.so/java.jakarta.unit.so and then types of java-javax/java-jakarta in the unit config, something like that...

How does that sound?

ac000 avatar Oct 29 '24 01:10 ac000

Hello,

3 components as you describe seems me a good idea. Your approach with current codebase in a javax package, my branch in a Jakarta package will result in a big code duplication but can be a first easy move to build a "common" codebase between both packages.

Right now, It's not really clear for me how to use autoconf to specify a target (javax/jakarta) but I will take a look.

About the module naming, including java version could be an option, as described in how-to / configuring modules / java documentation page.

I will try to work on it that way, thanks for your comments.

gdufrene avatar Oct 30 '24 18:10 gdufrene

Hello,

3 components as you describe seems me a good idea. Your approach with current codebase in a javax package, my branch in a Jakarta package will result in a big code duplication but can be a first easy move to build a "common" codebase between both packages.

Right, once we have the two things working we can see what common code can be factored out. We'll already have the C code under src/java as common code...

Right now, It's not really clear for me how to use autoconf to specify a target (javax/jakarta) but I will take a look.

Not autoconf as such, but a home grown build system.

auto/modules/java would grow a new --target option, but looking at this, it's certainly quite a bit more involved than our other language modules, with there being several java related build scripts... hopefully most of it can be shared...

About the module naming, including java version could be an option, as described in how-to / configuring modules / java documentation page.

My current thinking is that we would have javax.unit.so or jakarta.unit.so as the new module basenames with java.javax.unit.so the default, if you specify --target=jakarta you'd get java.jakarta.unit.so, however nothings ever that simple and currently unit expects modules to be named MODULENAME.unit.so, so it wouldn't find java.{jakarta,javax}.unit.so... hmm...

I will try to work on it that way, thanks for your comments.

I'm here to give any assistance I can...

ac000 avatar Oct 31 '24 04:10 ac000

Hello @ac000

I updated my branch with a "work in progress" solution.

When working on it, I realize that most of dependencies between C code and java are method and class references. No direct impacts on the target specifications (Javax / Jakarta) for most of the code.

The runtime impacts are embedded in the compiled java code packaged as "nginx-unit-jsc-java-$version.jar" (most of it).

Right now, I created two directories inside src/java :

  • jsc-jakarta
  • jsc-javax

Both are defined as maven project to help with dependencies, versions, compilation. I know it could add a requirement for building the module, but maven or its "wrapper" mvnw are popular for java developers. They won't be lost and can contribute the project from their favorite IDE.

I try to update auto/modules/java to handle a "target" argument (default to javax) to build the module with the correct dependencies. Getting dependencies with maven is easy, export them in a directory then add their names in the corresponding include file for java module.

A lot more things to do, but it seems me quite promising. I need to figure how to handle the websocket support properly for jakarta (and some other feature such as mime types, multipart ... ). If possible I'd like to reuse functions available in nginx.

With my current environment I still often get some trouble in loading an application config with the java module. Processes get locked during loading the application : logs freeze and config endpoint do not respond anymore. It seems to be a sync threading problem between processes, maybe around kevent ? It happens approximately 8 times over 10.

gdufrene avatar Nov 01 '24 20:11 gdufrene

HI,

Sounds like you're making good progress!

Both are defined as maven project to help with dependencies, versions, compilation. I know it could add a requirement for building the module, but maven or its "wrapper" mvnw are popular for java developers. They won't be lost and can contribute the project from their favorite IDE.

If you think this is the best way to go, I'm happy to defer to you on that.

Will we still be able to target different version of java? As I know we do build various versions of the java language module for different versions of java...

With my current environment I still often get some trouble in loading an application config with the java module.

Hmm, this is not an issue I've been aware of and we do have pytests covering java and I haven't seen an issue with them...

It seems to be a sync threading problem between processes, maybe around kevent ?

Are you using FreeBSD?

ac000 avatar Nov 05 '24 15:11 ac000

Will we still be able to target different version of java? As I know we do build various versions of the java language module for different versions of java...

Yes, you can target a specified version of jvm with already defined configure module option "-home" ( but yes, maven can use something else based on JAVA_HOME environment variable ... maybe I should force usage of the specified java based on that argument, good point )

Are you using FreeBSD?

I'm using a mac with Homebrew to get building tools

Maybe I can try in a docker container or try with an other kind of module to check if I have the same issue occurs.

gdufrene avatar Nov 06 '24 20:11 gdufrene

Will we still be able to target different version of java? As I know we do build various versions of the java language module for different versions of java...

Yes, you can target a specified version of jvm with already defined configure module option "-home" ( but yes, maven can use something else based on JAVA_HOME environment variable ... maybe I should force usage of the specified java based on that argument, good point )

OK, cool. I haven't actually looked at how we currently do it...

Are you using FreeBSD?

I'm using a mac with Homebrew to get building tools

Hmm..

Maybe I can try in a docker container or try with an other kind of module to check if I have the same issue occurs.

Actually, I have my F5 intel mac I could try on...

If you install pytest, then if you've built unit and the java language module, no need to install them, you can then do pytest test/test_java_*

ac000 avatar Nov 06 '24 22:11 ac000

On my current branch with jakarta / java17 all tests fails. (probably because of the same reasons loading application config I pointed)

I switch back on master branch and try to run a first test with java17 runtime on "javax.servlet" expected code. It fails all tests except first one (config). Note that running a java8 application on a java17 runtime should work, even in a web application context according to me.

With an openjdk 8 for my system (Zulu for its support of arm64), it works and all tests green (except skipped one for normal reasons). And a simple Spring boot 2.7.x web application loads nicely.

Maybe I should first focus on why there is some differences between runtimes and find a way to work with java 17 on javax.servlet spec. My guesses :

  • The class discovery with "classgraph" library (as already spotted), required for every Annotation based servlet, and listeners in JSR
  • More Security reason around "privileged" code execution
  • Change of the default Garbage collector on the java process ?

when I compile java module with java 17, this warning triggered :

src/java/nginx/unit/websocket/AsyncChannelGroupUtil.java:122: warning: [removal] AccessController in java.security has been deprecated and marked for removal
            return AccessController.doPrivileged(new NewThreadPrivilegedAction(r));

that's why I suspect a security thing, even if the code seems to be specific to the websocket support. Some security changes on the jvm could be a reason.

more investigations to follow. I keep you informed.

gdufrene avatar Nov 16 '24 12:11 gdufrene

Well ... I'm back to a random deadlock hypothesis, due to the javaVM version ?

When re-configure application config, the process freeze randomly at least at 2 places :

  • when creating java VM
  • when starting web context And sometimes it goes well.

I'm not really confident on my skills to find and fix that kind of problem :(

added some logs in nxt_java.c to track nxt_java_start init steps

logs when freeze at createVM

2024/11/17 15:58:14.259 [info] 2715#143258 "bootdemo" application started
2024/11/17 15:58:14.259 [debug] 2715#143258 pthread_mutex_lock(14D7040F0) enter
2024/11/17 15:58:14.259 [debug] 2715#143258 process 2715 added
2024/11/17 15:58:14.259 [debug] 2715#143258 pthread_mutex_unlock(14D7040F0) exit
2024/11/17 15:58:14.259 [debug] 2714#143257 kevent(3): 1
2024/11/17 15:58:14.259 [debug] 2714#143257 kevent: id:12 ft:-1 fl:0025 ff:0 d:16 ud:14D605E00
2024/11/17 15:58:14.259 [debug] 2714#143257 timer expire minimum: 106188015:19788015
2024/11/17 15:58:14.259 [debug] 2714#143257 work queue: fast
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(0): 14D709B70
2024/11/17 15:58:14.259 [debug] 2714#143257 recvmsg(12, 2, 0): 16
2024/11/17 15:58:14.259 [debug] 2714#143257 port 12: message type:18 fds:-1,-1
2024/11/17 15:58:14.259 [debug] 2714#143257 app process 2715 is created
2024/11/17 15:58:14.259 [debug] 2714#143257 recvmsg(12, 2, 0): -1
2024/11/17 15:58:14.259 [debug] 2714#143257 recvmsg(12) not ready
2024/11/17 15:58:14.259 [debug] 2714#143257 kevent(3) changes:0 timeout:-1
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(40): 14D70AE80
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(84): 14D709C00
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(85): 14D70AD80
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(87): 14D70AEB0
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(85): 14D70AF10
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(76): 14D70ACB0
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(88): 14D70AF70
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(83): 14D70AFD0
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(78): 14D70B030
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(79): 14D70B080
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(78): 14D70B0D0
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(81): 14D70B120
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(76): 14D70B180
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(75): 14D70B1D0
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(81): 14D70B220
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(76): 14D70B280
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(68): 14D70B2D0
2024/11/17 15:58:14.259 [debug] 2715#143258 malloc(83): 14D70B320
2024/11/17 15:58:14.259 [debug] 0#143258 [unit] Create JavaVM

logs when freezing a little bit later avec starting context :

2024/11/17 15:37:44.604 [info] 2339#135224 "bootdemo" application started
2024/11/17 15:37:44.604 [debug] 2339#135224 pthread_mutex_lock(153F040F0) enter
2024/11/17 15:37:44.604 [debug] 2339#135224 process 2339 added
2024/11/17 15:37:44.604 [debug] 2339#135224 pthread_mutex_unlock(153F040F0) exit
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(0): 153F059E0
2024/11/17 15:37:44.604 [debug] 2338#135223 kevent(3): 1
2024/11/17 15:37:44.604 [debug] 2338#135223 kevent: id:12 ft:-1 fl:0025 ff:0 d:16 ud:153E063E0
2024/11/17 15:37:44.604 [debug] 2338#135223 timer expire minimum: 104958360:18558360
2024/11/17 15:37:44.604 [debug] 2338#135223 work queue: fast
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(40): 153F0AE80
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(84): 153F0AD80
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(85): 153F09C00
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(87): 153F0AEC0
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(85): 153F0AF20
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(76): 153F0AF80
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(88): 153F0AFD0
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(83): 153F0B030
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(78): 153F0B090
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(79): 153F0B0E0
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(78): 153F0B130
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(81): 153F0B180
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(76): 153F0B1E0
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(75): 153F0B230
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(81): 153F0B280
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(76): 153F0B2E0
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(68): 153F0B330
2024/11/17 15:37:44.604 [debug] 2339#135224 malloc(83): 153F0B380
2024/11/17 15:37:44.604 [debug] 2338#135223 recvmsg(12, 2, 0): 16
2024/11/17 15:37:44.604 [debug] 2338#135223 port 12: message type:18 fds:-1,-1
2024/11/17 15:37:44.604 [debug] 2338#135223 app process 2339 is created
2024/11/17 15:37:44.604 [debug] 2338#135223 recvmsg(12, 2, 0): -1
2024/11/17 15:37:44.604 [debug] 2338#135223 recvmsg(12) not ready
2024/11/17 15:37:44.604 [debug] 2338#135223 kevent(3) changes:0 timeout:-1
2024/11/17 15:37:44.604 [debug] 0#135224 [unit] Create JavaVM
2024/11/17 15:37:44.630 [debug] 0#135224 [unit] Create JavaVM result: 0
2024/11/17 15:37:44.630 [debug] 0#135224 [unit] init java Thread
2024/11/17 15:37:44.630 [debug] 0#135224 [unit] java Thread result: 0
2024/11/17 15:37:44.630 [debug] 0#135224 [unit] init classLoader
2024/11/17 15:37:44.630 [debug] 0#135224 [unit] new classLoader
2024/11/17 15:37:44.631 [debug] 0#135224 [unit] nxt_java_initContext
2024/11/17 15:37:44.652 [debug] 0#135224 [unit] registered Context methods: 0
2024/11/17 15:37:44.652 [debug] 0#135224 [unit] nxt_java_initRequest
2024/11/17 15:37:44.697 [debug] 0#135224 [unit] registered Request methods: 0
2024/11/17 15:37:44.698 [debug] 0#135224 [unit] registered HeadersEnumeration methods: 0
2024/11/17 15:37:44.698 [debug] 0#135224 [unit] registered HeaderNamesEnumeration methods: 0
2024/11/17 15:37:44.698 [debug] 0#135224 [unit] nxt_java_initResponse
2024/11/17 15:37:44.705 [debug] 0#135224 [unit] registered Response methods: 0
2024/11/17 15:37:44.705 [debug] 0#135224 [unit] nxt_java_initInputStream
2024/11/17 15:37:44.705 [debug] 0#135224 [unit] registered InputStream methods: 0
2024/11/17 15:37:44.706 [debug] 0#135224 [unit] nxt_java_initOutputStream
2024/11/17 15:37:44.706 [debug] 0#135224 [unit] registered OutputStream methods: 0
2024/11/17 15:37:44.706 [debug] 0#135224 [unit] nxt_java_jni_init
2024/11/17 15:37:44.706 [debug] 0#135224 [unit] nxt_java_jni_init: 0
2024/11/17 15:37:44.706 [debug] 0#135224 [unit] nxt_java_startContext...

And when it goes well :

2024/11/17 15:44:59.372 [info] 2621#138321 "bootdemo" application started
2024/11/17 15:44:59.372 [debug] 2620#138320 app process 2621 is created
2024/11/17 15:44:59.372 [debug] 2621#138321 pthread_mutex_lock(142E04AE0) enter
2024/11/17 15:44:59.372 [debug] 2621#138321 process 2621 added
2024/11/17 15:44:59.372 [debug] 2621#138321 pthread_mutex_unlock(142E04AE0) exit
2024/11/17 15:44:59.372 [debug] 2615#138312 free(142E05010)
2024/11/17 15:44:59.372 [debug] 2615#138312 free(14301CA00)
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(0): 142E04A60
2024/11/17 15:44:59.372 [debug] 2620#138320 recvmsg(12, 2, 0): -1
2024/11/17 15:44:59.372 [debug] 2620#138320 recvmsg(12) not ready
2024/11/17 15:44:59.372 [debug] 2615#138312 kevent(3) changes:0 timeout:-1
2024/11/17 15:44:59.372 [debug] 2620#138320 kevent(3) changes:0 timeout:-1
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(40): 142E05150
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(84): 144004080
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(85): 1440041C0
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(87): 144004220
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(85): 144004280
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(76): 1440040E0
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(88): 1440042E0
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(83): 144004340
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(78): 1440043A0
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(79): 1440043F0
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(78): 144004440
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(81): 144004490
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(76): 1440044F0
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(75): 144004540
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(81): 144004590
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(76): 1440045F0
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(68): 144004640
2024/11/17 15:44:59.372 [debug] 2621#138321 malloc(83): 144004690
2024/11/17 15:44:59.372 [debug] 0#138321 [unit] Create JavaVM
2024/11/17 15:44:59.398 [debug] 0#138321 [unit] Create JavaVM result: 0
2024/11/17 15:44:59.398 [debug] 0#138321 [unit] init java Thread
2024/11/17 15:44:59.399 [debug] 0#138321 [unit] java Thread result: 0
2024/11/17 15:44:59.399 [debug] 0#138321 [unit] init classLoader
2024/11/17 15:44:59.399 [debug] 0#138321 [unit] new classLoader
2024/11/17 15:44:59.399 [debug] 0#138321 [unit] nxt_java_initContext
2024/11/17 15:44:59.417 [debug] 0#138321 [unit] registered Context methods: 0
2024/11/17 15:44:59.417 [debug] 0#138321 [unit] nxt_java_initRequest
2024/11/17 15:44:59.445 [debug] 0#138321 [unit] registered Request methods: 0
2024/11/17 15:44:59.445 [debug] 0#138321 [unit] registered HeadersEnumeration methods: 0
2024/11/17 15:44:59.446 [debug] 0#138321 [unit] registered HeaderNamesEnumeration methods: 0
2024/11/17 15:44:59.446 [debug] 0#138321 [unit] nxt_java_initResponse
2024/11/17 15:44:59.450 [debug] 0#138321 [unit] registered Response methods: 0
2024/11/17 15:44:59.450 [debug] 0#138321 [unit] nxt_java_initInputStream
2024/11/17 15:44:59.450 [debug] 0#138321 [unit] registered InputStream methods: 0
2024/11/17 15:44:59.450 [debug] 0#138321 [unit] nxt_java_initOutputStream
2024/11/17 15:44:59.450 [debug] 0#138321 [unit] registered OutputStream methods: 0
2024/11/17 15:44:59.450 [debug] 0#138321 [unit] nxt_java_jni_init
2024/11/17 15:44:59.450 [debug] 0#138321 [unit] nxt_java_jni_init: 0
2024/11/17 15:44:59.450 [debug] 0#138321 [unit] nxt_java_startContext...
2024/11/17 15:45:00.377 [debug] 0#138321 [unit] nxt_java_init_threads...
2024/11/17 15:44:59.372 [debug] 2621#138321 process port (2621, 0) found
2024/11/17 15:45:00.377 [debug] 0#138321 [unit] nxt_unit_init...
2024/11/17 15:45:00.377 [debug] 2621#138321 [unit] add_port: port{2619,0} in_fd -1 out_fd 11 queue 0x0
2024/11/17 15:45:00.377 [debug] 2621#138321 [unit] posix_memalign(128, 128): 0x126ce1680
2024/11/17 15:45:00.377 [debug] 2621#138321 [unit] posix_memalign(128, 128): 0x126ce1780
2024/11/17 15:45:00.378 [debug] 2621#138321 [unit] add_port: port{2621,0} in_fd 6 out_fd 10 queue 0x136000000
2024/11/17 15:45:00.378 [debug] 2621#138321 [unit] close(9): 0
2024/11/17 15:45:00.378 [debug] 2621#138321 [unit] add_port: port{2621,65535} in_fd 8 out_fd -1 queue 0x1360a4000
2024/11/17 15:45:00.378 [debug] 2621#138321 [unit] sendmsg(13, 16, 16): 16
2024/11/17 15:45:00.378 [debug] 2621#138321 [unit] close(13): 0
2024/11/17 15:45:00.378 [debug] 2620#138320 kevent(3): 1
2024/11/17 15:45:00.378 [debug] 2620#138320 kevent: id:12 ft:-1 fl:0025 ff:0 d:16 ud:142F06650
2024/11/17 15:45:00.378 [debug] 2620#138320 timer expire minimum: 105393128:18994135
2024/11/17 15:45:00.378 [debug] 2620#138320 work queue: fast
2024/11/17 15:45:00.378 [debug] 2620#138320 recvmsg(12, 2, 16): 16
2024/11/17 15:45:00.378 [debug] 2620#138320 port 12: message type:19 fds:6,-1
2024/11/17 15:45:00.378 [debug] 2620#138320 pthread_mutex_lock(142E04AE0) enter
2024/11/17 15:45:00.378 [debug] 2620#138320 pthread_mutex_unlock(142E04AE0) exit
2024/11/17 15:45:00.378 [debug] 2621#138321 [unit] close(91): 0
2024/11/17 15:45:00.378 [debug] 2620#138320 process 2621 ready
...

gdufrene avatar Nov 17 '24 15:11 gdufrene

Do you have a simple reproducer app and config?

ac000 avatar Nov 19 '24 20:11 ac000

Hello @ac000 ,

actually the war provided by @tippexs on that issue and associated unit config, with my working directory, produce the problem on my system when unit java module is configured and compiled with java 17 : no return from http call to reconfigure unit and a complete freeze of application.

Same for a spring 2.7.18 application based on javax specs with the same java 17 compiled module.

demo2-0.0.1-SNAPSHOT.war.zip

gdufrene avatar Nov 22 '24 12:11 gdufrene

That app starts reliably on Linux with OpenJDK 21.

It does fail to start sometimes on macOS with OpenJDK 23.

ac000 avatar Nov 27 '24 03:11 ac000

@gdufrene

Been a while! and if you've moved onto other things no worries and even though Unit has effectively transitioned into maintenance mode, I had been meaning to test that app on FreeBSD where with FreeBSD 14 and OpenJDK 24 it seems to start reliably.

So yes, seems there is something buggy in the java stack (whether in Unit or elsewhere) on macOS...

ac000 avatar Apr 30 '25 17:04 ac000