hippo4j icon indicating copy to clipboard operation
hippo4j copied to clipboard

[Bug] Start client side log printing, reflection error, resulting in thread pool failure

Open Jamel-jun opened this issue 1 year ago • 7 comments

  • java.version: 17
  • spring boot.version: 2.7.12
  • hippo4j-spring-boot-starter.version: 1.5.0
  • 依赖项只有hippo4j以及web image

问题必现

BUG日志:客户端应用启动时打印如下日志

image

目前解决方案

启动jar时加上vm参数:--add-opens java.base/java.util.concurrent=ALL-UNNAMED

Jamel-jun avatar Jun 05 '23 08:06 Jamel-jun

Java9 broke reflection and as the version number goes up, the breaks get worse. Essentially, accessing non-exported elements of other modules (modules introduced in java 9) does not work without adding a bevy of complicated lines when starting the java executable. "modules" as a concept did not exist prior to java9, hence, the java8 and below javadoc makes no mention of any of this.--add-opens.

see: https://stackoverflow.com/questions/74723932/java-17-reflection-issue

iwangjie avatar Jul 03 '23 01:07 iwangjie

Java9 broke reflection and as the version number goes up, the breaks get worse. Essentially, accessing non-exported elements of other modules (modules introduced in java 9) does not work without adding a bevy of complicated lines when starting the java executable. "modules" as a concept did not exist prior to java9, hence, the java8 and below javadoc makes no mention of any of this.--add-opens.

see: https://stackoverflow.com/questions/74723932/java-17-reflection-issue

I have solved the problem through -- add-opens. Should I use other implementation solutions to be compatible with jdk17 and avoid reflection?

Jamel-jun avatar Jul 03 '23 02:07 Jamel-jun

Java9 broke reflection and as the version number goes up, the breaks get worse. Essentially, accessing non-exported elements of other modules (modules introduced in java 9) does not work without adding a bevy of complicated lines when starting the java executable. "modules" as a concept did not exist prior to java9, hence, the java8 and below javadoc makes no mention of any of this.--add-opens. see: https://stackoverflow.com/questions/74723932/java-17-reflection-issue

I have solved the problem through -- add-opens. Should I use other implementation solutions to be compatible with jdk17 and avoid reflection?

Unfortunately, there is no better solution, and as the article states, the compatibility issues associated with upgrading the JDK may persist for a long time to be resolved.

iwangjie avatar Jul 03 '23 02:07 iwangjie

Java9 broke reflection and as the version number goes up, the breaks get worse. Essentially, accessing non-exported elements of other modules (modules introduced in java 9) does not work without adding a bevy of complicated lines when starting the java executable. "modules" as a concept did not exist prior to java9, hence, the java8 and below javadoc makes no mention of any of this.--add-opens. see: https://stackoverflow.com/questions/74723932/java-17-reflection-issue

I have solved the problem through -- add-opens. Should I use other implementation solutions to be compatible with jdk17 and avoid reflection?

Unfortunately, there is no better solution, and as the article states, the compatibility issues associated with upgrading the JDK may persist for a long time to be resolved.

iwangjie avatar Jul 03 '23 02:07 iwangjie

高版本jdk对反射修改做了限制,不过可以使用unsafe修改,juc里大量使用unsafe,所以unsafe就变得相对safe了。see this:https://github.com/fuxiuzhan/fuled-component/blob/master/fuled-dynamic-threadpool-starter/src/main/java/com/fxz/fuled/dynamic/threadpool/ThreadPoolRegistry.java

fuxiuzhan avatar Aug 11 '23 06:08 fuxiuzhan

/** * Thread-pool param replace. * * @param executor dynamic thread-pool executor * @param threadPoolParameterInfo thread-pool parameter info */ private void threadPoolParamReplace(ThreadPoolExecutor executor, ThreadPoolParameterInfo threadPoolParameterInfo) {

#如下直接生成新的queue替换原有的引用,如果原始的queue里已经有堆积的任务,直接替换是不是会丢任务?

BlockingQueue workQueue = BlockingQueueTypeEnum.createBlockingQueue(threadPoolParameterInfo.getQueueType(), threadPoolParameterInfo.getCapacity()); ReflectUtil.setFieldValue(executor, "workQueue", workQueue); // fix https://github.com/opengoofy/hippo4j/issues/1063 ThreadPoolExecutorUtil.safeSetPoolSize(executor, threadPoolParameterInfo.corePoolSizeAdapt(), threadPoolParameterInfo.maximumPoolSizeAdapt()); executor.setKeepAliveTime(threadPoolParameterInfo.getKeepAliveTime(), TimeUnit.SECONDS); executor.allowCoreThreadTimeOut(EnableEnum.getBool(threadPoolParameterInfo.getAllowCoreThreadTimeOut())); executor.setRejectedExecutionHandler(RejectedPolicyTypeEnum.createPolicy(threadPoolParameterInfo.getRejectedType())); if (executor instanceof DynamicThreadPoolExecutor) { Optional.ofNullable(threadPoolParameterInfo.getExecuteTimeOut()) .ifPresent(executeTimeOut -> ((DynamicThreadPoolExecutor) executor).setExecuteTimeOut(executeTimeOut)); } }

fuxiuzhan avatar Aug 11 '23 06:08 fuxiuzhan

/** * Thread-pool param replace. * * @param executor dynamic thread-pool executor * @param threadPoolParameterInfo thread-pool parameter info */ private void threadPoolParamReplace(ThreadPoolExecutor executor, ThreadPoolParameterInfo threadPoolParameterInfo) {

#如下直接生成新的queue替换原有的引用,如果原始的queue里已经有堆积的任务,直接替换是不是会丢任务?

BlockingQueue workQueue = BlockingQueueTypeEnum.createBlockingQueue(threadPoolParameterInfo.getQueueType(), threadPoolParameterInfo.getCapacity()); ReflectUtil.setFieldValue(executor, "workQueue", workQueue); // fix #1063 ThreadPoolExecutorUtil.safeSetPoolSize(executor, threadPoolParameterInfo.corePoolSizeAdapt(), threadPoolParameterInfo.maximumPoolSizeAdapt()); executor.setKeepAliveTime(threadPoolParameterInfo.getKeepAliveTime(), TimeUnit.SECONDS); executor.allowCoreThreadTimeOut(EnableEnum.getBool(threadPoolParameterInfo.getAllowCoreThreadTimeOut())); executor.setRejectedExecutionHandler(RejectedPolicyTypeEnum.createPolicy(threadPoolParameterInfo.getRejectedType())); if (executor instanceof DynamicThreadPoolExecutor) { Optional.ofNullable(threadPoolParameterInfo.getExecuteTimeOut()) .ifPresent(executeTimeOut -> ((DynamicThreadPoolExecutor) executor).setExecuteTimeOut(executeTimeOut)); } }

We may need to verify that this problem exists. If it does, we may need to add extra steps here to ensure that tasks in the old queue are not lost when the work queue is replaced:

  • Try locking the thread pool or old queue to prevent new tasks from entering the queue;
  • Try to transfer tasks from the old queue to the new queue through the drainTo method;

If you don't mind, you can try to verify this problem first :)

Createsequence avatar Aug 18 '23 18:08 Createsequence