amoro icon indicating copy to clipboard operation
amoro copied to clipboard
verified publisher icon apache

[Bug]: Using a concurrency-unsafe data type

Open zhongqishang opened this issue 1 year ago • 2 comments

What happened?

Error occurred while accessing the optimize/optimizerGroups/all/tables.

Affects Versions

master

What table formats are you seeing the problem on?

Iceberg

What engines are you seeing the problem on?

AMS

How to reproduce

Occasionally

Relevant log output

2024-10-21 17:48:49,050 ERROR [JettyServerThreadPool-88] [org.apache.amoro.server.dashboard.DashboardServer] [] - An error occurred while processing the url:http://xxx.com/ams/v1/optimize/optimizerGroups/all/tables
java.util.ConcurrentModificationException: null
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1390) ~[?:1.8.0_392]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_392]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_392]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_392]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_392]
        at java.util.stream.LongPipeline.reduce(LongPipeline.java:461) ~[?:1.8.0_392]
        at java.util.stream.LongPipeline.sum(LongPipeline.java:419) ~[?:1.8.0_392]
        at org.apache.amoro.server.table.TableRuntime.getQuotaTime(TableRuntime.java:618) ~[amoro-ams-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
        at org.apache.amoro.server.table.TableRuntime.calculateQuotaOccupy(TableRuntime.java:627) ~[amoro-ams-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
        at org.apache.amoro.server.dashboard.utils.OptimizingUtil.buildTableOptimizeInfo(OptimizingUtil.java:46) ~[amoro-ams-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_392]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_392]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_392]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_392]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_392]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_392]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[?:1.8.0_392]
        at org.apache.amoro.server.dashboard.controller.OptimizerGroupController.getOptimizerTables(OptimizerGroupController.java:102) ~[amoro-ams-0.8-SNAPSHOT.jar:0.8-SNAPSHOT]
        at io.javalin.core.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:20) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServlet.addHandler$lambda-0(JavalinServlet.kt:96) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:99) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8]
        at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995) ~[?:1.8.0_392]
        at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137) ~[?:1.8.0_392]
        at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServletHandler.executeNextTask$lambda-11$lambda-10(JavalinServletHandler.kt:119) ~[javalin-4.6.8.jar:4.6.8]
        at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) ~[?:1.8.0_392]
        at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628) ~[?:1.8.0_392]
        at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996) ~[?:1.8.0_392]
        at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:119) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8]
        at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995) ~[?:1.8.0_392]
        at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137) ~[?:1.8.0_392]
        at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServletHandler.executeNextTask$lambda-11$lambda-10(JavalinServletHandler.kt:119) ~[javalin-4.6.8.jar:4.6.8]
        at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) ~[?:1.8.0_392]
        at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628) ~[?:1.8.0_392]
        at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996) ~[?:1.8.0_392]
        at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:119) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8]
        at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:995) ~[?:1.8.0_392]
        at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2137) ~[?:1.8.0_392]
        at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) ~[javalin-4.6.8.jar:4.6.8]
        at io.javalin.http.JavalinServlet.service(JavalinServlet.kt:89) ~[javalin-4.6.8.jar:4.6.8]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) ~[jakarta.servlet-api-4.0.3.jar:4.0.3]
        at io.javalin.jetty.JavalinJettyServlet.service(JavalinJettyServlet.kt:58) ~[javalin-4.6.8.jar:4.6.8]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) ~[jakarta.servlet-api-4.0.3.jar:4.0.3]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at io.javalin.jetty.JettyServer$start$wsAndHttpHandler$1.doHandle(JettyServer.kt:52) ~[javalin-4.6.8.jar:4.6.8]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) ~[jetty-servlet-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) [jetty-server-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) [jetty-util-9.4.51.v20230217.jar:9.4.51.v20230217]
        at java.lang.Thread.run(Thread.java:750) [?:1.8.0_392]

Anything else

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

zhongqishang avatar Oct 21 '24 09:10 zhongqishang

Hi, can I be assigned to work on this issue? I've identified the cause of this ConcurrentModificationException and have a potential fix in mind.

guilherme-dsantos avatar Oct 22 '24 08:10 guilherme-dsantos

@guilherme-dsantos Assigned to you.

zhongqishang avatar Oct 22 '24 08:10 zhongqishang