dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Bug] [service] while query workflow definition and workflow lineage, both throw an exception 'query process definition list paging error'

Open nickzibow opened this issue 1 year ago • 4 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

1、when i visit the page of workflow definition, it throw an error like below Error-workFlow

Error-workFlow

2、when i visit the page of workflow lineage, it throw an error like below Error-workLineage

Error-workLineage.txt

What you expected to happen

I found a similar problem in the issue, but it has not been solved yet: https://github.com/apache/dolphinscheduler/issues/13577

I've googled but couldn't get any help. Google said it is because there are duplicate records and fields in the two tables t_ds_process_definition and t_ds_process_definition_log, but our team has checked the database and has not found any duplicate records and fields. Moreover, I have not upgraded dolphinscheduler and have been using version 3.2.0, so I hope to get any solution ideas, thank you!

How to reproduce

1、env is openstack VMs deploy k8s 2、use helm deploy: https://github.com/apache/dolphinscheduler/tree/3.2.0-release/deploy/kubernetes/dolphinscheduler 3、After using it for a period of time, after increasing the k8s VM memory and restarting dolphinscheduler, the above exception was found.

Anything else

No response

Version

3.2.x

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

nickzibow avatar May 20 '24 10:05 nickzibow

[Error-workFlow.log] [DEBUG] 2024-05-20 17:35:55.159 +0800 org.eclipse.jetty.server.HttpChannelState:[414] - unhandle HttpChannelState@64bbc922{s=HANDLING rs=BLOCKING os=COMMITTED is=IDLE awp=false se=false i=true al=0} [DEBUG] 2024-05-20 17:35:55.159 +0800 org.eclipse.jetty.server.HttpChannelState:[423] - nextAction(false) COMPLETE HttpChannelState@64bbc922{s=HANDLING rs=COMPLETING os=COMMITTED is=IDLE awp=false se=false i=false al=0} [DEBUG] 2024-05-20 17:35:55.159 +0800 org.eclipse.jetty.server.HttpChannel:[462] - action COMPLETE HttpChannelOverHttp@608451ea{s=HttpChannelState@64bbc922{s=HANDLING rs=COMPLETING os=COMMITTED is=IDLE awp=false se=false i=false al=0},r=5,c=false/false,a=HANDLING,uri=https://uat-ds.pacific-textiles.com:80/dolphinscheduler/alert-groups/list,age=16} [DEBUG] 2024-05-20 17:35:55.159 +0800 org.eclipse.jetty.server.HttpConnection:[751] - generate: NEED_HEADER for SendCallback@65dbed25[PROCESSING][i=HTTP/1.1{s=200,h=7,cl=-1},cb=org.eclipse.jetty.server.HttpChannel$SendCallback@67539971] (null,[p=0,l=10,c=32768,r=10],false)@START [ERROR] 2024-05-20 17:35:55.159 +0800 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[52] - 分页查询工作流定义列表错误 java.lang.IllegalStateException: Duplicate key Schedule(id=8, processDefinitionCode=13453576351553, processDefinitionName=null, projectName=null, definitionDescription=null, startTime=Mon May 20 00:00:00 HKT 2024, endTime=Mon May 20 23:00:00 HKT 2024, timezoneId=Asia/Shanghai, crontab=2 3 0 * * ? , failureStrategy=CONTINUE, warningType=NONE, createTime=Sun May 19 21:21:09 HKT 2024, updateTime=Sun May 19 21:21:09 HKT 2024, userId=1, userName=null, releaseState=OFFLINE, warningGroupId=0, processInstancePriority=MEDIUM, workerGroup=default, tenantCode=default, environmentCode=-1, environmentName=null) at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133) at java.util.HashMap.merge(HashMap.java:1255) at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320) at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.queryProcessDefinitionListPaging(ProcessDefinitionServiceImpl.java:580) at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$FastClassBySpringCGLIB$$e8e34ed9.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386) at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:704) at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$EnhancerBySpringCGLIB$$a0a57fe3.queryProcessDefinitionListPaging() at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.queryProcessDefinitionListPaging(ProcessDefinitionController.java:511) at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$FastClassBySpringCGLIB$$dc9bf5db.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:119) at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708) at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$EnhancerBySpringCGLIB$$e4ecb2c9.queryProcessDefinitionListPaging() at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1070) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:497) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:584) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:750) [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.server.HttpOutput:[500] - complete(org.eclipse.jetty.util.Callback$3@5d02df44) s=CLOSING,api=BLOCKED,sc=false,e=null s=false e=null, c=HeapByteBuffer@2c18430c[p=0,l=0,c=0,r=0]={<<<>>>} [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.server.HttpChannel:[986] - sendResponse info=null content=HeapByteBuffer@4c9227fa[p=0,l=185,c=32768,r=185]={<<<\x85\x8f;\x0e\xC20\x10D\xAfbm\xEdH\xB1I\x84\xE4\x1b\xA4\xA1\x82\nQ\xAc\xEc...\xCb\xA6\x142q\xE8\xD1:\xB7\x1e]\xA4\xF5\x03F\xF1\xC8\xEc\x0e\x01\x00\x00>>>,"updateT...lor-dis} complete=true committing=false callback=GzipBufferCB@25b71ad7[content=HeapByteBuffer@2c18430c[p=0,l=0,c=0,r=0]={<<<>>>} last=true copy=null buffer=HeapByteBuffer@4c9227fa[p=0,l=185,c=32768,r=185]={<<<\x85\x8f;\x0e\xC20\x10D\xAfbm\xEdH\xB1I\x84\xE4\x1b\xA4\xA1\x82\nQ\xAc\xEc...\xCb\xA6\x142q\xE8\xD1:\xB7\x1e]\xA4\xF5\x03F\xF1\xC8\xEc\x0e\x01\x00\x00>>>,"updateT...lor-dis} deflate=null ] [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.server.HttpConnection:[751] - generate: NEED_CHUNK for SendCallback@28c178e2[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@6595567a] (null,[p=0,l=185,c=32768,r=185],true)@COMMITTED [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.server.HttpConnection:[751] - generate: FLUSH for SendCallback@28c178e2[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@6595567a] (null,[p=0,l=185,c=32768,r=185],true)@COMPLETING [DEBUG] 2024-05-20 17:35:55.160 +0800 org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor:[268] - Using 'application/json', given [application/json, text/plain, /] and supported [application/json, application/+json, application/json, application/+json, application/cbor, application/xml;charset=UTF-8, text/xml;charset=UTF-8, application/+xml;charset=UTF-8, application/xml;charset=UTF-8, text/xml;charset=UTF-8, application/+xml;charset=UTF-8] [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.io.WriteFlusher:[270] - write: WriteFlusher@77b3607{IDLE}->null [HeapByteBuffer@1c163e7f[p=0,l=6,c=1024,r=6]={<<<\r\nB9\r\n>>>\n\n\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00},HeapByteBuffer@4c9227fa[p=0,l=185,c=32768,r=185]={<<<\x85\x8f;\x0e\xC20\x10D\xAfbm\xEdH\xB1I\x84\xE4\x1b\xA4\xA1\x82\nQ\xAc\xEc...\xCb\xA6\x142q\xE8\xD1:\xB7\x1e]\xA4\xF5\x03F\xF1\xC8\xEc\x0e\x01\x00\x00>>>,"updateT...lor-dis}] [DEBUG] 2024-05-20 17:35:55.160 +0800 org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor:[119] - Writing [Status{code='10122', msg='分页查询工作流定义列表错误', data=null}] [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.io.WriteFlusher:[112] - update WriteFlusher@77b3607{WRITING}->null:IDLE-->WRITING [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.io.ManagedSelector:[612] - Selector sun.nio.ch.EPollSelectorImpl@2f096645 woken up from select, 1/1/1 selected [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.server.HttpOutput:[785] - write(array HeapByteBuffer@3683ff8c[p=0,l=104,c=8000,r=104]={<<<{"code":10122,"msg":"\xE5\x88\x86\xE9...s":false,"failed":true}>>>ccess":tr...\x00\x00\x00\x00\x00\x00\x00}) [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.ManagedSelector:[628] - Selector sun.nio.ch.EPollSelectorImpl@2f096645 processing 1 keys, 0 updates [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.ChannelEndPoint:[276] - flushed 191 SocketChannelEndPoint@6e43a252{l=/10.244.7.243:12345,r=/10.244.4.99:47444,OPEN,fill=-,flush=W,to=2/30000}{io=0/0,kio=0,kro=1}->HttpConnection@21b95bd1[p=HttpParser{s=CONTENT,0 of -1},g=HttpGenerator@10523c3e{s=COMPLETING}]=>HttpChannelOverHttp@608451ea{s=HttpChannelState@64bbc922{s=HANDLING rs=COMPLETING os=COMMITTED is=IDLE awp=false se=false i=false al=0},r=5,c=false/false,a=HANDLING,uri=https://uat-ds.pacific-textiles.com:80/dolphinscheduler/alert-groups/list,age=17} [DEBUG] 2024-05-20 17:35:55.160 +0800 org.eclipse.jetty.http.HttpGenerator:[603] - generateHeaders HTTP/1.1{s=200,h=7,cl=-1} last=false content=HeapByteBuffer@c7e9c61[p=0,l=10,c=32768,r=10]={<<<\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00>>>"msg":"\xE6\x88....not,n=} [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.server.HttpOutput:[843] - write(array) s=OPEN,api=BLOCKING,sc=false,e=null aggregated !flush HeapByteBuffer@2e072802[p=0,l=104,c=32768,r=104]={<<<{"code":10122,"msg":"\xE5\x88\x86\xE9...s":false,"failed":true}>>>\xE1&/\xC2\x89XA\x87\x89...urn n+(} [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.WriteFlusher:[427] - Flushed=true written=191 remaining=0 WriteFlusher@77b3607{WRITING}->null [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.ManagedSelector:[664] - selected 1 sun.nio.ch.SelectionKeyImpl@2923b281 SocketChannelEndPoint@523c6865{l=/10.244.7.243:12345,r=/10.244.4.99:40356,OPEN,fill=FI,flush=-,to=15/30000}{io=1/1,kio=1,kro=1}->HttpConnection@4ad389fb[p=HttpParser{s=START,0 of -1},g=HttpGenerator@21525d7f{s=START}]=>HttpChannelOverHttp@69566870{s=HttpChannelState@2167e06f{s=IDLE rs=BLOCKING os=OPEN is=IDLE awp=false se=false i=true al=0},r=1,c=false/false,a=IDLE,uri=null,age=0} [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.WriteFlusher:[112] - update WriteFlusher@77b3607{IDLE}->null:WRITING-->IDLE [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor:[228] - org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor@6dac4bc0 compressing java.util.zip.Deflater@4b195b04 [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.ChannelEndPoint:[340] - onSelected 1->0 r=true w=false for SocketChannelEndPoint@523c6865{l=/10.244.7.243:12345,r=/10.244.4.99:40356,OPEN,fill=FI,flush=-,to=15/30000}{io=1/0,kio=1,kro=1}->HttpConnection@4ad389fb[p=HttpParser{s=START,0 of -1},g=HttpGenerator@21525d7f{s=START}]=>HttpChannelOverHttp@69566870{s=HttpChannelState@2167e06f{s=IDLE rs=BLOCKING os=OPEN is=IDLE awp=false se=false i=true al=0},r=1,c=false/false,a=IDLE,uri=null,age=0} [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.server.HttpConnection:[751] - generate: FLUSH for SendCallback@28c178e2[PROCESSING][i=null,cb=org.eclipse.jetty.server.HttpChannel$SendCallback@6595567a] (null,[p=185,l=185,c=32768,r=0],true)@COMPLETING [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.server.HttpChannel:[986] - sendResponse info=null content=HeapByteBuffer@4cbd5526[p=0,l=10,c=32768,r=10]={<<<\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00>>>"msg":"\xE6\x88....indent} complete=false committing=true callback=GzipBufferCB@99a2b5d[content=HeapByteBuffer@2e072802[p=104,l=104,c=32768,r=0]={{"code":1...":true}<<<>>>\xE1&/\xC2\x89XA\x87\x89...urn n+(} last=false copy=null buffer=HeapByteBuffer@4cbd5526[p=0,l=10,c=32768,r=10]={<<<\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00>>>"msg":"\xE6\x88....indent} deflate=java.util.zip.Deflater@4b195b04 ] [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.WriteFlusher:[270] - write: WriteFlusher@77b3607{IDLE}->null [HeapByteBuffer@1c163e7f[p=0,l=7,c=1024,r=7]={<<<\r\n0\r\n\r\n>>>\n\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00}] [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.ChannelEndPoint:[352] - task CEP:SocketChannelEndPoint@523c6865{l=/10.244.7.243:12345,r=/10.244.4.99:40356,OPEN,fill=FI,flush=-,to=16/30000}{io=1/0,kio=1,kro=1}->HttpConnection@4ad389fb[p=HttpParser{s=START,0 of -1},g=HttpGenerator@21525d7f{s=START}]=>HttpChannelOverHttp@69566870{s=HttpChannelState@2167e06f{s=IDLE rs=BLOCKING os=OPEN is=IDLE awp=false se=false i=true al=0},r=1,c=false/false,a=IDLE,uri=null,age=0}:runFillable:BLOCKING [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.io.WriteFlusher:[112] - update WriteFlusher@77b3607{WRITING}->null:IDLE-->WRITING [DEBUG] 2024-05-20 17:35:55.161 +0800 org.eclipse.jetty.server.HttpChannel:[1044] - COMMIT for /dolphinscheduler/projects/11730214956320/process-definition on HttpChannelOverHttp@cc59866{s=HttpChannelState@3f5aea4a{s=HANDLING rs=BLOCKING os=COMMITTED is=IDLE awp=false se=false i=true al=0},r=4,c=false/false,a=HANDLING,uri=https://uat-ds.pacific-textiles.com:80/dolphinscheduler/projects/11730214956320/process-definition?pageSize=10&pageNo=1,age=35} 200 null HTTP/1.1 Date: Mon, 20 May 2024 09:35:55 GMT Vary: Origin Vary: Access-Control-Request-Method Vary: Access-Control-Request-Headers Content-Type: application/json Vary: Accept-Encoding, User-Agent Content-Encoding: gzip

nickzibow avatar May 20 '24 10:05 nickzibow

[Error-workLineage.log] [ERROR] 2024-05-20 17:26:07.790 +0800 org.apache.dolphinscheduler.api.controller.WorkFlowLineageController:[113] - 查询血缘失败 java.lang.IllegalStateException: Duplicate key org.apache.dolphinscheduler.dao.entity.WorkFlowLineage@79a5fae4 at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133) at java.util.HashMap.merge(HashMap.java:1255) at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320) at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) at org.apache.dolphinscheduler.api.service.impl.WorkFlowLineageServiceImpl.queryWorkFlowLineage(WorkFlowLineageServiceImpl.java:185) at org.apache.dolphinscheduler.api.controller.WorkFlowLineageController.queryWorkFlowLineage(WorkFlowLineageController.java:110) at org.apache.dolphinscheduler.api.controller.WorkFlowLineageController$$FastClassBySpringCGLIB$$30b325.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:119) at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708) at org.apache.dolphinscheduler.api.controller.WorkFlowLineageController$$EnhancerBySpringCGLIB$$aafe88e7.queryWorkFlowLineage() at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1070) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:497) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:584) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656) at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:750)

nickzibow avatar May 20 '24 10:05 nickzibow

@pegasas Would you like to fix it?

SbloodyS avatar Jun 28 '24 07:06 SbloodyS

Hi, @SbloodyS ,

i am based 262a57a4d9c45189f5aaa29d51cda23531233882 refs/tags/3.2.0

from my view, it seems there exists multiple t_ds_schedules with the same process_definition_code, so that will return multiple records with same code after left join. image image

would you mind to check our t_ds_schedules table to see if there is records with the same process_definition_code? @nickzibow

pegasas avatar Jun 28 '24 09:06 pegasas

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Jul 29 '24 00:07 github-actions[bot]

Fixed.

SbloodyS avatar Jul 29 '24 02:07 SbloodyS