incubator-streampark icon indicating copy to clipboard operation
incubator-streampark copied to clipboard

[Bug]Can't trigger savepoint

Open westPointz opened this issue 2 years ago • 3 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

Java Version

1.8

Scala Version

2.12.x

StreamPark Version

2.1.1

Flink Version

1.16.1

deploy mode

kubernetes-application

What happened

Can't trigger savepoint

Error Exception

java.util.concurrent.CompletionException: java.lang.reflect.InvocationTargetException
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1702)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.streampark.flink.client.FlinkClient$.$anonfun$proxy$1(FlinkClient.scala:80)
	at org.apache.streampark.flink.proxy.FlinkShimsProxy$.$anonfun$proxy$1(FlinkShimsProxy.scala:60)
	at org.apache.streampark.common.util.ClassLoaderUtils$.runAsClassLoader(ClassLoaderUtils.scala:38)
	at org.apache.streampark.flink.proxy.FlinkShimsProxy$.proxy(FlinkShimsProxy.scala:60)
	at org.apache.streampark.flink.client.FlinkClient$.proxy(FlinkClient.scala:75)
	at org.apache.streampark.flink.client.FlinkClient$.triggerSavepoint(FlinkClient.scala:65)
	at org.apache.streampark.flink.client.FlinkClient.triggerSavepoint(FlinkClient.scala)
	at org.apache.streampark.console.core.service.impl.SavePointServiceImpl.lambda$trigger$0(SavePointServiceImpl.java:320)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
	... 3 more
Caused by: java.util.concurrent.ExecutionException: org.apache.flink.util.concurrent.FutureUtils$RetryException: Could not complete the operation. Number of retries has been exhausted.
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
	at org.apache.streampark.flink.client.trait.FlinkClientTrait.triggerSavepoint(FlinkClientTrait.scala:569)
	at org.apache.streampark.flink.client.trait.FlinkClientTrait.triggerSavepoint$(FlinkClientTrait.scala:563)
	at org.apache.streampark.flink.client.impl.KubernetesNativeApplicationClient$.org$apache$streampark$flink$client$trait$KubernetesNativeClientTrait$$super$triggerSavepoint(KubernetesNativeApplicationClient.scala:35)
	at org.apache.streampark.flink.client.trait.KubernetesNativeClientTrait.$anonfun$doTriggerSavepoint$1(KubernetesNativeClientTrait.scala:134)
	at org.apache.streampark.flink.client.trait.KubernetesNativeClientTrait.executeClientAction(KubernetesNativeClientTrait.scala:115)
	at org.apache.streampark.flink.client.trait.KubernetesNativeClientTrait.doTriggerSavepoint(KubernetesNativeClientTrait.scala:133)
	at org.apache.streampark.flink.client.trait.KubernetesNativeClientTrait.doTriggerSavepoint$(KubernetesNativeClientTrait.scala:127)
	at org.apache.streampark.flink.client.impl.KubernetesNativeApplicationClient$.doTriggerSavepoint(KubernetesNativeApplicationClient.scala:104)
	at org.apache.streampark.flink.client.trait.FlinkClientTrait.triggerSavepoint(FlinkClientTrait.scala:169)
	at org.apache.streampark.flink.client.trait.FlinkClientTrait.triggerSavepoint$(FlinkClientTrait.scala:155)
	at org.apache.streampark.flink.client.impl.KubernetesNativeApplicationClient$.triggerSavepoint(KubernetesNativeApplicationClient.scala:35)
	at org.apache.streampark.flink.client.FlinkClientHandler$.triggerSavepoint(FlinkClientHandler.scala:56)
	at org.apache.streampark.flink.client.FlinkClientHandler.triggerSavepoint(FlinkClientHandler.scala)
	... 16 more
Caused by: org.apache.flink.util.concurrent.FutureUtils$RetryException: Could not complete the operation. Number of retries has been exhausted.
	at org.apache.flink.util.concurrent.FutureUtils.lambda$retryOperationWithDelay$6(FutureUtils.java:293)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:610)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1085)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)
	... 3 more
Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.rest.util.RestClientException: Response was neither of the expected type([simple type, class org.apache.flink.runtime.rest.handler.async.AsynchronousOperationResult<org.apache.flink.runtime.rest.messages.job.savepoints.SavepointInfo>]) nor an error.
	at java.base/java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:367)
	at java.base/java.util.concurrent.CompletableFuture.completeRelay(CompletableFuture.java:376)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1074)
	... 4 more
Caused by: org.apache.flink.runtime.rest.util.RestClientException: Response was neither of the expected type([simple type, class org.apache.flink.runtime.rest.handler.async.AsynchronousOperationResult<org.apache.flink.runtime.rest.messages.job.savepoints.SavepointInfo>]) nor an error.
	at org.apache.flink.runtime.rest.RestClient.parseResponse(RestClient.java:550)
	at org.apache.flink.runtime.rest.RestClient.lambda$submitRequest$3(RestClient.java:514)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
	... 4 more
Caused by: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException: class org.apache.flink.util.SerializedThrowable cannot be cast to class org.apache.flink.util.SerializedThrowable (org.apache.flink.util.SerializedThrowable is in unnamed module of loader 'app'; org.apache.flink.util.SerializedThrowable is in unnamed module of loader org.apache.streampark.flink.proxy.ChildFirstClassLoader @57564bf6) (through reference chain: org.apache.flink.runtime.rest.handler.async.AsynchronousOperationResult["operation"]->org.apache.flink.runtime.rest.messages.job.savepoints.SavepointInfo["failure-cause"])
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:390)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:349)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1822)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:567)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:449)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:542)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:565)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:449)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4569)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2867)
	at org.apache.flink.runtime.rest.RestClient.parseResponse(RestClient.java:523)
	... 6 more
Caused by: java.lang.ClassCastException: class org.apache.flink.util.SerializedThrowable cannot be cast to class org.apache.flink.util.SerializedThrowable (org.apache.flink.util.SerializedThrowable is in unnamed module of loader 'app'; org.apache.flink.util.SerializedThrowable is in unnamed module of loader org.apache.streampark.flink.proxy.ChildFirstClassLoader @57564bf6)
	at org.apache.flink.runtime.rest.messages.json.SerializedThrowableDeserializer.deserialize(SerializedThrowableDeserializer.java:49)
	at org.apache.flink.runtime.rest.messages.json.SerializedThrowableDeserializer.deserialize(SerializedThrowableDeserializer.java:34)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:542)
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:565)
	... 20 more

Screenshots

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!(您是否要贡献这个PR?)

Code of Conduct

westPointz avatar Oct 20 '23 08:10 westPointz

i meet the same problem can yon slove this probelm

wsczm avatar Nov 09 '23 01:11 wsczm

Looks like a bug. 🐛

wolfboys avatar Dec 07 '23 04:12 wolfboys

i also meet the same problem

Li-GL avatar Jan 11 '24 07:01 Li-GL