chunjun icon indicating copy to clipboard operation
chunjun copied to clipboard

standlone模式 oracle到hive1,有时候直接成功 ,有时候失败,再试一次就成功

Open LeonYoah opened this issue 3 years ago • 4 comments

Search before asking

  • [x] I had searched in the issues and found no similar question.

  • [ ] I had googled my question but i didn't get any help.

  • [ ] I had read the documentation: ChunJun doc but it didn't help me.

Description

2022-08-17 04:05:22,240 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Received JobGraph submission f65908e085b81cac56dce810005055e2 (Flink_Job). 2022-08-17 04:05:22,240 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Submitting job f65908e085b81cac56dce810005055e2 (Flink_Job). 2022-08-17 04:05:22,264 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/rpc/jobmanager_4 . 2022-08-17 04:05:22,265 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Initializing job Flink_Job (f65908e085b81cac56dce810005055e2). 2022-08-17 04:05:22,266 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Using restart back off time strategy NoRestartBackoffTimeStrategy for Flink_Job (f65908e085b81cac56dce810005055e2). 2022-08-17 04:05:22,269 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Running initialization on master for job Flink_Job (f65908e085b81cac56dce810005055e2). 2022-08-17 04:05:22,315 INFO com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat [] - timeZone = sun.util.calendar.ZoneInfo[id="America/New_York",offset=-18000000,dstSavings=3600000,useDaylight=true,transitions=235,lastRule=java.util.SimpleTimeZone[id=America/New_York,offset=-18000000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] 2022-08-17 04:05:22,332 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Successfully ran initialization on master in 63 ms. 2022-08-17 04:05:22,365 INFO com.dtstack.chunjun.connector.oracle.source.OracleInputFormat [] - createInputSplitsInternal success, splits is [ { "mod": 0, "isPolling": false, "splitStrategy": "mod", "rangeEndLocationOperator": " < ", "partitionNumber": 0, "totalNumberOfPartitions": 1 } ] 2022-08-17 04:05:22,366 INFO org.apache.flink.runtime.scheduler.adapter.DefaultExecutionTopology [] - Built 1 pipelined regions in 0 ms 2022-08-17 04:05:22,367 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - No state backend has been configured, using default (Memory / JobManager) MemoryStateBackend (data in heap memory / checkpoints to JobManager) (checkpoints: 'hdfs://glnew:8020/checkpoints/metadata', savepoints: 'null', asynchronous: TRUE, maxStateSize: 5242880) 2022-08-17 04:05:22,368 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No checkpoint found during restore. 2022-08-17 04:05:22,368 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Using failover strategy org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy@7850dbc8 for Flink_Job (f65908e085b81cac56dce810005055e2). 2022-08-17 04:05:22,368 INFO org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl [] - JobManager runner for job Flink_Job (f65908e085b81cac56dce810005055e2) was granted leadership with session id 00000000-0000-0000-0000-000000000000 at akka.tcp://flink@glnew:6123/user/rpc/jobmanager_4. 2022-08-17 04:05:22,370 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Starting execution of job Flink_Job (f65908e085b81cac56dce810005055e2) under job master id 00000000000000000000000000000000. 2022-08-17 04:05:22,371 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Starting scheduling with scheduling strategy [org.apache.flink.runtime.scheduler.strategy.PipelinedRegionSchedulingStrategy] 2022-08-17 04:05:22,371 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Flink_Job (f65908e085b81cac56dce810005055e2) switched from state CREATED to RUNNING. 2022-08-17 04:05:22,372 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: oraclesourcefactory (1/1) (0dcf1185673d44aadc4ed91fa0bdf365) switched from CREATED to SCHEDULED. 2022-08-17 04:05:22,372 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: hivesinkfactory (1/1) (c3e0fe598839c22a55a5801fb667f784) switched from CREATED to SCHEDULED. 2022-08-17 04:05:22,372 INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Cannot serve slot request, no ResourceManager connected. Adding as pending request [SlotRequestId{b9d2597dd077659a15d6a8ed1508bda2}] 2022-08-17 04:05:22,373 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Connecting to ResourceManager akka.tcp://flink@glnew:6123/user/rpc/resourcemanager_*(00000000000000000000000000000000) 2022-08-17 04:05:22,375 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Resolved ResourceManager address, beginning registration 2022-08-17 04:05:22,375 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registering job manager [email protected]://flink@glnew:6123/user/rpc/jobmanager_4 for job f65908e085b81cac56dce810005055e2. 2022-08-17 04:05:22,376 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registered job manager [email protected]://flink@glnew:6123/user/rpc/jobmanager_4 for job f65908e085b81cac56dce810005055e2. 2022-08-17 04:05:22,378 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - JobManager successfully registered at ResourceManager, leader id: 00000000000000000000000000000000. 2022-08-17 04:05:22,378 INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Requesting new slot [SlotRequestId{b9d2597dd077659a15d6a8ed1508bda2}] and profile ResourceProfile{UNKNOWN} with allocation id 0377dfed53086efc343ba0f1ce669298 from resource manager. 2022-08-17 04:05:22,378 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Request slot with profile ResourceProfile{UNKNOWN} for job f65908e085b81cac56dce810005055e2 with allocation id 0377dfed53086efc343ba0f1ce669298. 2022-08-17 04:05:22,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: oraclesourcefactory (1/1) (0dcf1185673d44aadc4ed91fa0bdf365) switched from SCHEDULED to DEPLOYING. 2022-08-17 04:05:22,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: oraclesourcefactory (1/1) (attempt #0) with attempt id 0dcf1185673d44aadc4ed91fa0bdf365 to 192.168.1.188:41941-6be8a1 @ slave1 (dataPort=43852) with allocation id 0377dfed53086efc343ba0f1ce669298 2022-08-17 04:05:22,472 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: hivesinkfactory (1/1) (c3e0fe598839c22a55a5801fb667f784) switched from SCHEDULED to DEPLOYING. 2022-08-17 04:05:22,472 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Sink: hivesinkfactory (1/1) (attempt #0) with attempt id c3e0fe598839c22a55a5801fb667f784 to 192.168.1.188:41941-6be8a1 @ slave1 (dataPort=43852) with allocation id 0377dfed53086efc343ba0f1ce669298 2022-08-17 04:05:22,925 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: hivesinkfactory (1/1) (c3e0fe598839c22a55a5801fb667f784) switched from DEPLOYING to RUNNING. 2022-08-17 04:05:22,928 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: oraclesourcefactory (1/1) (0dcf1185673d44aadc4ed91fa0bdf365) switched from DEPLOYING to RUNNING. 2022-08-17 04:05:23,129 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: hivesinkfactory (1/1) (c3e0fe598839c22a55a5801fb667f784) switched from RUNNING to FAILED on 192.168.1.188:41941-6be8a1 @ slave1 (dataPort=43852). org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat ClassLoader info: URL ClassLoader: Class not resolvable through given classloader. at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:331) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:150) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:519) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_333] Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat at java.net.URLClassLoader.findClass(URLClassLoader.java:387) ~[?:1.8.0_333] at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_333] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_333] at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_333] at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_333] at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2011) ~[?:1.8.0_333] at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1875) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2209) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:508) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:466) ~[?:1.8.0_333] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:317) ~[flink-dist_2.12-1.12.7.jar:1.12.7] ... 6 more 2022-08-17 04:05:23,165 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - Calculating tasks to restart to recover the failed task 0a448493b4782967b150582570326227_0. 2022-08-17 04:05:23,166 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - 2 tasks should be restarted to recover the failed task 0a448493b4782967b150582570326227_0. 2022-08-17 04:05:23,169 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Flink_Job (f65908e085b81cac56dce810005055e2) switched from state RUNNING to FAILING. org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:666) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:446) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_333] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_333] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_333] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_333] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.Actor.aroundReceive(Actor.scala:517) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.Actor.aroundReceive$(Actor.scala:515) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.12-1.12.7.jar:1.12.7] Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat ClassLoader info: URL ClassLoader: Class not resolvable through given classloader. at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:331) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:150) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:519) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_333] Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat at java.net.URLClassLoader.findClass(URLClassLoader.java:387) ~[?:1.8.0_333] at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_333] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_333] at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_333] at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_333] at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2011) ~[?:1.8.0_333] at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1875) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2209) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:508) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:466) ~[?:1.8.0_333] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:317) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:150) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:519) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_333] 2022-08-17 04:05:23,179 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: oraclesourcefactory (1/1) (0dcf1185673d44aadc4ed91fa0bdf365) switched from RUNNING to CANCELING. 2022-08-17 04:05:23,188 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: oraclesourcefactory (1/1) (0dcf1185673d44aadc4ed91fa0bdf365) switched from CANCELING to CANCELED. 2022-08-17 04:05:23,189 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Flink_Job (f65908e085b81cac56dce810005055e2) switched from state FAILING to FAILED. org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:666) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:446) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_333] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_333] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_333] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_333] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.12.7.jar:1.12.7] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.Actor.aroundReceive(Actor.scala:517) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.Actor.aroundReceive$(Actor.scala:515) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.12-1.12.7.jar:1.12.7] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.12-1.12.7.jar:1.12.7] Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat ClassLoader info: URL ClassLoader: Class not resolvable through given classloader. at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:331) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:150) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:519) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_333] Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat at java.net.URLClassLoader.findClass(URLClassLoader.java:387) ~[?:1.8.0_333] at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_333] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_333] at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_333] at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_333] at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2011) ~[?:1.8.0_333] at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1875) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2209) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2454) ~[?:1.8.0_333] at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2378) ~[?:1.8.0_333] at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2236) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1692) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:508) ~[?:1.8.0_333] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:466) ~[?:1.8.0_333] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:317) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:150) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:519) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573) ~[flink-dist_2.12-1.12.7.jar:1.12.7] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_333] 2022-08-17 04:05:23,196 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Stopping checkpoint coordinator for job f65908e085b81cac56dce810005055e2. 2022-08-17 04:05:23,196 INFO org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore [] - Shutting down 2022-08-17 04:05:23,196 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 0dcf1185673d44aadc4ed91fa0bdf365. 2022-08-17 04:05:23,200 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 0dcf1185673d44aadc4ed91fa0bdf365. 2022-08-17 04:05:23,214 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Job f65908e085b81cac56dce810005055e2 reached terminal state FAILED. 2022-08-17 04:05:23,231 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Stopping the JobMaster for job Flink_Job(f65908e085b81cac56dce810005055e2). 2022-08-17 04:05:23,234 INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Suspending SlotPool. 2022-08-17 04:05:23,234 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Close ResourceManager connection c5aefb004df07f11d30802d7bcfa1fb6: Stopping JobMaster for job Flink_Job(f65908e085b81cac56dce810005055e2).. 2022-08-17 04:05:23,234 INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Stopping SlotPool. 2022-08-17 04:05:23,235 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Disconnect job manager [email protected]://flink@glnew:6123/user/rpc/jobmanager_4 for job f65908e085b81cac56dce810005055e2 from the resource manager.

Code of Conduct

LeonYoah avatar Aug 17 '22 08:08 LeonYoah

image 保持环境变量和CHUNJUN-DIST目录结构和我的一致,应该是没有问题的 image

Paddy0523 avatar Aug 17 '22 08:08 Paddy0523

还是那样。。

LeonYoah avatar Aug 17 '22 10:08 LeonYoah

你需要start-cluster.sh 之前,把chunjun 的全部jar包移动或者复制到flink lib的目录下,这样才能找到。 你可以把之前的taskmanager用 stop-cluster.sh 关掉,再执行上述操作试下。

lvyanquan avatar Aug 19 '22 07:08 lvyanquan

你需要start-cluster.sh 之前,把chunjun 的全部jar包移动或者复制到flink lib的目录下,这样才能找到。 你可以把之前的taskmanager用 stop-cluster.sh 关掉,再执行上述操作试下。

现在存在日志类冲突的问题,我试过直接放上去会没有日志

Paddy0523 avatar Aug 23 '22 02:08 Paddy0523