seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

Published 20 hours ago • verified publisher icon apache

[Bug] [Zeta] restore job failed casued by NPE

Open gitfortian opened this issue 9 months ago • 0 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

1、submit job v2.streaming.conf.template
image

2、then use seatunnel.sh -s to trigger savepoint image 3、restore job with sh bin/seatunnel.sh -r 841558753575698433 1、submit job v2.streaming.conf.template
image

2、then use seatunnel.sh -s to trigger savepoint image 3、restore job with sh bin/seatunnel.sh -r 841558753575698433 May 11, 2024 2:16:22 PM com.hazelcast.internal.config.AbstractConfigLocator INFO: Loading configuration '/mnt/data/sdb1/apps/seatunnel-2.3.4/config/seatunnel.yaml' from System property 'seatunnel. May 11, 2024 2:16:22 PM com.hazelcast.internal.config.AbstractConfigLocator INFO: Using configuration file at /mnt/data/sdb1/apps/seatunnel-2.3.4/config/seatunnel.yaml May 11, 2024 2:16:22 PM org.apache.seatunnel.engine.common.config.SeaTunnelConfig INFO: seatunnel.home is /mnt/data/sdb1/apps/seatunnel-2.3.4 May 11, 2024 2:16:22 PM com.hazelcast.internal.config.AbstractConfigLocator INFO: Loading configuration '/mnt/data/sdb1/apps/seatunnel-2.3.4/config/hazelcast.yaml' from System property 'hazelcast. May 11, 2024 2:16:22 PM com.hazelcast.internal.config.AbstractConfigLocator INFO: Using configuration file at /mnt/data/sdb1/apps/seatunnel-2.3.4/config/hazelcast.yaml May 11, 2024 2:16:22 PM com.hazelcast.internal.config.AbstractConfigLocator INFO: Loading configuration '/mnt/data/sdb1/apps/seatunnel-2.3.4/config/hazelcast-client.yaml' from System property 'haz May 11, 2024 2:16:22 PM com.hazelcast.internal.config.AbstractConfigLocator INFO: Using configuration file at /mnt/data/sdb1/apps/seatunnel-2.3.4/config/hazelcast-client.yaml 2024-05-11 14:16:23,067 INFO [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 res 2024-05-11 14:16:23,141 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5. 2024-05-11 14:16:23,142 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5. 2024-05-11 14:16:23,167 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect 2024-05-11 14:16:23,171 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect 2024-05-11 14:16:23,204 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5. 2024-05-11 14:16:23,205 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated withion: 5.1, local address: /127.0.0.1:39351 2024-05-11 14:16:23,206 INFO [c.h.i.d.Diagnostics ] [main] - hz.client_1 [seatunnel] [5.1] Diagnostics disabl 2024-05-11 14:16:23,217 INFO [c.h.c.i.s.ClientClusterService] [hz.client_1.event-4] - hz.client_1 [seatunnel] [5.1]

Members [1] { Member [hygon-input-006]:5801 - f12ff7c0-cfea-4f75-8e7b-ce2e8ad620c2 }

2024-05-11 14:16:23,244 INFO [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 secon 2024-05-11 14:16:23,396 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5. 2024-05-11 14:16:23,403 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Removed connectionction: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:39351->hygon-input-006/127.0.0.1:1, lastWriteTime=2024-05-11 14:16:23.341, closedTime=2024-05-11 14:16:23.397, connected server version=5.1} 2024-05-11 14:16:23,403 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5. 2024-05-11 14:16:23,406 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5. 2024-05-11 14:16:23,406 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client...... 2024-05-11 14:16:23,406 ERROR [o.a.s.c.s.SeaTunnel ] [main] -

===============================================================================

2024-05-11 14:16:23,406 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Fatal Error,

2024-05-11 14:16:23,406 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Please submit bug report in https://github.com/a

2024-05-11 14:16:23,408 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Reason:SeaTunnel job executed failed

2024-05-11 14:16:23,410 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Exception StackTrace:org.apache.seatunnel.core.s at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:19 at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: java.lang.NullPointerException at sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:77) at sun.nio.fs.UnixPath.(UnixPath.java:71) at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281) at java.nio.file.Paths.get(Paths.java:84) at org.apache.seatunnel.core.starter.utils.FileUtils.getConfigPath(FileUtils.java:49) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:12 ... 2 more

2024-05-11 14:16:23,410 ERROR [o.a.s.c.s.SeaTunnel ] [main] -

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed f at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:19 at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: java.lang.NullPointerException at sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:77) at sun.nio.fs.UnixPath.(UnixPath.java:71) at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281) at java.nio.file.Paths.get(Paths.java:84) at org.apache.seatunnel.core.starter.utils.FileUtils.getConfigPath(FileUtils.java:49) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:12 ... 2 more

SeaTunnel Version

2.3.4 & 2.3.5

SeaTunnel Config

env {
  # You can set SeaTunnel environment configuration here
  parallelism = 2
  job.mode = "STREAMING"
  checkpoint.interval = 2000
}

source {
  # This is a example source plugin **only for test and demonstrate the feature source plugin**
  FakeSource {
    parallelism = 2
    result_table_name = "fake"
    row.num = 16
    schema = {
      fields {
        name = "string"
        age = "int"
      }
    }
  }

  # If you would like to get more information about how to configure SeaTunnel and see full list of source plugins,
  # please go to https://seatunnel.apache.org/docs/category/source-v2
}

sink {
  Console {
  }

  # If you would like to get more information about how to configure SeaTunnel and see full list of sink plugins,
  # please go to https://seatunnel.apache.org/docs/category/sink-v2
}

Running Command

1、nohup sh bin/seatunnel.sh --config config/v2.streaming.conf.template &
2、sh bin/seatunnel.sh -s 841558753575698433
3、sh bin/seatunnel.sh -r 841558753575698433

Error Exception

2024-05-11 14:35:40,621 INFO  [.p.t.AuthenticationMessageTask] [hz.main.priority-generic-operation.thread-0] - [hygon-input-006]:5801 [seatunnel] [5.1] Received auth from Connection[id=9, /127.0.0.1:5801->/127.0.0.1:43219, qualifier=null, endpoint=[127.0.0.1]:43219, remoteUuid=164f61a5-5bff-4cb5-af09-1353cef22cde, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 164f61a5-5bff-4cb5-af09-1353cef22cde, client name: hz.client_1, client version: 5.1
2024-05-11 14:35:40,814 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-0] - [hygon-input-006]:5801 [seatunnel] [5.1] Connection[id=9, /127.0.0.1:5801->/127.0.0.1:43219, qualifier=null, endpoint=[127.0.0.1]:43219, remoteUuid=164f61a5-5bff-4cb5-af09-1353cef22cde, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2024-05-11 14:35:40,814 INFO  [c.h.c.i.ClientEndpointManager ] [hz.main.event-2] - [hygon-input-006]:5801 [seatunnel] [5.1] Destroying ClientEndpoint{connection=Connection[id=9, /127.0.0.1:5801->/127.0.0.1:43219, qualifier=null, endpoint=[127.0.0.1]:43219, remoteUuid=164f61a5-5bff-4cb5-af09-1353cef22cde, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=164f61a5-5bff-4cb5-af09-1353cef22cde, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1715409340620, latest clientAttributes=lastStatisticsCollectionTime=1715409340662,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1715409340600,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=16405725184,os.freePhysicalMemorySize=380497080320,os.freeSwapSpaceSize=1071628288,os.maxFileDescriptorCount=131072,os.openFileDescriptorCount=23,os.processCpuTime=6580000000,os.systemLoadAverage=0.22,os.totalPhysicalMemorySize=540914114560,os.totalSwapSpaceSize=1073737728,runtime.availableProcessors=128,runtime.freeMemory=270340072,runtime.maxMemory=477626368,runtime.totalMemory=324534272,runtime.uptime=1700,runtime.usedMemory=54194200, labels=[]}


2024-05-11 14:16:23,244 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 secon
2024-05-11 14:16:23,396 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.
2024-05-11 14:16:23,403 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Removed connectionction: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:39351->hygon-input-006/127.0.0.1:1, lastWriteTime=2024-05-11 14:16:23.341, closedTime=2024-05-11 14:16:23.397, connected server version=5.1}
2024-05-11 14:16:23,403 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.
2024-05-11 14:16:23,406 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.
2024-05-11 14:16:23,406 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
2024-05-11 14:16:23,406 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 

===============================================================================


2024-05-11 14:16:23,406 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Fatal Error, 

2024-05-11 14:16:23,406 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Please submit bug report in https://github.com/a

2024-05-11 14:16:23,408 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Reason:SeaTunnel job executed failed 

2024-05-11 14:16:23,410 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.s
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:19
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: java.lang.NullPointerException
        at sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:77)
        at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
        at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
        at java.nio.file.Paths.get(Paths.java:84)
        at org.apache.seatunnel.core.starter.utils.FileUtils.getConfigPath(FileUtils.java:49)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:12
        ... 2 more
 
2024-05-11 14:16:23,410 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - 
===============================================================================



Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed f
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:19
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: java.lang.NullPointerException
        at sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:77)
        at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
        at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
        at java.nio.file.Paths.get(Paths.java:84)
        at org.apache.seatunnel.core.starter.utils.FileUtils.getConfigPath(FileUtils.java:49)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:12
        ... 2 more

Zeta or Flink or Spark Version

No response

Java or Scala Version

1.8

Screenshots

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

gitfortian avatar May 11 '24 06:05 gitfortian