seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [seatunnel-core-base] Error when tar directory 'plugins' to file 'plugins.tar.gz'

Open xinda77 opened this issue 2 years ago • 8 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

when run seatunnel job ,i want to know which version can we use 2.1.0 or 2.1.1 . like this simple error Exception in thread "main" java.nio.file.AccessDeniedException: plugins.tar.gz at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at org.apache.seatunnel.utils.CompressionUtils.tarGzip(CompressionUtils.java:62) at org.apache.seatunnel.SparkStarter$ClusterModeSparkStarter.buildCommands(SparkStarter.java:381) at org.apache.seatunnel.SparkStarter.main(SparkStarter.java:98) 0 [main] INFO org.apache.seatunnel.utils.CompressionUtils - Tar directory 'plugins' to file 'plugins.tar.gz'. 1 [main] ERROR org.apache.seatunnel.utils.CompressionUtils - Error when tar directory 'plugins' to file 'plugins.tar.gz'.

SeaTunnel Version

2.1.1

SeaTunnel Config

..............

Running Command

....................

Error Exception

Exception in thread "main" java.nio.file.AccessDeniedException: plugins.tar.gz
		at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
		at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
		at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
		at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
		at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
		at java.nio.file.Files.newOutputStream(Files.java:216)
		at org.apache.seatunnel.utils.CompressionUtils.tarGzip(CompressionUtils.java:62)
		at org.apache.seatunnel.SparkStarter$ClusterModeSparkStarter.buildCommands(SparkStarter.java:381)
		at org.apache.seatunnel.SparkStarter.main(SparkStarter.java:98)
	0 [main] INFO org.apache.seatunnel.utils.CompressionUtils - Tar directory 'plugins' to file 'plugins.tar.gz'. 1 [main] ERROR org.apache.seatunnel.utils.CompressionUtils - Error when tar directory 'plugins' to file 'plugins.tar.gz'.

Flink or Spark Version

2.4.0-cdh6.3.2

Java or Scala Version

1.8

Screenshots

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

xinda77 avatar Apr 25 '22 05:04 xinda77

Please provide your environment, like Spark deployment mode (yarn or standalone), and your submit shell command.

Hisoka-X avatar Apr 25 '22 08:04 Hisoka-X

And please make sure your SeaTunenl package permission is can be access with your start shell command's user.

Hisoka-X avatar Apr 25 '22 08:04 Hisoka-X

cluster mode, why use shell to tar resource ? Don't be so different between versions. Don't refactor stable modules and focus on other directions

xinda77 avatar Apr 26 '22 05:04 xinda77

cluster mode, why use shell to tar resource ? Don't be so different between versions. Don't refactor stable modules and focus on other directions

Compress resource is in Java code. Use shell to call java program. And can you provide permission for plugins.tar.gz and project root path?

Hisoka-X avatar Apr 26 '22 10:04 Hisoka-X

In the user directory(/home), and I can tar resource under the shell, 1.5 X and 2.1.0 are OK. The changes in this part are so big that the test work is not done well. users can't run job with root

xinda77 avatar Apr 27 '22 03:04 xinda77

cluster mode, why use shell to tar resource ? Don't be so different between versions. Don't refactor stable modules and focus on other directions

Compress resource is in Java code. Use shell to call java program. And can you provide permission for plugins.tar.gz and project root path?

Did you develop this code?, Can we go back to 2.1.0? Don't be so fancy on these simple functions that we can't be used

xinda77 avatar Apr 29 '22 02:04 xinda77

cluster mode, why use shell to tar resource ? Don't be so different between versions. Don't refactor stable modules and focus on other directions

Compress resource is in Java code. Use shell to call java program. And can you provide permission for plugins.tar.gz and project root path?

Did you develop this code?, Can we go back to 2.1.0? Don't be so fancy on these simple functions that we can't be used

plugins.tar.gz have a long history, not develop on 2.1.1

Hisoka-X avatar Apr 29 '22 02:04 Hisoka-X

mark

liuhe36 avatar Aug 05 '22 15:08 liuhe36