seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [Release] v2.3.4 binary distribution broken in Linux due to CRLF

Open tongyifan opened this issue 1 year ago • 8 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

Deployment from binary distribution not working in Linux due to CRLF

image

SeaTunnel Version

2.3.4

SeaTunnel Config

-

Running Command

export version="2.3.4"
wget "https://archive.apache.org/dist/seatunnel/${version}/apache-seatunnel-${version}-bin.tar.gz"
tar -xzvf "apache-seatunnel-${version}-bin.tar.gz"

cd apache-seatunnel-2.3.4
sh bin/install-plugin.sh 2.3.4

Error Exception

Install SeaTunnel connectors plugins, usage version is 2.3.4
install connector :  connector-amazondynamodb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-assert
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-cassandra
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-cdc-mysql
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-cdc-mongodb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-cdc-sqlserver
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-clickhouse
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-datahub
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-dingtalk
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-doris
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-elasticsearch
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-email
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-ftp
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-hadoop
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-local
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-oss
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-jindo-oss
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-s3
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-file-sftp
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-google-sheets
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-google-firestore
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-hive
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-base
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-feishu
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-gitlab
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-github
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-jira
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-klaviyo
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-lemlist
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-myhours
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-notion
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-onesignal
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-http-wechat
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-hudi
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-iceberg
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-influxdb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-iotdb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-jdbc
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-kafka
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-kudu
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-maxcompute
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-mongodb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-neo4j
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-openmldb
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-pulsar
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-rabbitmq
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-redis
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-s3-redshift
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-sentry
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-slack
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-socket
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-starrocks
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-tablestore
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-selectdb-cloud
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-hbase
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found
install connector :  connector-amazonsqs
bin/install-plugin.sh: line 48: /home/tongyifan/apache-seatunnel-2.3.4/mvnw: cannot execute: required file not found

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

tongyifan avatar Feb 20 '24 17:02 tongyifan

+1,mvnw 脚本格式为windows,非unix格式,即便使用dos2unix转换格式也无法使用。

dzygoon avatar Feb 21 '24 06:02 dzygoon

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

sunhailin-Leo avatar Feb 21 '24 07:02 sunhailin-Leo

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

确实可以了,我完全没想到转换整个目录的文件格式,估计不止是mvnw 这个脚本的问题。

dzygoon avatar Feb 21 '24 07:02 dzygoon

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

确实可以了,我完全没想到转换整个目录的文件格式,估计不止是mvnw 这个脚本的问题。

如果用仓库分支那里构建是没问题的,就是 apache 上的 bin 包有问题~

sunhailin-Leo avatar Feb 21 '24 07:02 sunhailin-Leo

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

确实可以了,我完全没想到转换整个目录的文件格式,估计不止是mvnw 这个脚本的问题。

如果用仓库分支那里构建是没问题的,就是 apache 上的 bin 包有问题~

我用最新分支windows maven构建也是有这个问题的

BranGao19 avatar Feb 21 '24 09:02 BranGao19

@tongyifan @dzygoon

  • Linux 我试了一下用这个命令:find <seatunnel_home 路径> -type f -print0 | xargs -0 dos2unix -- ,然后再执行插件安装就好了
  • 肯定不知道谁提交配置或者脚本的时候给了 windows 的换行符上去。。。

确实可以了,我完全没想到转换整个目录的文件格式,估计不止是mvnw 这个脚本的问题。

如果用仓库分支那里构建是没问题的,就是 apache 上的 bin 包有问题~

我用最新分支windows maven构建也是有这个问题的

修改下./.mvn/wrapper/maven-wrapper.properties的换行符,有人提交代码的时候把win换行符换上了。

dzygoon avatar Feb 22 '24 01:02 dzygoon

Mac上遇到了同样的问题, 换成v2.3.3就正常了

qxhy123 avatar Feb 22 '24 11:02 qxhy123

May be used temporarily, https://dist.apache.org/repos/dist/dev/seatunnel/2.3.4/ But he's temporary, waiting for a new release

Carl-Zhou-CN avatar Feb 28 '24 09:02 Carl-Zhou-CN

@tongyifan @dzygoon @sunhailin-Leo @qxhy123 hi ,the issue has been resolved, please re-download the distribution

Carl-Zhou-CN avatar Mar 01 '24 09:03 Carl-Zhou-CN

@tongyifan @dzygoon @sunhailin-Leo @qxhy123 hi ,the issue has been resolved, please re-download the distribution

good job :D

dzygoon avatar Mar 01 '24 09:03 dzygoon