seatunnel
seatunnel copied to clipboard
The maxcompute source does not support the conversion of date type
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
The maxcompute source does not support the conversion of date type ,when batch job running 。
SeaTunnel Version
v2.3.0
SeaTunnel Config
env {
spark.app.name = "seatunnel"
spark.executor.instances = 2
spark.executor.cores = 1
spark.executor.memory = "1g"
// 这个配置必需填写
#spark.sql.catalogImplementation = "hive"
}
source {
Maxcompute {
accessId="accessId"
accesskey="accesskey"
endpoint="http://service.cn-shanghai.maxcompute.aliyun.com/api"
project="noah_ods"
table_name="bak_go_goal_new_t_index_info"
#partition_spec="<your partition spec>"
#split_row = 10000
}
}
sink {
Console {}
}
Running Command
./bin/start-seatunnel-spark-connector-v2.sh --config config/maxcompute2rds1.conf -e client -m 'local[2]'
Error Exception
"ERROR ParallelBatchPartitionReader: BatchPartitionReader execute failed.
org.apache.seatunnel.connectors.seatunnel.maxcompute.exception.MaxcomputeConnectorException: ErrorCode:[COMMON-12], ErrorDescription:[Source reader operation failed, such as (open, close) etc...] - java.lang.ClassCastException: java.util.Date cannot be cast to java.lang.Long
at org.apache.seatunnel.connectors.seatunnel.maxcompute.util.MaxcomputeTypeMapper.resolveObject(MaxcomputeTypeMapper.java:285)"
Flink or Spark Version
spark-2.4.8
Java or Scala Version
scala-2.13.10
Screenshots
No response
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
maxcompute 2.0 date type , You can view this connection . https://help.aliyun.com/document_detail/159541.html
I want to try this
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.