apache
[Bug] paimon oss needs Schema file created first
Search before asking
- [X] I searched in the issues and found nothing similar.
Paimon version
0.8
Compute Engine
spark3.4
Minimal reproduce step
i want to test the performance of paimon in aliyun oss ,when i use the following code :
ds.write().format("paimon")
.mode("append")
.option("write.merge-schema", "true")
.option("fs.oss.accessKeyId", accessId)
.option("fs.oss.accessKeySecret", accessKey)
.option("fs.oss.endpoint", ossEndpoint)
.save(basePath);
the console showed me that Exception in thread "main" java.lang.IllegalArgumentException: Schema file not found in location oss://hdtttt/mor3. Please create table first.
at org.apache.paimon.table.FileStoreTableFactory.lambda$create$0(FileStoreTableFactory.java:61)
at java.util.Optional.orElseThrow(Optional.java:290)
at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:59)
at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:45)
at org.apache.paimon.spark.SparkSource.loadTable(SparkSource.scala:85)
at org.apache.paimon.spark.SparkSource.getTable(SparkSource.scala:67)
What doesn't meet your expectations?
is there any options config that solves this problem? can paimon auto-initialize the table first commit, if it did not exist in the specified base path like hudi?
Anything else?
No response
Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
