paimon
paimon copied to clipboard
[Bug] interface for obtaining table information throw exception
Search before asking
- [x] I searched in the issues and found nothing similar.
Paimon version
0.7.0-incubating 1.0.1
Compute Engine
JavaAPI
Minimal reproduce step
step1. create table using version 0.7.0-incubating
package com.hawk.paimon;
import org.apache.paimon.catalog.Catalog;
import org.apache.paimon.catalog.CatalogContext;
import org.apache.paimon.catalog.CatalogFactory;
import org.apache.paimon.catalog.Identifier;
import org.apache.paimon.fs.Path;
import org.apache.paimon.schema.Schema;
import org.apache.paimon.table.Table;
import org.apache.paimon.types.DataTypes;
import java.util.List;
import java.util.concurrent.TimeUnit;
public class CreateTable {
public static String tableName = "paimon_version_07";
public static void main(String[] args) {
CatalogContext catalogContext = CatalogContext.create(new Path("file:///tmp/paimon_version"));
try (Catalog catalog = CatalogFactory.createCatalog(catalogContext)){
Identifier identifier = Identifier.create("test",tableName);
List<String> databases = catalog.listDatabases();
if (!databases.contains(identifier.getDatabaseName())){
catalog.createDatabase(identifier.getDatabaseName(), false);
}
List<String> tables = catalog.listTables(identifier.getDatabaseName());
if (!tables.contains(identifier.getObjectName())){
Schema.Builder schemaBuilder = Schema.newBuilder();
schemaBuilder.column("id", DataTypes.SMALLINT());
schemaBuilder.column("name", DataTypes.STRING());
schemaBuilder.column("age", DataTypes.TINYINT());
// schemaBuilder.primaryKey("id");
schemaBuilder.comment("changelog table");
catalog.createTable(identifier, schemaBuilder.build(), false);
}
TimeUnit.SECONDS.sleep(1);
Table table = catalog.getTable(identifier);
System.out.println("===============================\n" + table.rowType());
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
step2. run the above code again using version 1.0.1 throw exception
Caused by: java.lang.RuntimeException: You should define a 'bucket-key' for bucketed append mode.
at org.apache.paimon.schema.SchemaValidation.validateBucket(SchemaValidation.java:581)
at org.apache.paimon.schema.SchemaValidation.validateTableSchema(SchemaValidation.java:99)
at org.apache.paimon.table.AbstractFileStoreTable.copyInternal(AbstractFileStoreTable.java:351)
at org.apache.paimon.table.AbstractFileStoreTable.copy(AbstractFileStoreTable.java:295)
at org.apache.paimon.table.FileStoreTableFactory.createWithoutFallbackBranch(FileStoreTableFactory.java:127)
at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:90)
at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:80)
at org.apache.paimon.catalog.AbstractCatalog.getDataOrFormatTable(AbstractCatalog.java:423)
at org.apache.paimon.catalog.AbstractCatalog.getTable(AbstractCatalog.java:415)
at org.apache.paimon.catalog.CachingCatalog.getTable(CachingCatalog.java:254)
at com.hawk.paimon.CreateTable.main(CreateTable.java:37)
What doesn't meet your expectations?
Primary key table is ok
APPendOnly table throw exception
version 0.8 and above is ok
Anything else?
because the Schemaserializer #deserialize method performs the following logic when serializing version 0.7 of the appendOnly table schema
if (version <= 1 && !options.containsKey(CoreOptions.BUCKET.key())) {
options.put(CoreOptions.BUCKET.key(), "1");
}
Be changed to
if (schema.primaryKeys().isEmpty() && schema.bucketKeys().isEmpty() && !(schema.version() == TableSchema.PAIMON_07_VERSION && bucket == 1)) {
throw new RuntimeException(
"You should define a 'bucket-key' for bucketed append mode.");
}
No response
Are you willing to submit a PR?
- [x] I'm willing to submit a PR!