paimon icon indicating copy to clipboard operation
paimon copied to clipboard

[Bug] interface for obtaining table information throw exception

Open hawk9821 opened this issue 10 months ago • 0 comments

Search before asking

  • [x] I searched in the issues and found nothing similar.

Paimon version

0.7.0-incubating 1.0.1

Compute Engine

JavaAPI

Minimal reproduce step

step1. create table using version 0.7.0-incubating

package com.hawk.paimon;

import org.apache.paimon.catalog.Catalog;
import org.apache.paimon.catalog.CatalogContext;
import org.apache.paimon.catalog.CatalogFactory;
import org.apache.paimon.catalog.Identifier;
import org.apache.paimon.fs.Path;
import org.apache.paimon.schema.Schema;
import org.apache.paimon.table.Table;
import org.apache.paimon.types.DataTypes;

import java.util.List;
import java.util.concurrent.TimeUnit;

public class CreateTable {

    public static String tableName = "paimon_version_07";
    public static void main(String[] args) {
        CatalogContext catalogContext = CatalogContext.create(new Path("file:///tmp/paimon_version"));
        try (Catalog catalog = CatalogFactory.createCatalog(catalogContext)){
            Identifier identifier = Identifier.create("test",tableName);
            List<String> databases = catalog.listDatabases();
            if (!databases.contains(identifier.getDatabaseName())){
                catalog.createDatabase(identifier.getDatabaseName(), false);
            }
            List<String> tables = catalog.listTables(identifier.getDatabaseName());
            if (!tables.contains(identifier.getObjectName())){
                Schema.Builder schemaBuilder = Schema.newBuilder();
                schemaBuilder.column("id", DataTypes.SMALLINT());
                schemaBuilder.column("name", DataTypes.STRING());
                schemaBuilder.column("age", DataTypes.TINYINT());
                // schemaBuilder.primaryKey("id");
                schemaBuilder.comment("changelog table");
                catalog.createTable(identifier, schemaBuilder.build(), false);
            }
            TimeUnit.SECONDS.sleep(1);
            Table table = catalog.getTable(identifier);
            System.out.println("===============================\n" + table.rowType());
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

step2. run the above code again using version 1.0.1 throw exception

Caused by: java.lang.RuntimeException: You should define a 'bucket-key' for bucketed append mode.
	at org.apache.paimon.schema.SchemaValidation.validateBucket(SchemaValidation.java:581)
	at org.apache.paimon.schema.SchemaValidation.validateTableSchema(SchemaValidation.java:99)
	at org.apache.paimon.table.AbstractFileStoreTable.copyInternal(AbstractFileStoreTable.java:351)
	at org.apache.paimon.table.AbstractFileStoreTable.copy(AbstractFileStoreTable.java:295)
	at org.apache.paimon.table.FileStoreTableFactory.createWithoutFallbackBranch(FileStoreTableFactory.java:127)
	at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:90)
	at org.apache.paimon.table.FileStoreTableFactory.create(FileStoreTableFactory.java:80)
	at org.apache.paimon.catalog.AbstractCatalog.getDataOrFormatTable(AbstractCatalog.java:423)
	at org.apache.paimon.catalog.AbstractCatalog.getTable(AbstractCatalog.java:415)
	at org.apache.paimon.catalog.CachingCatalog.getTable(CachingCatalog.java:254)
	at com.hawk.paimon.CreateTable.main(CreateTable.java:37)

What doesn't meet your expectations?

Primary key table is ok

APPendOnly table throw exception

version 0.8 and above is ok

Anything else?

because the Schemaserializer #deserialize method performs the following logic when serializing version 0.7 of the appendOnly table schema

if (version <= 1 && !options.containsKey(CoreOptions.BUCKET.key())) {
          options.put(CoreOptions.BUCKET.key(), "1");
 }

Be changed to

if (schema.primaryKeys().isEmpty() && schema.bucketKeys().isEmpty() && !(schema.version() == TableSchema.PAIMON_07_VERSION && bucket == 1)) {
                throw new RuntimeException(
                        "You should define a 'bucket-key' for bucketed append mode.");
            }

No response

Are you willing to submit a PR?

  • [x] I'm willing to submit a PR!

hawk9821 avatar Feb 19 '25 08:02 hawk9821