amoro icon indicating copy to clipboard operation
amoro copied to clipboard

[Improvement]: Supports float and double type fields as primary keys for Mixed-format tables

Open YesOrNo828 opened this issue 2 years ago • 0 comments

Search before asking

  • [X] I have searched in the issues and found no similar issues.

What would you like to be improved?

create catalog arctic with ( 'type'='arctic', 'metastore.url'='thrift://***:/mixed_iceberg_simple' );

create table arctic.demo.floa (id float, primary key (id) not enforced); create table sour (id float) with ('connector'='datagen'); insert into arctic.demo.floa select * from sour;

Caused by: java.lang.IllegalArgumentException: Cannot add field id as an identifier field: must not be float or double field
at com.netease.arctic.shaded.org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:220)
at com.netease.arctic.shaded.org.apache.iceberg.Schema.validateIdentifierField(Schema.java:130)
at com.netease.arctic.shaded.org.apache.iceberg.Schema.lambda$new$0(Schema.java:106)
at com.netease.arctic.shaded.org.apache.iceberg.Schema.<init>(Schema.java:106)
at com.netease.arctic.shaded.org.apache.iceberg.Schema.<init>(Schema.java:91)
at com.netease.arctic.shaded.org.apache.iceberg.flink.FlinkSchemaUtil.freshIdentifierFieldIds(FlinkSchemaUtil.java:86)
at com.netease.arctic.shaded.org.apache.iceberg.flink.FlinkSchemaUtil.convert(FlinkSchemaUtil.java:68)
at com.netease.arctic.flink.catalog.ArcticCatalog.createTable(ArcticCatalog.java:290)
at org.apache.flink.table.catalog.CatalogManager.lambda$createTable$10(CatalogManager.java:661)
at org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:841)

How should we improve?

Mixed-format table's primary keys can be defined as either float or double via Flink SQL.

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

YesOrNo828 avatar Nov 21 '23 06:11 YesOrNo828