flink-connector-kudu icon indicating copy to clipboard operation
flink-connector-kudu copied to clipboard

Published 20 hours ago • verified publisher icon collabH

Support Flink 1.13.x DynmicTableSource/Sink

Open collabH opened this issue 3 years ago • 15 comments

collabH avatar Dec 17 '21 05:12 collabH

supprot 1.13.x branch：feature_support_with_flink113x!

collabH avatar Dec 18 '21 06:12 collabH

大佬，支持kudu1.10不，编译出错

duokaluo88 avatar Dec 18 '21 10:12 duokaluo88

大佬，支持kudu1.10不，编译出错

支持的改下对应的kudu-client version就行

collabH avatar Dec 18 '21 11:12 collabH

刚试了下，还要将RowDataUpsertOperationMapper将newDeleteIgnore 改回newDelete

duokaluo88 avatar Dec 18 '21 11:12 duokaluo88

嗯嗯，目前flink1.13.x只兼容kudu1.12以上的版本，1.12以下的可以看下readMe去1.12的tag版本然后check pickly flink1.13增加的代码

collabH avatar Dec 18 '21 13:12 collabH

image flink版本1.13.5，用flink sql 创建catalog会出现这样的错误

duokaluo88 avatar Dec 20 '21 05:12 duokaluo88

image flink版本1.13.5，用flink sql 创建catalog会出现这样的错误

看着像是依赖问题

collabH avatar Dec 20 '21 05:12 collabH

image image image 在TableFactoryService里有这么一段判断，但我看这两个说移除了？在KuduCatalogFactory 重新加上后问题解决-。-

duokaluo88 avatar Dec 20 '21 10:12 duokaluo88

不是flink1.13版本的吗，1.13这俩个被过期了的

collabH avatar Dec 20 '21 10:12 collabH

image image image 在TableFactoryService里有这么一段判断，但我看这两个说移除了？在KuduCatalogFactory 重新加上后问题解决-。-

看了，这个flink1.13的tableService有些问题，他升级的catalogFactory提供了新的方法替代了这俩个过期的，但是tableFactoryService里却还是使用的旧的 image

collabH avatar Dec 20 '21 11:12 collabH

请问能提供一个基于sql 的kudu catalog创建示例吗？

Dkbei avatar Feb 09 '22 10:02 Dkbei

请问能提供一个基于sql 的kudu catalog创建示例吗？

CREATE CATALOG catalog_name
  WITH (key1=val1, key2=val2, ...)

这种语法硬就可以，with里的key可以看下KuduCatalogFactory这个类

collabH avatar Feb 09 '22 10:02 collabH

请问能提供一个基于sql 的kudu catalog创建示例吗？

CREATE CATALOG catalog_name
  WITH (key1=val1, key2=val2, ...)

这种语法硬就可以，with里的key可以看下KuduCatalogFactory这个类 https://github.com/collabH/flink-connector-kudu/blob/8aa9400a4b239e4276891b21c7cad7a9cb58cf5c/src/main/java/org/colloh/flink/kudu/connector/table/catalog/KuduCatalogFactory.java#L42

collabH avatar Feb 09 '22 10:02 collabH

请问能提供一个基于sql 的kudu catalog创建示例吗？

CREATE CATALOG catalog_name
  WITH (key1=val1, key2=val2, ...)

这种语法硬就可以，with里的key可以看下KuduCatalogFactory这个类

https://github.com/collabH/flink-connector-kudu/blob/8aa9400a4b239e4276891b21c7cad7a9cb58cf5c/src/main/java/org/colloh/flink/kudu/connector/table/catalog/KuduCatalogFactory.java#L42

多谢，基于kudu1.10.0成功创建

Dkbei avatar Feb 09 '22 13:02 Dkbei

时态表无法关联 Exception in thread "main" org.apache.flink.table.api.ValidationException: Could not determine a type inference for lookup function 'default_catalog.default_database.kudu_source'. Lookup functions support regular type inference. However, for convenience, the output class can simply be a Row or RowData class in which case the input and output types are derived from the table's schema with default conversion.

opoioa avatar Mar 04 '22 02:03 opoioa