flink-connector-hbase icon indicating copy to clipboard operation
flink-connector-hbase copied to clipboard

Fix problem due to HBaseRowDataLookupFunction returned to reuse objects

Open Jackson-xky opened this issue 2 years ago • 7 comments

HBaseRowDataLookupFunction.lookup中返回的结果是Collections.singletonList(serde.convertToNewRow(result)),这里默认都是用convertToNewRow方法,内部是在复用同一个对象,当开启了lookup.cache的时候,会导致缓存的结果只都是同一个对象,从而导致结果错乱,由于这里目前不能判断是否使用cache,因此只能默认使用convertToNewRow返回一个新对象来避免上面的情况

Jackson-xky avatar Sep 15 '23 11:09 Jackson-xky

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

boring-cyborg[bot] avatar Sep 15 '23 11:09 boring-cyborg[bot]

Thank you for your contribution! Would you mind creating a bug jira with a description about the bug and then modify this PR with that ticket number and also an English description? Thank you in advance!

ferenc-csaky avatar Sep 27 '23 23:09 ferenc-csaky

HBaseRowDataLookupFunction. The returned result is the Collections in the lookup. SingletonList (serde. ConvertToNewRow (result). The default convertToNewRow method is used here, and the same object is reused internally. When lookup.cache is enabled, the cache results will only be the same object, resulting in confusion. You can avoid this by default by using convertToNewRow to return a new object

Jackson-xky avatar Oct 16 '23 08:10 Jackson-xky

@ferenc-csaky WDYT?

MartijnVisser avatar Nov 13 '23 10:11 MartijnVisser

@Jackson-xky This is a serious problem in the hbase lookup join. Are you still working on it? I'd be happy to help.

Tan-JiaLiang avatar Apr 19 '24 10:04 Tan-JiaLiang

@MartijnVisser @ferenc-csaky It doesn't look like he's had time to deal with the problem, and I'd like to take it on.

Tan-JiaLiang avatar Apr 24 '24 02:04 Tan-JiaLiang

@Tan-JiaLiang I think it is okay to take over. We can keep the separate commits, so the original change will keep its author, even the commit msg can be modified while the original author remains the same.

ferenc-csaky avatar Apr 24 '24 10:04 ferenc-csaky