starrocks-connector-for-apache-flink icon indicating copy to clipboard operation
starrocks-connector-for-apache-flink copied to clipboard

[Enhancement] look function support partial cache

Open Jin-H opened this issue 1 year ago • 4 comments

I found that the current connector supports the lookup function, but the implementation is to fully scan the table data and store it in the memory. If the table data is large, the memory usage of flink tm will be relatively high, and it may even be oom. I think we should support caching some data just like jdbc connector. Are there any plans to enhance the lookup function?

Jin-H avatar Oct 14 '23 12:10 Jin-H

Would you like to contribute the feature?

banmoy avatar Oct 17 '23 02:10 banmoy

I'm willing to implement this feature, but it will take some time.

Jin-H avatar Oct 17 '23 06:10 Jin-H

I think it's ok, and first assign this issue to you. Thanks.

banmoy avatar Oct 17 '23 07:10 banmoy

ok

Jin-H avatar Oct 17 '23 07:10 Jin-H