my-flink-project icon indicating copy to clipboard operation
my-flink-project copied to clipboard

Published 20 hours ago • verified publisher icon wuchong

商品数量比较大超过Int32 怎么办？

Open yijiang1121 opened this issue 5 years ago • 4 comments

首先很感谢你写的教程和示范代码！ 在HotItems.java中line134，allItems是一个Java List，如果ListState状态中存的商品数目比较大超过Int32时怎么办呢？

yijiang1121 avatar Dec 22 '19 08:12 yijiang1121

@wuchong I did some research and this ticket seems to be related (though not exactly the same). https://issues.apache.org/jira/browse/FLINK-6761

My previous question above was about the number of items you can put into a ListState. What happens if that number is greater than Int32 limitation?

Thanks much!

yijiang1121 avatar Apr 11 '20 07:04 yijiang1121

ListState 底下是个 Java List，所以如果超过 int32，首先 Java 会报异常。 其次，如果 ListState 的总 size 超过了JNI 的上限 （2G?），rocksdb 那边也会报异常。

wuchong avatar Apr 11 '20 10:04 wuchong

谢谢 @wuchong ！如果ListState存的item数目超过int32有什么建议吗？简单回复即可。多谢

yijiang1121 avatar Apr 24 '20 02:04 yijiang1121

这么大的 topn 就不建议用 state 了，直接用数据库去排序吧。

wuchong avatar Apr 24 '20 03:04 wuchong