Qingwen Zhao

Results 6 comments of Qingwen Zhao

@jhsenjaliya @tedyu After this upgrade, does we still support HBase 0.98?

Yes. We expect after 0.5, Eagle can support Kafka 0.10 & storm 0.10. There is a concern. We have a coprocessor module in Eagle for HBase 0.98. I am not...

@kaori-seasons My suggestion is we could implement this as a usual sampling UDF, but not only for GNN。 On the other hand, Geaflow has supported the edge iterator, and based...

@huhao0926, which version or branch code do you use? I'm not sure if this issue happens in a batch or stream job. The issue occurs when the output bucket buffer...

正常情况下,outbuffer 超过128MB 就会清空底层 buffer并重构一个新的outBuffer。 这里能写到2GB,有种情况是, 传过来的 ListData 太大了。 你看看source function里, 是不是一个batch 写太多数据了。

> 正常情况下,outbuffer 超过128MB 就会清空底层 buffer并重构一个新的outBuffer。这段逻辑是在哪里保证的; 传过来的 ListData 太大了。我是写的sql的方式,源数据是从kafka消费的新数据,我不是特别了解source function。 目前的本地改动的方式如下,不知道是否合理,有空您也可以帮忙指导下,感谢 for (T datum : data) { this.recordSerializer.serialize(datum, false, outBuffer); // OutOfMemory if (outBuffer.getBufferSize() >= this.maxBufferSize) { // 将判断逻辑写在这里 this.sendBuffer(channel, outBuffer,...