DataX icon indicating copy to clipboard operation
DataX copied to clipboard

There seems to be a bug in datax's extraction of mongodb data. You need to set the batchSize for each find(). Will cause memory overflow。

Open zhangconan opened this issue 10 months ago • 4 comments

In MongodbReader, data is obtained through dbCursor = col.find(filter).iterator();, but the amount of data obtained in each batch is not set, which will cause memory overflow. Need to be modified like this: dbCursor = col.find(filter).batchSize(1000).iterator();

zhangconan avatar Apr 25 '24 12:04 zhangconan

最近遇到两个mongo的问题了,我会在5月中旬之前修复

FuYouJ avatar Apr 26 '24 08:04 FuYouJ

最近遇到两个mongo的问题了,我会在5月中旬之前修复

GOOD!!!

zhangconan avatar Apr 28 '24 13:04 zhangconan

最近遇到两个mongo的问题了,我会在5月中旬之前修复

你好,请问这两个问题修复了吗?

zhangconan avatar Jul 16 '24 09:07 zhangconan

没 没好的办法

FuYouJ avatar Jul 27 '24 02:07 FuYouJ