meteor-aggregate icon indicating copy to clipboard operation
meteor-aggregate copied to clipboard

How to use $push of $group mongo with data > 16MB?

Open thearabbit opened this issue 5 years ago • 2 comments

I would like to use $push on $group with Million of data. Ex:

data = [
  {item: 'A', date, qty, price, amount..},
  {item: 'A', date, qty, price, amount..},
  {item: 'B', date, qty, price, amount..},
  ...........
]
----------------
Inventories.aggregate([
    {
        $group: {
            _id: "$item", 
            data: {
                $push: {
                    date: "$date",
                    qty:"$qty",
                    price:"$price",
                    amount:"$amount",
                }
            }
        }
    }
])

I got the error Exceeded memory limit for $group, but didn't allow external sort. Pass allowDiskUse:true to opt in. And then I tried

.......([....], { allowDiskUse: true })

I got the error BSONObj size: 19046166 (0x1229F16) is invalid. Size must be between 0 and 16793600(16MB) First element: id: 293825087070.

Could help me???

thearabbit avatar Jan 25 '19 00:01 thearabbit

@thearabbit hay, did you get this resolved? I got a similar issue a while ago which was actually caused by the underlying mongodb driver - an update to meteor/mongo fixed this in my case.

sakulstra avatar Mar 22 '19 17:03 sakulstra

Not yet, Please...

thearabbit avatar Mar 22 '19 22:03 thearabbit