vogels icon indicating copy to clipboard operation
vogels copied to clipboard

How can i get all the records using scan ... now i am getting only 780 items .. I need to scan all 50000 items

Open Ashvin09 opened this issue 8 years ago • 4 comments

Ashvin09 avatar Jun 23 '17 22:06 Ashvin09

It's useful if you can post code when raising issues but I have a feeling that you may be hitting the scan limits set by AWS:

The result set from a Scan is limited to 1 MB per call. You can use the LastEvaluatedKey from the scan response to retrieve more results.

You can use parallelScan(n) to increase the throughput.

clarkie avatar Jun 24 '17 08:06 clarkie

UserModel.scan() .where('email').equals(email.toLowerCase()).exec((err, data) => {

Ashvin09 avatar Jun 24 '17 08:06 Ashvin09

how can i use LastEvaluatedKey ....??? or parallelScan ..?? what is totalSegments in parallelScan and how to get it for records like 1lac

Ashvin09 avatar Jun 24 '17 08:06 Ashvin09

@Ashvin09 Please use Global secondary index on email field, no need of scan operation.

avtaniket avatar Sep 08 '17 14:09 avtaniket