aws-sdk-js icon indicating copy to clipboard operation
aws-sdk-js copied to clipboard

How to list objects in a date range?

Open slim-hmidi opened this issue 5 years ago • 14 comments

Confirm by changing [ ] to [x] below:

Describe the question I'm looking to list all the objects stored in S3 bucket between two dates using aws s3 javascript sdk. According to ListObjects there is no parameter allowing to do that except a prefixor delimiter but in my case they are useless. Is there any solution to do that or I have to get the returned data then filter them according to LastModified?

slim-hmidi avatar Feb 19 '20 10:02 slim-hmidi

@slim-hmidi Thank-you for reaching out to us with your issue.

You are right, unfortunately we currently don't have any params to specify a range. You can use the LastModified object returned from the list objects and write a customized function. Please reach out if you need help writing that.

ajredniwja avatar Feb 22 '20 04:02 ajredniwja

@ajredniwja Thanks for your answer. I did it manually. Is there any intention to add a parameter or new api for that in the next version or there is no use case for that?

slim-hmidi avatar Feb 22 '20 12:02 slim-hmidi

The support for an additional param has to come from the service side, so I can open a ticket with them, but due to lack of requests for this particular feature it might sit at the backlog for a while.

ajredniwja avatar Feb 24 '20 22:02 ajredniwja

Interesting #3103 was opened as a feature request for this issue, but then it was closed as a duplicate. I am adding my +1 to this request. Looping through all of the filenames and then filtering by date is very slow for large buckets. We use S3 for access logs and the number of files is huge after only 3 days.

phdunham avatar Jul 29 '20 19:07 phdunham

Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.

github-actions[bot] avatar Jul 30 '21 00:07 github-actions[bot]

Can we please re-open this issue? I'll need this feature soon!

pontusvision avatar Sep 09 '21 12:09 pontusvision

Yes please, this feature could help us to improve performances. Thanks.

montoyadamien avatar Jan 07 '22 13:01 montoyadamien

We would also really like to have this feature!

rbrt avatar Sep 23 '22 21:09 rbrt

Yes please!

rufinoNL avatar Jan 11 '23 17:01 rufinoNL

Last modified isn't very useful for us to filter by, as it doesn't always match the upload+storage date.

Ability to filter by date range would be very useful.

PositiveControl avatar Feb 23 '23 21:02 PositiveControl

+1 Very much needed. How can we otherwise process files uploaded since the previous batch processing date without moving files around?

alexandr2110pro avatar Apr 06 '23 10:04 alexandr2110pro

Yes, this would be very helpful for us as well.

vini-delascio-fmg avatar Jul 26 '23 03:07 vini-delascio-fmg

+1

gonzalovilaseca avatar Nov 14 '23 11:11 gonzalovilaseca

+1

khanhnguyendev avatar Feb 01 '24 01:02 khanhnguyendev