goofys icon indicating copy to clipboard operation
goofys copied to clipboard

ls is not consistent when directory has more than 1000 files

Open davedags opened this issue 3 years ago • 1 comments

I know S3's ls max page size is 1000, but I would expect goofys to get around this by iterating over all pages, 1000 at a time, until it fetches all. I saw another issue that said seemed to have fixed this after v 0.20, however I just tried the latest version and 0.23 and the results are still inconsistent.

If I repeatedly do ls | wc -l in a directory that I know has more than 1000, it repeatedly says 1000. However, seemingly random, at some point the command returns the correct count.

Is this a bug?

davedags avatar Mar 28 '21 22:03 davedags

can you post --debug_s3 and --debug_fuse logs? Do you have an example bucket where this happens?

kahing avatar Jul 12 '21 01:07 kahing