FluentStorage icon indicating copy to clipboard operation
FluentStorage copied to clipboard

Change S3 AwsS3DirectoryBrowser to apply FilePrefix filtering at the server

Open SRJames opened this issue 2 years ago • 1 comments

Currently the fileprefix is applied client side. If there are many files in a path then this can be very slow.

A small change to AWSS3DirectoryBrowser would allow the filtering to be applied on the server. E.g.

Task ListFolderAsync(List<Blob> container, string path, ListOptions options, CancellationToken cancellationToken)
      {
         var request = new ListObjectsV2Request()
         {
            BucketName = _bucketName,
            Prefix = FormatFolderPrefix(path),
            Delimiter = "/"   //this tells S3 not to go into the folder recursively
         };

         // if we have provided a file prefix, then append it on.
         // as S3 can perform the filtering at source
         if (!string.IsNullOrEmpty(options.FilePrefix))
         {
            request.Prefix += options.FilePrefix;
         }

SRJames avatar Feb 03 '23 14:02 SRJames

Looks good, can you file a PR?

robinrodricks avatar Feb 04 '23 05:02 robinrodricks