storage icon indicating copy to clipboard operation
storage copied to clipboard

Add `exists` method to storage

Open caseycrogers opened this issue 2 years ago • 18 comments

I would like to be able to check if a file exists in Supabase storage without having to download it: final bool fileExists = await supabase.storage.from('some-bucket-id').exists('/some/path');

caseycrogers avatar Aug 09 '22 22:08 caseycrogers

There is currently no api enpoint for this. See the documentation. For the time being, you can create your own postgres function, which queries the storage.objects table.

Vinzent03 avatar Aug 10 '22 19:08 Vinzent03

@caseycrogers Thanks for the feedback!

As @Vinzent03 has said, there is currently no APIs for that, so I would recommend creating an issue on the Storage API repo.

dshukertjr avatar Aug 11 '22 08:08 dshukertjr

Come to think about it, what is the use case for this method? In what scenario would you not know if a file exists or not?

The way use storage is that after I upload a file, I either save the public URL or the path to the file in a database, and when I need to access the file, I would just access it via the saved URL or generate a URL from the saved path.

dshukertjr avatar Nov 18 '22 09:11 dshukertjr

You may have paths, where you don't have to save the path, because it's not random, but constructed from other parameter. For example, you may have a directory with profile images, where the file name is the uid of the user. You don't have to specifically save the filename, because you already know the uid.

Vinzent03 avatar Nov 18 '22 10:11 Vinzent03

@Vinzent03 In that example, if they had saved the path on the database, they would save 1 extra http request, hence would be able to speed up whatever process that follows.

I fear that adding a exists method might steer the developers toward bad practices, where they would start making extra requests that can be mitigated, but what do you think?

dshukertjr avatar Nov 18 '22 11:11 dshukertjr

@dshukertjr I'm not sure if you fully understood my use case. Let's say profile images are stored in a predefined directory with the user's uid as the file's name. How do you know on the client whether you already uploaded an image or not? I think the only ways are providing an extra method in the client library, or the user uses a head request and checks for 404.

Vinzent03 avatar Dec 07 '22 21:12 Vinzent03

@Vinzent03 I think I do understand the use case now. I was referring to a use case where you want to display other users' profile images, but I believe you are referring to a use case where you are displaying the logged in user's own profile image.

At least for now, there are some workarounds like doing a head request or to use a erorrBuilder on the profile image to show a fallback image in case there are no profile image set yet!

Let's keep this issue open to see if more users demand this feature!

dshukertjr avatar Dec 08 '22 11:12 dshukertjr

Ran into this same exact use case today, so just want to give my +1 to this feature getting implemented.

While I do have a fallback image, I'd rather not show the request error in the console if it could be avoided.

EDIT:

Just wanted to include the database postgres function call I created for those stuck on this:

BEGIN RETURN
  (SELECT EXISTS
     (SELECT id
      FROM storage.objects
      WHERE bucket_id='avatars'
        AND OWNER=UID
        AND name=PATH
      LIMIT 1));

END;

And the call to it:

  const { data, error } = await supabase.rpc('storage_avatar_exists', {
    uid: '542eb4f2-c890-47d0-a32c-4ebba849a011',
    path: '542eb4f2-c890-47d0-a32c-4ebba849a011/avatar.png'
  });

jherman avatar Dec 14 '22 18:12 jherman

Transferring this issue to the storage-api repo to discuss its implementation on the backend.

dshukertjr avatar Jan 20 '23 03:01 dshukertjr

We already have a backend method to check the existence of an object, simply using a HEAD request instead of a GET on a specific object will return empty content if the item exists else 404

We should probably update the client SDK to implement this method

fenos avatar Jan 23 '23 09:01 fenos

Running into this issue. User Avatars. Bucket is a public bucket, so I do not generate the url to save it to db.
I generate the url dynamically, baseStorageurl + sessionid + avatar.jpg

I want to check if the folder exists before i create the img source url like above.

shriharip avatar Apr 06 '23 09:04 shriharip

We already have a backend method to check the existence of an object, simply using a HEAD request instead of a GET on a specific object will return empty content if the item exists else 404

We should probably update the client SDK to implement this method

tried now using HEAD on a deleted file, still 200. tried GET, got 400, with status 404 on json message image

revulvgina avatar Jun 19 '23 16:06 revulvgina

And why is this yet not available?

softmarshmallow avatar Jan 28 '24 12:01 softmarshmallow

Authed HEAD request for reference

const response = await fetch(`${process.env.SUPABASE_URL}/storage/v1/object/${bucket}/${path}`, {
  method: 'HEAD',
  headers: {
    authorization: session.access_token,
  },
});

phoenixbox avatar Mar 05 '24 22:03 phoenixbox

This will be a useful method. It looks like Google Cloud Storage (and by extension, Firebase Storage?) has a similar method.

utkarsh1097 avatar May 31 '24 20:05 utkarsh1097

This is 100% needed, basic feature of storage that doesn't exist yet

Owez avatar Jun 22 '24 18:06 Owez

Not a good way yo handle things, but for those who does not care about performance that much and looking for batch types of work, heres some extended utilities.

import type { SupabaseClient } from "@supabase/supabase-js";
import type { FileObject } from "@supabase/storage-js";

export namespace SupabaseStorageExtensions {
  type StorageClient = SupabaseClient["storage"];

  async function list(storage: StorageClient, bucket: string, path: string) {
    const { data, error } = await storage.from(bucket).list(path);

    if (error) {
      throw error;
    }

    return data;
  }

  export async function tree(
    storage: StorageClient,
    bucket: string,
    path = ""
  ) {
    let files: Record<string, FileObject> = {};
    let stack: string[] = [path];

    while (stack.length > 0) {
      const currentPath = stack.pop();
      const items = await list(storage, bucket, currentPath!);

      const promises = items.map(async (item) => {
        const itemPath = currentPath + "/" + item.name;
        const isfile = !!item.id;
        if (isfile) {
          files[itemPath] = item;
        } else {
          stack.push(itemPath);
        }
      });

      await Promise.all(promises);
    }

    return files;
  }

  export async function rmdir(
    storage: StorageClient,
    bucket: string,
    path: string
  ) {
    const files = await tree(storage, bucket, path);
    const paths = Object.keys(files);
    const { data, error } = await storage.from(bucket).remove(paths);

    if (error) {
      throw error;
    }

    return data;
  }

  export async function exists(
    storage: StorageClient,
    bucket: string,
    path: string
  ) {
    const { data, error } = await storage.from(bucket).list(path);

    if (error) {
      throw error;
    }

    return data.some((file) => file.name === path.split("/").pop());
  }
}

Alternatively, you can use createSignedUrl, it will return error when used on non-exist object.

softmarshmallow avatar Jun 26 '24 09:06 softmarshmallow

@dshukertjr

I demand this feature 😛 I am intentionally avoiding saving a path_to_file and instead I have a uid for profiles, that I then construct whenever I need it.

getPublicUrl may only be constructing a URL and not actually querying, but maybe querying internal storage schema and store something like result.data.exists?

if (profiles) {
	for (let i = 0; i < profiles.length; i++) {
		const result = supabase.storage
			.from('profiles')
			.getPublicUrl(`/profiles/${profiles[i].id}/preview.png`, {
				transform: t200by200
			});
		
		// getPublicUrl may only be constructing a URL and not actually querying, 
		// but maybe query internal storage schema and store something like `result.data.exists`
		console.log(result.data.exists);
		if (result.data.exists) {
		         profiles[i]['image_url'] = result.data.publicUrl;
		}
	}

	profiles = data;
}

Although I can definitely see how this feature could be abused and result in way too many API calls just to check if they should put a URL for an image into a component. It's way cheaper to just leave an img_url even if it fails than query each image and check if it is available.

PaperPrototype avatar Aug 12 '24 18:08 PaperPrototype