unstorage icon indicating copy to clipboard operation
unstorage copied to clipboard

Inconsistency between clear and getKeys

Open cjpearson opened this issue 1 year ago • 1 comments

Environment

[email protected], node v20.9.0

Reproduction

Currently getKeys and clear behave differently when they are passed a prefix.

const prefix = 'test';
let keys = await storage.getKeys(prefix);
console.log(keys); // ['test:123', 'test:abc']
await storage.clear(prefix);
keys = await storage.getKeys(prefix);
console.log(keys); // ['test:123', 'test:abc']

clear seems to ignore the prefix if it is not a mount, while getKeys finds all keys which begin with the prefix. Since the options are named the same I had assumed they would behave similarly.

Describe the bug

Is this behavior intentional? Currently I'm working around it by individually deleting keys, but it would be nice if clear also worked.

const prefix = 'test';
let keys = await storage.getKeys(prefix);
console.log(keys); // ['test:123', 'test:abc']
await Promise.all(keys.map((k) => this.storage.removeItem(k)));
keys = await storage.getKeys(prefix);
console.log(keys); // []

Additional context

No response

Logs

No response

cjpearson avatar Nov 09 '23 13:11 cjpearson

Thank you for taking the time to enter this issue--I second! I am using Redis with Nuxt. Initially thought that I could make life easier by using clear, but ran into the same.

While I believe the description for clear is technically accurate, i.e., "Removes all stored key/values. If a base is provided, only mounts matching base will be cleared," I do believe that it would be best to provide some kind of "bulk delete" capability driven by a prefix. Either that or some kind of guidance with respect to how to efficiently perform the series of necessary atomic deletes. As it stands, it looks like I will have to do what you are doing and additionally augment with something to limit the concurrency:

export const invalidateCache = async ({ group = getEnv(), name = "", key = "", } = {}) => {
  const prefix = [group, name, key].filter((v) => v != null).map((v) => String(v).replace(/[^\w]/gi, "")).join(":");
  const keys = await useStorage("cache").getKeys(prefix);
  // TODO: impose concurrency limit
  await Promise.allSettled(keys.map(async (k) => {
    console.debug("purging", k);
    await useStorage("cache").removeItem(k, {
      removeMeta: true
    });
  }), );
  // In an ideal world something like the following would work
  // await useStorage("cache").clear(prefix);
};

nuxt.config.ts

nitro: {
    storage: {
      cache: {
        driver: "redis",
        username: process.env.NUXT_REDIS_USERNAME,
        password: process.env.NUXT_REDIS_PASSWORD,
        url: process.env.NUXT_REDIS_URL,
      },
    },
  },

I would imagine that this is a common use case. This could just be ignorance/user error on my behalf (wouldn't be the first time). If anyone is more clueful, I am all ears. Thanks again for this great project. 👍

babalugats76 avatar Feb 28 '24 05:02 babalugats76