Option for custom serialization / hashing function

[wouterraateland]

Bug report

Description / Observed Behavior

I'm using SWR with Supabase. The fetching logic is similar for each query, so I'd like to use a single fetcher and use the PostgREST query as the SWR key as follows:

useSWR(supabase.from("table").select("*").eq("id", 1), supabaseFetcher);

The problem with this setup, is that supabase.from("table").select("*").eq("id", 1) gets serialized into a different key each time. Thus, the request won't be cached and will be retried on every render...

Possible Solution

It'd be great if the standard serialize or stableHash function would be extensible. In this case a simple extension to stableHash would do the trick:

const customHash = (arg: any): string => {
  if (arg instanceof PostgrestFilterBuilder) return arg.url.toString();
  else return stableHash(arg)
}

[shuding]

How about using the .url value as the key?

useSWR(supabase.from("table").select("*").eq("id", 1).url, supabaseFetcher);

[wouterraateland]

In this case, using the .url value as the key would work, but is not ideal. The encapsulating PostgrestFilterBuilder provides some nice error handling. By just passing the .url, this error handling would have to be re-implemented.

In general, let O be an object with property O.p that can be serialized in a stable way. Then, I can imagine situations in which it would be hard to reconstruct the functionality of O given O.p. Of course, one could implement some sort of lookup table to find O given O.p, but that would require extra code. I therefor think that passing O as key, with a custom serialization function would be a cleaner solution.

[PortableSteve]

It'd be great if the standard serialize or stableHash function would be extensible.

I've got another case where this would be useful. Sometimes the fetcher returns data that includes some fields that I don't want included in the hash/compare function - for example a timestamp field that always changes. I don't want my component re-rendering if only the timestamp changes. I realise this can be achieved with a custom compare() function, but it would be nice to be able to make use of stableHash().

One option might be to provide an optional transform function that takes the result of the fetcher, and returns the data that should be used by the hash function:

const transform = (data) => {
   const { timestamp, ...rest } = data;
   return rest;
};

Another option might be to expose stableHash() so that it could be called from custom compare() functions.

Also, it would be great to update the documentation which still mentions using dequal by default (I don't believe this is the case any more).

[msdrigg]

This solution would help me as well

Also, @wouterraareland, what does your supabaseFetcher look like? Is there a package for this?

[msdrigg]

So I tried reading through the docs to work out how to work around this. I see the useSWR handler either calls JSON.stringify or toString to hash items. I am wondering if implementing .toJSON or toString on PostgrestFilterBuilder would solve this issue?

[msdrigg]

Ok so I tried my solution and it was not sufficient. It turns out that I was misreading the stableHash code. In reality neither toString or stringify is being called. There is no workaround on the supabase side that I see. It needs to be resolved on the swr side

[shuding]

Do you think if extending the compare API will help?

useSWR(key, fetcher, {
  compare: (a, b, defaultCompare) => {
    const { timestamp: timestamp1, ...rest1 } = a
    const { timestamp: timestamp2, ...rest2 } = b
    return defaultCompare(rest1, rest2)
  }
})

Where the defaultCompare uses the stableHash builtin.

[msdrigg]

Compare wouldn't solve the issue because compare only deals with the returned data. We need a new api that deals with the keys.

The issue is that serialize (which uses stableHash under the hood) needs to have controllable behavior because serialize(args) generates a key that is used to put the data in the cache.

The current behavior of stableHash fails because

stableHash(supabase.from("table").select("*")) != stableHash(supabase.from("table").select("*"))

So when the data comes back from the supabase call, it can't be accessed by the hook because the creates a new key which doesn't see the data that just got returned.

The reason for this behavior is because

supabase.from("table").select("*") !== supabase.from("table").select("*")

It would be sufficient for stableHash to return JSON.stringify(args) on this object because

JSON.stringify(supabase.from("table").select("*")) === JSON.stringify(supabase.from("table").select("*"))

It would, however, be a more global solution for swr to offer an api like this to cover the use cases where JSON.stringify wouldn't work:

useSWR(key: SWRKey, fetcher, {
  hasher: (key, defaultHash) => {
    return defaultHash(key.url)
  }
})

OR, alternatively just this:

useSWR(key: SWRKey, fetcher, {
  hasher: (key) => {
    return key.url.href
  }
})

[msdrigg]

I made a pr that solves this problem for my situation, but I don't know what I need to do to get it ready to merge. Can anybody watching here give some feedback on it?

[shuding]

Thanks for sharing the details and opening a PR!

For SWR's users, I think the best way is still to have a built-in solution for Supabase (and all other popular libs) so there's no need to have extra configurations. I'd love to understand if this can be supported by respecting the toJSON method if the object has it.

[msdrigg]

This would work for supabase usecase because the toJSON method enough information to make supabase queries unique. But I dont think it would work for any object in general.

[psteinroe]

I would argue that JSON.stringify(c.from('table').select('*')) is not stable enough to be used as a hash because it returns everything, including the tokens and realtime connection options.

 {"shouldThrowOnError":false,"url":"https://localhost/rest/v1/table?select=*","headers":{"X-Client-Info":"supabase-js/1.33.3","apikey":"131","Authorization":"Bearer 131"},"schema":"public","_subscription":null,"_realtime":{"accessToken":null,"channels":[],"endPoint":"wss://localhost/realtime/v1/websocket","headers":{"X-Client-Info":"supabase-js/1.33.3"},"params":{"apikey":"131"},"timeout":10000,"heartbeatIntervalMs":30000,"longpollerTimeout":20000,"pendingHeartbeatRef":null,"ref":0,"conn":null,"sendBuffer":[],"serializer":{"HEADER_LENGTH":1},"stateChangeCallbacks":{"open":[],"close":[],"error":[],"message":[]},"reconnectTimer":{"tries":0}},"_headers":{"X-Client-Info":"supabase-js/1.33.3","apikey":"131","Authorization":"Bearer 131"},"_schema":"public","_table":"conversation","method":"GET"}

Summarising the discussions, I guess these are our options

• we would either have to implement toString() or toJSON() on the supabase client,
• have the ability to overwrite stableHash,
• or use some kind of wrapper that allows building the query without the client and composes it within the fetcher, e.g. https://github.com/alfredosalzillo/supabase-swr.

[msdrigg]

I think having the api keys in the hash would not affect stability because different api keys could return a different message. So we need to use the keys in the hash.

Maybe the heartbeat info and connection options could cause problems but I believe thats all

[promer94]

Could this solve the problem ?

const supabaseMiddleware = (useSWRNext) => {
  return (supabaseConfig, fetcher, config) => {
    const swr = useSWRNext(
      supabaseConfig.url.toString(),
      () =>supabaseFetcher(supabaseConfig),
      config
    );
    return swr;
  };
};
useSWR(supabase.from("table").select("*").eq("id", 1), supabaseFetcher, { use: [supabaseMiddleware] });

[msdrigg]

Wait?? Theres middleware? This really seems like it could work. Ill test it out and reply if it does

[promer94]

@msdrigg https://swr.vercel.app/docs/middleware

[psteinroe]

I did a quick test and it seems to work... I cannot believe that we did not see that before. Thanks @promer94!!

import useSWR, {
  BareFetcher,
  Key,
  Middleware,
  SWRConfiguration,
  SWRHook,
} from 'swr';
import { PostgrestError } from '@supabase/supabase-js';
import { PostgrestFilterBuilder } from '@supabase/postgrest-js';
import { SWRResponse } from 'swr/dist/types';

const createSupabaseFetcher = <Type>(
  mode: 'single' | 'maybeSingle' | 'multiple'
) => {
  return async (fb: PostgrestFilterBuilder<Type>) => {
    fb = fb.throwOnError(true);
    if (mode === 'single') {
      const { data } = await fb.single();
      return data;
    }
    if (mode === 'maybeSingle') {
      const { data } = await fb.maybeSingle();
      return data;
    }
    const { data } = await fb;
    return data;
  };
};

const supabaseMiddleware: Middleware = <Type>(useSWRNext: SWRHook) => {
  return (
    key: Key,
    fetcher: BareFetcher<Type> | null,
    config: SWRConfiguration
  ) => {
    const fb = key as PostgrestFilterBuilder<Type>;
    if (!(fb['url'] instanceof URL)) throw new Error('Key is not an instance of PostgrestFilterBuilder');
    if (!fetcher) throw new Error('No fetcher provided');
    fb['url'].searchParams.sort();
    return useSWRNext(
      fb['url'].toString().split('/rest/v1/')[1],
      () => fetcher(fb),
      config
    );
  };
};

function useSWRSupabase<Type>(
  query: PostgrestFilterBuilder<Type>,
  mode: 'single',
  config?: SWRConfiguration
): SWRResponse<Type, PostgrestError>;
function useSWRSupabase<Type>(
  query: PostgrestFilterBuilder<Type>,
  mode: 'maybeSingle',
  config?: SWRConfiguration
): SWRResponse<Type | null, PostgrestError>;
function useSWRSupabase<Type>(
  query: PostgrestFilterBuilder<Type>,
  mode: 'multiple',
  config?: SWRConfiguration
): SWRResponse<Type[], PostgrestError>;
function useSWRSupabase<Type>(
  query: PostgrestFilterBuilder<Type>,
  mode: 'single' | 'maybeSingle' | 'multiple',
  config?: SWRConfiguration
): SWRResponse<Type | Type[], PostgrestError> {
  return useSWR(query, createSupabaseFetcher<Type>(mode), {
    ...config,
    use: [...(config?.use ?? []), supabaseMiddleware],
  });
}

export { useSWRSupabase };

[psteinroe]

Just a quick update: I created an entire cache library based on the poc above. It comes with support for queries, pagination queries, infinite scroll queries, mutations and subscriptions. Would love to get some feedback! https://github.com/psteinroe/supabase-cache-helpers