Better documentation on the revalidation behavior of the infinite hook

[grazianodev]

Bug report

Description / Observed Behavior

I have created a simple component that fetches a list of posts via GraphQL requests (cursor-based) using useSWRInfinite. Apparently everything works fine, i.e. the pages are all fetched correctly, in the correct order, etc. but when inspecting the 'Network' tab I can see that every time I load a new page, the first page is also fetched again. So, when I click 'Load more' two requests are made, one for the next page, the other for the first page (unnecessary).

I'm reporting it as a bug because I've seen it happen with the official infinite loading example, although only occasionally (in my app it always happens). I think the problem has something to do with the fact that the 'index' that is passed to 'getKey' correctly increases by 1 when loading more, but on re-render it resets to 0 (this always happen in the official example, too), so when the component re-renders and executes 'getKey' it fetches the first page again because 'index' is 0 again.

Expected Behavior

I would expect that only the request for the next page is made.

Repro Steps / Code Example

Here's my component:

// Index.js 

const Index = ( { posts } ) => {	

  const getKey = ( index, previousData ) => {

    console.log( "index", index ) // 'index' increases by 1 after loading more, then resets to 0
    if( index === 0 ) return [ `/libs/api/posts`, "" ]
    
    return [ `/libs/api/posts`, previousData?.pageInfo?.endCursor ]
  
  }

  const { 
    data, error, mutate, size, setSize, isValidating 
  } = useSWRInfinite( getKey, getAllPosts, { initialData: [ posts ] } )

  return(
    <div>
      <LogInOutLink />
      {
        data.map( ( page, index ) => {
          return(
            <div key={ index }>
              <h3>Page n. { index }</h3>
              <ul>{ page.edges.map( ( { node } ) => <li key={ node.id }>{ node.title }</li> ) }</ul>
            </div>
          )
        })
      }
      <button onClick={ () => setSize( size + 1 ) }>Load more</button>
    </div>
  )	

}

export const getStaticProps = async () => {

  const posts = await getAllPosts( "/libs/api/posts" )

  return { props: { posts } }

}

export default Index

And here's my fetcher:

const getAllPosts = async ( key, cursor = "" ) => {
   
   const query = `
      query getPosts( 
         $after: String, 
         $first: Int = 8 	
      ) {
         postsRoot: contentNodes(	
            after: $after, 
            first: $first, 
            where: { contentTypes: [ POST, NEWS ] } 
         ) {
            ${PAGINATION_FIELDS}
            ${POSTS_LIST_FIELDS}
         }
      }
   `
   
   const variables = cursor ? { after: cursor } : {}

   const data = await fetchAPI( query, variables )
 
   return data?.postsRoot

}

export default getAllPosts

export const fetchAPI = async ( query, variables = {} ) => {
 
   const res = await fetch( `${process.env.NEXT_PUBLIC_BACKEND_BASE_URL}/graphql`, {
      method: "POST",
      headers: {
         "Content-Type": "application/json"
      },
      body: JSON.stringify({
         query,
         variables
      })
   })
 
   const json = await res.json()
   
   if( json.errors ) {
     console.error( json.errors )
     throw new Error('Failed to fetch API')
   }
   
   return json.data

}

Additional Context

I encountered the problem with the latest version (0.3.9), so I also tried with 0.3.0 but to no luck.

[shuding]

Thanks for reporting! That is the expected behavior and let me explain why it happens (and only happens for the first page).

First all SWR hooks will refetch when you open a new page, even if it's cached. This is the same behavior as traditional data fetching / SSR to keep the data fresh. But with useSWRInfinite, we don't want to refetch all pages because it's gonna be a performance disaster. Imagine that you have 100 pages on a route and when hitting "Back", they all refetch at the same time.

So the way useSWRInfinite handles it is only refetch the first page. Because most apps with the infinite loading design work like a timeline (think about FB and Twitter), new changes will mostly come up at the top. With a cursor / offset based pagination, when the first page changes, SWR will naturally refetch the next page and repeat that process. However, if you want to force refetching all pages, there's an option revalidateAll for the useSWRInfinite hook.

So this is actually a way to keep your data up to date and you don't need to worry about.

[grazianodev]

Thank you for explaining. Can validation be "turned off" though? I understand it's an integral part of the library, but I'm creating a pretty traditional blog that will be updated a few times a day at most, so making sure the data is as fresh as it can is not a concern in my use case. I would rather prefer to avoid the extra requests to validate it.

[shuding]

@grazianodev yeah we've considered adding an option to skip revalidation when the cache exists. But we don't have a strong opinion on it. Personally I think it's okay to add it in future versions.

[mohammedhammoud]

@shuding Please do add this option, I encountered the same thing and thought it was a bug.

[shuding]

@mohammedhammoud we now have a new hook in the latest beta version (try it by installing swr@beta):

import useSWRImmutable from 'swr/immutable'

As well as an revalidateWhenStale: boolean option for useSWR.

We are going to document it when it gets finalized in the next stable release.

[Brawl345]

Just run into the same issue and thought it was a bug...

@shuding useSWRImmutable looks great but there is no function for useSWRInfinite? useSWRImmutable is just for the normal, non-page fetching or am I missing something?

[shuding]

@Brawl345 yeah we should have made it clearer in the docs (or as a blog post), gonna reopen this issue for tracking.

If you want to use immutable mode for useSWRInfinite, you can do the following (but it will still revalidate the first page when loading new ones):

useSWRInfinite(..., ..., {
  revalidateIfStale: false,
  // you can also disable focus refetch
})

[yume-chan]

Because most apps with the infinite loading design work like a timeline (think about FB and Twitter), new changes will mostly come up at the top.

However, in a typical timeline there are also things like comments, like count, etc. that need to be updated. useSWRInfinite only updates the first page (if no new post returned), that sounds strange.

I know the existence of { revalidateAll: true}, which I believe is never a good idea. Due to useSWRInfinite only loads pages in sequence (or due to cursor-based pagination can only load pages in sequence), when a user already has tens or hundreds of pages loaded, I can't imagine how long it will take to load the next page. And due to useSWRInfinite only returns data after all pages are loaded, it's basically same as no pagination (or even worse due to extra requests).

If I understand correctly, when there are new items in the first page, even with { revalidateAll: false } (which is the default) it will still reload all pages. I think I don't need to repeat the result.

I'm prototyping a Twitter-like SNS, so I recently checked how Twitter works. It has two cursors, "top" and "bottom", so it can load new tweets from both ends. It also uses server-side events to update like/reshare/comment counts, instead of reloading each pages.

Anyway, I don't expect useSWRInfinite to fit all usages because there are infinite different methods to load an infinite list. But the behavior now is especially strange and not useful.

[shuding]

Thanks @yume-chan, very helpful feedback! I think overall infinite loading is a tricky problem, like you said a solid API implementation like Twitter needs bi-directional cursoring and some pushing mechanism because we can't afford reloading every page, but every page can be changed. { revalidateAll: false } is good for the case that list changes will usually be popped up to the top, and smaller changes for each page such as comments/likes are controlled by useSWR hooks inside that item.

That said, I'd love to do more research for a better API design that:

• Supports the Twitter-like complex use case
• Has the ability to revalidate/mutate a specific page
• Easier to write

[grazianodev]

Hi, coming back to this after a LONG time and I was happy to read that there is the option to disable revalidation now, however it's not working for me :( The first page is still revalidated every time a new page is fetched, even with revalidateIfStale: false

Here's some quick test code that is not working for me:

import { Suspense } from 'react'
import useSWRInfinite from 'swr/infinite'

import ErrorBoundary from "../components/ErrorBoundary"

const fetcher = url => fetch( url ).then( res => res.json() )

const PostsList = () => {

   const { data: posts, error, isValidating, size, setSize } = useSWRInfinite(
      ( pageIndex, previousData ) => {
         return 'http://localhost:3001/api/posts?page=' + (pageIndex+1)
      }   
   , (url) => {
      return fetcher( url )
   }, { 
      initialSize: 1, 
      revalidateIfStale: false,
      revalidateOnMount: false,
      revalidateOnFocus: false,
      revalidateOnReconnect: false,
      suspense: true 
   })

   return(
      <div className="PostsList">
         {
            posts?.map( page => 
               page?.map( post => <h1 key={ post.image.index }>{ post.title }</h1> )   
            )
         }
         {
            isValidating 
            ? <p>Loading posts...</p>
            : <button onClick={ () => setSize(size+1) }>Load posts</button>
         }
         { error && <p>Error!</p> }
      </div>  
   )

}

export default function Home() {

  return (
      <ErrorBoundary>
         <Suspense fallback={ <p>Loading posts...</p> }>
            <PostsList /> 
         </Suspense> 
      </ErrorBoundary>
  )

}

Any ideas? Thank you!

[grazianodev]

My bad, I thought revalidateIfStale was the option that controlled revalidating the first page on every new fetch (not just on mount), then I've discovered about revalidateFirstPage. Everything's working, sorry.