How to precache ALL pages

[ifeanyiisitor]

Hey thanks lots 🙏 for the work you have put into this library. It's indeed giving me life.

One thing I'm wondering is how I would go about precaching all the pages of my app. Let's say I have the following pages

• /
• /about
• /posts

I want all of them to be precached so that they are all available offline once the app has been loaded the first time. At the moment I'm using a custom webpack config to copy the .next/build-manifest.json file over to public/build-manifest. Then once the app loads the first time, I register an activated handler that fetches the build-manifest.json file and then adds them to the cache. It works but it seems like a roundabout way of achieving it and it depends somewhat on implementation details. How would I accomplish the same in a more canonical fashion?

At the moment, my next.config.js file looks like this

const pwa = require('next-pwa')
const withPlugins = require('next-compose-plugins')
const WebpackShellPlugin = require('webpack-shell-plugin-next')

module.exports = withPlugins([
  [
    {
      webpack: (config, { isServer }) => {
        if (isServer) {
          config.plugins.push(
            new WebpackShellPlugin({
              onBuildExit: {
                scripts: [
                  'echo "Transfering files ... "',
                  'cp -r .next/build-manifest.json public/build-manifest.json',
                  'echo "DONE ... "',
                ],
                blocking: false,
                parallel: true,
              },
            })
          )
        }

        return config
      },
    },
  ],
  // NOTE: For some reason, the PWA plugin must come after the shell plugin
  [
    pwa,
    {
      pwa: {
        dest: 'public',
        register: false,
        skipWaiting: true,
      },
    },
  ],
])

And my service worker hook looks like this

import { useEffect } from 'react'
import { Workbox } from 'workbox-window'

export function useServiceWorker() {
  useEffect(() => {
    if (
      typeof window !== 'undefined' &&
      'serviceWorker' in navigator &&
      (window as any).workbox !== undefined
    ) {
      const wb: Workbox = (window as any).workbox

      wb.addEventListener('activated', async (event) => {
        console.log(`Event ${event.type} is triggered.`)
        console.log(event)

        const manifestResponse = await fetch('/build-manifest.json')
        const manifest = await manifestResponse.json()

        const urlsToCache = [
          location.origin,
          ...manifest.pages['/[[...params]]'].map(
            (path: string) => `${location.origin}/_next/${path}`
          ),

          `${location.origin}/about`,
          ...manifest.pages['/about'].map((path: string) => `${location.origin}/_next/${path}`),

          `${location.origin}/posts`,
          ...manifest.pages['/posts'].map((path: string) => `${location.origin}/_next/${path}`),
        ]

        // Send that list of URLs to your router in the service worker.
        wb.messageSW({
          type: 'CACHE_URLS',
          payload: { urlsToCache },
        })
      })

      wb.register()
    }
  }, [])
}

Any help is greatly appreciated. Thanks.

[mattvb91]

Did you manage to make progress on this?

[CodeDoctorDE]

This would be cool. Currently only the home page will be saved.

[CodeDoctorDE]

any updates?

[sfiquet]

I managed to precache all my pages with next-pwa by specifying generateBuildId and pwa.additionalManifestEntries.

This approach is not automatised, you have to build the manifest entries list yourself. But, given the risk of cache bloat, I'm not convinced it's a good idea to precache everything by default anyway.

In a nutshell:

• Decide which HTML and JSON files to precache
  • if you use standard <a> links: precache HTML files only
  • if you use <Link> client-side navigation: precache JSON files and at minimum the HTML file for the start url. Decide whether to precache all HTML files or just have a fallback page.
• Generate the build id yourself (e.g. with nanoid or uuid) and pass it to the Next.js build via generateBuildId.
• Generate the list of entries to precache and pass it to next-pwa via pwa.additionalManifestEntries
  • Use the build id as the revision for HTML entries
  • Include the build id in the url for JSON entries with revision set to null
  • If you want to precache the content of the public folder, you have to do it yourself
• To precache the home page HTML: set pwa.dynamicStartUrl to false(default true puts it in the runtime cache instead). Note that this doesn't precache the JSON.
• Implement as a config function to avoid running your build functions for every single Next.js command

Example for build id OUEmUvoIwu1Azj0i9Vad1: Urls:

HTML	JSON
/	/_next/data/OUEmUvoIwu1Azj0i9Vad1/index.json
/about	/_next/data/OUEmUvoIwu1Azj0i9Vad1/about.json
/posts/myfirstpost	/_next/data/OUEmUvoIwu1Azj0i9Vad1/posts/myfirstpost.json

ManifestEntry objects for /posts/myfirstpost:

HTML:

{
  url: '/posts/myfirstpost',
  revision: 'OUEmUvoIwu1Azj0i9Vad1',
}

JSON:

{
  url: '/_next/data/OUEmUvoIwu1Azj0i9Vad1/posts/myfirstpost.json',
  revision: null,
}

Caveat: There is potential for trouble if you precache too many pages. Keep an eye on the size of your cache. Remember that a single JS file for a dynamic page could translate into thousands of pages, depending on what happens in getStaticPaths.

I wrote a more detailed description in this post: Precaching pages with next-pwa

[L1lith]

This feature would be incredible for me. I'm trying to build a site that is available for people without a consistent internet connection, and I've focused on keeping my website incredibly small so caching the entire site shouldn't be a large problem. It would also be cool if we had an option to trigger this behavior programmatically so that users could press the download button before the service worker goes ham on the network requests.

[petrkucerak]

I found a solution, but it isn't relevant in all scenarios. If your app is small and all your pages are connected via <Link/> from the index page, this can help you. Simply, you can allow in the next.config.js file cacheOnFrontEndNav: true.

But the consequence is that loading your index page will consume more time.

[wevertoum]

any updates?

[kaaax0815]

This issue seems needed. Is there atleast a simple way to precache some sites manually?

[joshua-alday]

We have implemented a method using the additionalManifestEntries option on the nextjs config, but one of the issues we run into is that the cache will not revalidate when updates are deployed. Would be nice to have this feature officially developed with the additional ability to easily revalidate the cache if, say, the deployed build has a different ID from client.

[zhefciad]

Still no updates? We need this