More Tooling for Static Analysis of Pages

[spiicy-sauce]

Motivation

I'm building a blog, and I want to be able to show a listing page that links to all pages within a posts directory.

Problem

Since posts are just pages, I can theoretically generate a list of page-metadata statically at build-time--I've tested this with import.meta.glob, and it works mostly fine, but I have to manually construct links from the filenames, which is brittle since it's bypassing any custom routing functionality I may create in the future:

async function getPosts(): Promise<PostMetadata[]> {
  const files = import.meta.globEager('/pages/posts/**/*.page.*([a-zA-Z0-9])');
  return map(files, (file, path) => {
    const { title, publishedAt } = file;
    return {
      publishedAt,
      title,
      url: path.replace('index.page.tsx', '').replace('pages/', ''),
    };
  });
}

Obviously I could manually create this list of URLs, but I've found even this hacky solution to be extremely nice from a developer-experience perspective, and would like to see some tooling to support this further.

Proposed solution

Expose an API for accessing routing details for all pre-rendered pages. It seems there may potentially be some overlap with https://github.com/brillout/vite-plugin-ssr/issues/49 here. I'm actually not sure if this is potentially an intractable problem, since this would require knowledge of all pages that will be prerendered (and what their routes would look like) before prerender hooks are called.

[brillout]

I agree there should be an API for that.

In the meantime, have you seen pageContext._pageRoutes? You still have to manually determine the URLs but it will at least provide you with a consistent list of page IDs and their routes.

But a non-internal and higher-level API should be made available to users. E.g.

type PageInstance = {
  url: string,
  pageContext: {
    Page: ReactComponent | VueComponent | unknown
    pageExports: Record<string, unknown>
    // ...
  }
}

const pageInstances: PageInstance[] =
  // `partial: true`: disable `vite-plugin-ssr` warning upon dynamic routes (which cannot be resolved
  //                  without the user providing a list of URLs).
  // `loadPageContext: true` => loads all `.page.js` files
  // `loadPageContext: false` => only loads the `.page.route.js` files
  await pageContext.getPageInstances({ partial: true, loadPageContext: true })

[spiicy-sauce]

Yeah, I saw _pageRoutes, I'll check that out. The only gap with that approach is that pageContext isn't passed to prerender. I assume that it's not passed for good reason, but does that mean that this approach will only work for SSR and not SSG?

[brillout]

I don't see a use case for accessing pageContext.getPageInstances() in prerender() hooks.

If loadPageContext: true then getPageInstances() calls all prerender() hooks.

This means that your sitemap.page.js can call getPageInstances({ loadPageContext: true }) and get the same list of page instances than SSG.

[spiicy-sauce]

Thanks for pointing that out, because it made me realize I was doing something silly, which was using prerender as a proxy for PROD to omit draft post links during prerendering:

// getPosts.ts
async function getPosts(includeUnpublished?: boolean): Promise<PostMetadata[]> {
  const files = import.meta.globEager('/pages/posts/**/*.page.*([a-zA-Z0-9])');
  const all = map(files, (file, path) => {
    const { title, teaser, posterURL, publishedAt } = file;
    return {
      publishedAt,
      teaser,
      posterURL,
      title,
      url: path.replace('index.page.tsx', '').replace('pages/', ''),
    };
  });

  return includeUnpublished
    ? all
    : all.filter(({ publishedAt }) => !!publishedAt);
}

// index.page.server.ts
export { prerender, onBeforeRender };

async function onBeforeRender(pageContext: PageContext) {
  const includeUnpublished = true;
  const posts = await getPosts(includeUnpublished);
  return {
    pageContext: {
      pageProps: {
        ...pageContext.pageProps,
        posts,
      },
    },
  };
}

async function prerender() {
  const includeUnpublished = false;
  const posts = await getPosts(includeUnpublished);
  return {
    url: '/',
    pageContext: {
      pageProps: {
        posts,
      },
    },
  };
}

I updated to a better pattern that no longer requires defining a custom prerender:

// index.page.server.ts
export { onBeforeRender };

async function onBeforeRender(pageContext: PageContext) {
  const includeUnpublished = !import.meta.env.PROD;
  const posts = await getPosts(includeUnpublished);
  return {
    pageContext: {
      pageProps: {
        ...pageContext.pageProps,
        posts,
      },
    },
  };
}

[naveennamani]

Is there any way I can map _pageRoutes.pageId to filePaths? Currently pageContext._pageRoutes contains a list with {pageId, fileSystemRoute, pageRouteFile} where as pageContext._allPageFiles contains a list with {filePath, loadFile}. Currently I'm using condition like filePath.startsWith(pageId) to determine the filePath corresponds to pageId. I think filePath can also be added to pageContext._pageRoutes.

[brillout]

@naveennamani What is it you want to achieve?

[naveennamani]

In some of my pages I'm exporting some metadata and urls of related pages (same as fileSystemRoute in pageContext._pageRoutes). In onBeforeRender hook, I wanted to load metadata of the related pages. Current I was able to figure out pageId of the url by filtering pageContext._pageRoutes. And also, I was able to use loadFile function in pageContext._allPageFiles[".page"] to obtain the pageExports of a .page file I'm interested in. But I was unable to link pageId from pageContext._pageRoutes with filePath from pageContext._allPageFiles.

This is the code I'm currently using

export function onBeforeRender(pageContext) {
  const {_pageRoutes, _allPageFiles, pageExports: {relatedArticles} } = pageContext;
  // search for relatedArticles[0]
 const {pageId, fileSystemRoute} = _pageRoutes.filter(({pageId, fileSystemRoute})=>fileSystemRoute===relatedArticle[0])[0];
 // search for matching filePath
 const {filePath, loadFile} = _allPageFiles[".page"].filter(({filePath, loadFile})=>filePath.startsWith(pageId))[0];
 const {Page, metaData} = await loadFile();
  return { pageContext: {relatedArticles: { metaData}} }
}

Currently I'm using filePath.startsWith(pageId) to check for the filePath corresponding to a fileId. I'll try to give a working example if you require further details.

[brillout]

In general, eagerly loading pages is an anti-pattern for performance & scalability; it goes against Vite's lazy-transpiling approach.

An alternative is to use https://github.com/antfu/vite-plugin-glob with a custom transformer (see https://github.com/antfu/vite-plugin-glob#custom-queries). The transformer would extract the meta data in a performant way. That way we can get meta data for all pages and stay performant & scalable.

[brillout]

Closing because:

In general, eagerly loading pages is an anti-pattern for performance & scalability; it goes against Vite's lazy-transpiling approach.

For those who still want to do this, see https://github.com/brillout/vite-plugin-ssr/issues/49#issuecomment-1570930924:

(It will require to read private pageContext._* properties but we can make them public/stable.)