Idea: Next Style SSR rendering with almost zero effort

[LionsAd]

Note: I know that all this would apply more to WMR by now, but the documentation is not as helpful regarding providing a list of URLs like here.

Note 2: My background is Drupal + CMS and I have a list of pages to pre-render, but I don't know in advance what a route component will render, because passing the data to the pre-render array is inefficient.

Note 3: Hi @developit ;)

What is the current behaviour?

• Prerendering happens inside the webpack build, that makes ISG difficult / impossible (prerender command, like in the experimental fast renderer)
• Prerendering needs the page data before it renders the page to work fully
• Prerendering via express would be possible, but is again hard as it would need the page data in advance to ensure the data can come via __PREACT_CLI__DATA__

What is the motivation or use case for changing this behaviour?

I want to be able to:

• Supply a list of URLs
• Run render and have the data that was rendered end up in PREACT_CLI__DATA

Describe the solution you'd like

The __PREACT_CLI__DATA__ is essentially a page cache for the data of the page. But that means you don't need it in advance as long as a wrapper hook for usePrerenderData is used, which works as follows:

function MyComponent(props) {
  const cid = props.id;

  const resolveCacheMiss = useCallback(() => {
    const [data, loading, error] = useDataLoader(props.id, props.type)

    if (error || loading) {
      return [data, loading, error];
    }

    const [processed, transformError] = transformMyComponent(data);

    return [processed, loading, transformError];
  }, [props.id, props.type]);

const [data, loading, error] = useStaticPageCache(cid, resolveCacheMiss)

if (error) { return <div>Error ...</div> }

if (loading) { return <div>Loading ...</div> }

return data;

by doing so the static page cache can essentially get the data from usePrerenderData OR retrieve it via resolveCacheMiss.

While in SSR context, it will always be a cache miss (that could be changed obviously) and is for the case that the API to run resolveCacheMiss is not available online.

function useStaticPageCache(cid, resolveCacheMiss) {
   cache = usePageCacheContext();
   
   if (ssr) {
     const [data, loading, error] = resolveCacheMiss();

     if (!error && !loading) {
       cache.setData(cid, data);
     }
     
     return [data, loading, error];
   }

   const [data, loading, error] = usePrerenderData(cache.props)

  return [data, loading, error]
}

The last piece of the puzzle is now just the PageCacheContext.Provider and here it get's a little bit tricky as I need to use a specific provider value.

const PageCache = createContext();

usePageCacheProvider(url, props) {
  const [state, setState] = useState(props);

  cache = { props }
  cache.setData = (cid, data) => {
    newState = state;
    newState[cid] = data;
    setState(state)
  };

  useEffect(() => {
    return () => async {
      await writeCache(url, state);
    };
  });

  return 
}

return function App(props) {
  cache = usePageCacheProvider(props.url, props);
  return <PageCache.Provider value={cache}>
}

License for all the above code: MIT

But then at the end, the whole request data gets stored. Now ideally it would directly flow into:

__PREACT_CLI__DATA__

but because that is not possible right now, the best alternative is to just pre-render everything twice and for express to just inject the CLI_DATA into the page template and write the __preact_render_data.json into the right directory.

The beauty of the whole Architecture is: It works regardless if in an express context or pre-render context and for the user they only need to change their code minimally.

While I can implement that obviously for my own projects or push to npm at some point, I think it directly fits preact's style of creating really useful, but really small and fast utilities that enable awesome functionality.

Thanks for reading!

[LionsAd]

Some problems with the idea (but it all works out still):

• preact-render-to-string does not support async data loading, so I needed to use sync-fetch for now
• I need to run it twice, to first populate the build-cache and then to render the page finally (but that works well)
• The --prerender is really slow ...

To solve that easily use this script, runs in ~ 60-70 ms instead of several seconds for a few pages once the data-build-cache is populated.

const prerender = require('preact-cli/lib/lib/webpack/prerender.js');
const urls = require('./prerender-urls.js');
const fs = require('fs');

const cwd = './build/';
const dest = './build/'
const src = './src/';

const RGX = '<script type="__PREACT_CLI_DATA__">%7B%22preRenderData%22:%7B%22url%22:%22/200.html%22%7D%7D</script>';
const template = fs.readFileSync('build/200.html', 'utf8');

function getPrerenderValues(url, routeData) {
  const values = {
    url, 
    CLI_DATA: { preRenderData: { url, ...routeData } },
  }

  return values;
}

function renderContent(url, routeData) {
  const values = getPrerenderValues(url, routeData);

  return prerender({ cwd, dest, src }, values);
}

function getPath(url, filename) {
  let path = url.endsWith('/') ? url : url + '/';
  if (path.startsWith('/')) {
    path = path.substr(1);
  }

  return path + filename;
}

function getPrerenderDataFile(url, routeData) {
  const data = JSON.stringify(routeData);
  const path = getPath(url, 'preact_prerender_data.json');

  return { path: dest + path, data: data};
}

function renderPrerenderData(url, routeData) {
  const values = getPrerenderValues(url, routeData);
  const cliData = values.CLI_DATA || {};
  const data = JSON.stringify(cliData);

  return `<script type="__PREACT_CLI_DATA__">${ encodeURI(data) }</script>`;
}

function renderPage(url, routeData) {
  const body = renderContent(url, routeData);
  const prerenderScript = renderPrerenderData(url, routeData);
  return template.replace(RGX, body + prerenderScript);
}

async function main() {
  const urlsData = await urls();
  const pages = urlsData.map((values) => {
  const { url, title, ...routeData } = values;

    return {
      url: url,
      data: renderPage(url, routeData),
    };
  });

  pages.map((page) => {
    // @todo Normalize + async write
    try {
      fs.mkdirSync('./build/' + getPath(page.url, ''));
    }
    catch (e) {
      // Ignore
    }

    fs.writeFileSync('./build/' + getPath(page.url, 'index.html'), page.data);
    return null;
  });
}

main()

[psabharwal123]

@LionsAd what part makes it faster than the pretender currently done by preact cli? Trying to figure out if we can use this

[psabharwal123]

Also with a lot of different pages wouldn't render twice slow it down?

[LionsAd]

@psabharwal123 It does not use webpack - webpack makes the prerendering really really slow.

Only difference in output files is that I do not inline the CSS (but you could run that plugin separately if really needed).

If --prerender works well for you but is just too slow, then the above will just work the same as usual.

e.g. I run:

$ NODE_ENV=production preact build
Done in 8.13s.

$ NODE_ENV=production preact build && node prerender.js
Done in 8.23s.

$ NODE_ENV=production preact build --prerenderUrls prerender-urls.js
Done in 13.14s

$ node prerender.js
Done in 0.05s.

So the above script takes 0.05 seconds instead of ~ 5 seconds for 10 items, so the script is ~ 1000x faster.

Using webpack for the pre-rendering is REALLY slowing preact down.

If you rely on my new useStaticPageCache hook above, then yeah we would need to re-render twice, but I am fixing that right now, but overall preact is so fast that rendering twice does not matter.

[rschristian]

Using webpack for the pre-rendering is REALLY slowing preact down.

I really doubt this is the case, and your benchmarks certainly haven't shown it. Your script is faster because you bypass a ton of extra work that Preact-CLI does over your built HTML. You lose all the power of html-webpack-plugin and you're not even running a minifer over the result by the looks of it. Terser is great at what it does, but it certainly values size over speed. Disabling it does speed things up significantly.

This is an apples-to-oranges comparison at best and I wouldn't regard those times as being realistic given all the post-build work one would need to do to create an a fair comparison.

Still, happy to review PRs if you have a solution (that doesn't drop a ton of functionality). There certainly are ways to improve; parallelizing as done in the experimental PR would certainly help.

[LionsAd]

@rschristian It is the out of the box comparison time, which is always the first benchmark for users of a framework, e.g. me. If I tell my manager: It will take 450 seconds to render 1000 pages (compared to 0.45 seconds with this approach), he'll bail out of preact, which is the problem I am trying to solve right now.

Except for CSS inlining, the HTML is for my use case 100% identical (compared all pages of my app), so I am not seeing what html-webpack-plugin really brings to the table here.

How can I disable terser and all optimizations of webpack that make the SSR slow?

Edit: Also for SSR via express() you would surely not suggest to use the html-webpack-plugin - right?

[rschristian]

It is the out of the box comparison time, which is always the first benchmark for users of a framework, e.g. me.

That's a bit of a horrifying thing to say, to be honest. You should be prioritizing UX over DX. That's the point of web dev, to provide a user experience. Devs are not the primary users.

Faster doesn't mean better for end users.

If I tell my manager: It will take 450 seconds to render 1000 pages (compared to 0.45 seconds with this approach), he'll bail out of preact,

Preact != Preact-CLI. Totally different things. You can use Preact with all sorts of build tools (or none at all!)

Certainly do whatever you need, but to call this a "fix" while cutting features to the bone is... odd. Totally can work in many situations, I have no doubt, but it's no where near equivalent so the benchmark times are junk. One is timing "a", the other is "a + b + c + d". Apples to oranges.

Except for CSS inlining, the HTML is for my use case 100% identical (compared all pages of my app), so I am not seeing what html-webpack-plugin really bring to the table here.

Here's preact-www's template (which powers https://preactjs.com). Anything that's specific to a route (or not generalizable to add to index.html) wouldn't work with your solution above.

Out-of-the-box, you lose out on preloading route assets as well as a ton of fine-grained control over your template.

How can I disable terser and all optimizations of webpack that make the SSR slow?

Use your preact.config.js to alter the Webpack config.

Edit: Also for SSR via express() you would surely not suggest to use the html-webpack-plugin - right?

If dynamic SSR is your need (rather than prerendering which can also be "SSR", but I digress) then I wouldn't suggest Preact-CLI at all. It doesn't make much sense to try to retroactively add it into CLI when A) we don't support it, so your solution could break at any time and B) there's a ton of established ecosystems out there that better support it.

While NextJS certainly has recurring issues with Preact, it does work most of the time I believe. CLI just isn't built for that. That's not the primary use case.

[LionsAd]

@rschristian

That's a bit of a horrifying thing to say, to be honest. You should be prioritizing UX over DX. That's the point of web dev, to provide a user experience. Devs are not the primary users.

Then --prerender is buggy, because the output of my fast build is the same as the default output. There is no route specific output and in no pre-rendered file (out of the box configuration, NODE_ENV=production) is any route bundle even referenced. And I an not doing anything special, when using the web, the route splitting all works.

As of now as said, the whole output is the same (except for minified CSS).

Out-of-the-box, you lose out on preloading route assets as well as a ton of fine-grained control over your template.

I mean, a template is just some template literals, but I am not seeing for letting someone modify a template for the HTML output, why do I need webpack for a configurable title, or ...?

And preloading route assets does not work for me right now.

Use your preact.config.js to alter the Webpack config.

I know how to do that, but not what to disable to make it fast. Could you show me the option to make webpack faster?

If dynamic SSR is your need (rather than prerendering which can also be "SSR", but I digress) then I wouldn't suggest Preact-CLI at all.

I am not sure I agree, your ssr-bundle.js works nicely with express, what is so bad to use what I use to pre-render pages to serve them to the user?

[rschristian]

Then --prerender is buggy, because the output of my fast build is the same as the default output. There is no route specific output

Apologies, apparently our preload is behind a flag. Odd.

But again, default output != all flags and configuration an end user could do, hence why we use html-webpack-plugin. Less moving pieces.

(out of the box configuration, NODE_ENV=production)

Setting that env var does nothing in CLI, so unless you're consuming that, it can safely be removed. Just thought I'd mention that.

I mean, a template is just some template literals, but I am not seeing for letting someone modify a template for the HTML output, why do I need webpack for a configurable title, or ...?

Sounds like you didn't glance at the linked file, it's certainly not "just some template literals".

Could you do it post-build? Probably, though it'd be quite the pain. Being able to reference assets from the Webpack build in the EJS template is pretty powerful and useful.

There's a reason why templating engines exist, and not everything is done with regex matches after all. Not everyone needs them though, and that's totally fine.

I know how to do that, but not what to disable to make it fast. Could you show me the option to make webpack faster?

There is no "the option", you need to go plugin-by-plugin disabling anything that processes the output HTML (start with html-webpack-plugin) in order to make an equivalent test.

I am not sure I agree, your ssr-bundle.js works nicely with express, what is so bad to use what I use to pre-render pages to serve them to the user?

waves at this thread

If it works for you, then great. But I wouldn't be recommending it. There's just a lot more ergonomic solutions out there that have guaranteed stability for that sort of use case. Preact-CLI doesn't provide that. The output files really aren't part of our public API, which means it could change and break your setup. I don't foresee that it would, but using intermediary output from another tool is quite risky, and that's why I could not in good conscience recommend it.

Not trying to tell you what to do or anything, just making sure it's clear that your solution is far from a 1:1 and has some degree of risk to it. I'm sure there will be some who come along and can use this, so thanks for providing it! Just want to provide the warnings that weren't included in your comments.

[LionsAd]

Apologies, apparently our preload is behind a flag. Odd.

Ahhh - got it. :) Thanks for that.

I think as long as routes are known, this can be done without webpack, too.

But again, default output != all flags and configuration an end user could do, hence why we use html-webpack-plugin. Less moving pieces.

Yes, I generally agree. I think for a solution that supports both pre-render and SSR (with preact + ssr-bundle.js) it would be best to render / pre-render outside of a webpack context though and essentially generate a template from the first render to then fill in the blanks later, e.g. everything that comes from prerenderData. Also helmet support would be nice.

And if routes are mapped in a .json file (e.g how the bundle does the references), then this can be supported as well. Worst case with one 200.html template file per route.

Before doing all that however I'll be looking at WMR as that seems to be the future anyway and does not use webpack.

Not trying to tell you what to do or anything, just making sure it's clear that your solution is far from a 1:1 and has some degree of risk to it. I'm sure there will be some who come along and can use this, so thanks for providing it! Just want to provide the warnings that weren't included in your comments.

Of course, totally understood. I am a maintainer of Drupal 7, which powers still ~ 500k sites, so I am well aware of the risk changes bring, but I also believe strongly in sharing is caring. If I were to publish that on npm or such I would certainly not depend on the internals of preact-cli.

I know understand your concerns somewhat better, but yes the script as is was not meant for production usage right now, but just as a starting point for anyone that can live with the limitations that not having control over the template brings - including that --preload won't work.

[rschristian]

I meant to reply to your original comment re:WMR, but forgot to apparently.

I know that all this would apply more to WMR by now, but the documentation is not as helpful regarding providing a list of URLs like here.

WMR discovers routes automatically on the page, so if it prerenders / and finds a <a href="/foo">, it'll also prerender /foo, discovering additional links along the way.

However, you can also add additional links for it to prerender, which is useful for non-discoverable pages like a /not-found. This is just an array added to your config file, see: https://wmr.dev/docs/configuration#customroutes

Before doing all that however I'll be looking at WMR as that seems to be the future anyway and does not use webpack.

I should warn that we haven't been able to maintain WMR as much as we'd otherwise have liked to. If you do like the Preact-CLI prerendering experience, it's probably the easiest transition, but WMR has pretty poor support for CJS packages (which, if you're relying on preact/compat, is a whole ton of the React ecosystem as React still is CJS-only). You may want to look into vite instead, which we do have a preset for.

WMR may work great, just want to give a warning. It's much more limited in compatibility than Preact-CLI here.

[psabharwal123]

@rschristian you had previously reccomended to look into wmr https://github.com/preactjs/preact-cli/pull/1501#issuecomment-862763299 but you you mentioned above it has not been maintained. We rely on preact/compat and I am sure a lot of other folks do as well. Is there a plan to look into the parallelization of ssr that @prateekbh had submitted?

[rschristian]

you had previously reccomended to look into wmr https://github.com/preactjs/preact-cli/pull/1501#issuecomment-862763299 but you you mentioned above it has not been maintained.

Yes, as in June, WMR was seeing a lot more development and the story was getting better and better quick. It's stalled a bit, however, and we're still trying to figure out what we want to do re:build tooling. The tools we have certainly are usable and do see quite a bit of use, but for our recommendation, the water is murky. Vite is my own suggestion, not necessarily that of the PreactJS org, so do with that what you will.

Is there a plan to look into the parallelization of ssr that @prateekbh had submitted?

PRs are welcome if you want to contribute. I'd be happy to review.

There are very few maintainers who've done any work on this in the past year and our time is quite limited. It hasn't been on my radar, and while I can't speak for anyone else, it's probably unlikely to be on anyone else's either looking at the history.

We rely on preact/compat

Mentioning preact/compat might've been bad phrasing on my part; preact/compat works just fine, it's that the larger ecosystem of React libraries is rocky in WMR. It doesn't have as good of a story for CJS deps which is the majority of the React ecosystem. You may be fine, you may not be. It depends on the deps you use, as always.

[rschristian]

Looking over issues again for v4 and I'll try to make some movement for our prerendering. If anyone has large repos to share I'd love to take a look for testing purposes.

That being said, I believe most of the concerns/wishes brought up here are well off the table and will not be supported. Prerendering in all likelihood won't be fundamentally different from it is now, both due to my own time limitations and avoiding feature creep (e.g., yes, our prerendering isn't conducive to ISG, but that's also sort of the point).

Because of that, I'll close this out, as we have a few other issues regarding faster prerendering already. Appreciate the time you took to write up all those examples/code snippets.