SEO Compatible Rendering (Pre-rendering)

[Archmonger]

trafficstars

Current Situation

Currently, sites built in ReactPy are not SEO compatible. This is a fairly common issue with JavaScript frameworks such as ReactJS.

Prior discussion: https://github.com/reactive-python/reactpy/discussions/486 Prior Issue: reactive-python/reactpy-django#93

Proposed Actions

To resolve this, there needs to be an initial HTTP render, followed by a JavaScript re-render.

It may be worth considering using Preact's hydrate API, but this would likely require ReactPy to have a database backend. However, a simpler design involves

1. Use the template tag to render the initial component as raw HTML.
2. When the JavaScript client requests it, re-render the component.

[Archmonger]

This is an "experimental feature" and belongs in v2 -> v3

[Archmonger]

In order to take advantage of why users want SSR (namely SEO), IDOM might need to be modified to use HTTP instead of websockets.

[rmorshea]

So my thinking on this is that only the initial page needs the be rendered in HTML. Subsequent updates could be communicated via websockets. Then again, React has a way to do this, so if we can do server-side rendering with a traditional React solution and communicate via sockets on localhost then that could be an option too. The latter is a bit more complicated with respect to the tooling, but less work with respect to actual implementation, so I'd probably lean towards the latter assuming it's possible.

[Archmonger]

I did some investigation and here's what I came up with.

The Ugly

• The currently accepted "best methods" for SSR within React seems to rely frameworks such as NextJS, Razzle, or NodeJS.
  • These frameworks are used to render only the initial page server-side.
  • As you mentioned, that's the preferred behavior.
• If we want native SSR that relies on existing JS frameworks, we might end up relying on transpiling.
  • I can't think of a good way to propagate renders from IDOM Server Side -> Any JS Framework Server Side (transpiled?) -> React Client Side
  • Unless we can figure out a cross-platform way to make that work, I don't recommend going down this path.

Suggested Approach

• My suggested solution is to do this within Python.
• In theory, it's currently possible to immediately develop SSR for Django IDOM due to template tags.
  • We would need to render the component's initial HTML as a string within the template tag and then pass that into body of the component div.
  • mountViewToElement's behavior of wiping the div on WS connect will need to change.
    • https://github.com/idom-team/django-idom/pull/27#issuecomment-970960286
  • Enabling or disabling SSR would configurable through settings.py:IDOM_SSR: bool = False
• There are a few things to consider with this design...
  • To avoid wasteful "double rendering" (SSR render + WS render), we need a way to signal mountViewToElement that the initial render has already been server-side rendered.
    • Likely just a ssr boolean parameter within mountViewToElement.
  • Rehydrating hook values from the SSR render into the WS render will be difficult and come with big limitations/hurdles.
    • We can't really store these values client side due to having to protect against value spoofing.
    • Will likely need to serialize and store the component's hook attributes within a database.
    • This limits SSR capabilities to web frameworks with integrated database capabilities.
    • This limits hooks' initial values to those serializable by dill.pickle.
    • On web frameworks without databases, we can fallback to "double rendering" the component.
    • Alternatively to database storage, we can use redis. But I'm personally not a huge fan of IDOM forcing redis usage.
• This template tag system should be replicated to all other frameworks we support.
  • https://github.com/idom-team/idom/issues/653

---

Resources

• https://blog.logrocket.com/why-you-should-render-react-on-the-server-side-a50507163b79/
• https://asperbrothers.com/blog/server-side-rendering-in-react/#:~:text=What%20is%20Server%2DSide%20Rendering,with%20all%20the%20static%20elements.
• https://www.freecodecamp.org/news/server-side-rendering-your-react-app-in-three-simple-steps-7a82b95db82e/

[Archmonger]

@rmorshea I've discovered react actually has an API for this.

https://beta.reactjs.org/reference/react-dom/client/hydrateRoot

[rmorshea]

We're technically using Preact right now, so we'd probably what we'd want to pay attention to its specific hydration details.

It would be interesting to see if there's anything special about pre-rendering. If not, then we could potentially just use vdom_to_html. If so, we might need a build step where we're require the user to have NodeJS installed so we can use preact build.

[Archmonger]

I don't think we should use that pre-rendering API.

If we do, then

1. We are making the assumption that IDOM knows where HTML static pages should be stored.
   • This would mandate users to define a HTML output directory via Options
2. Have users perform a build step
3. Would seems like like we'd be encroaching features that our backends do.

We may want to stare at the pre-rendering source and see if anything unique is being done, which we could implement as mutations. I doubt it though, so vdom_to_html seems like the right choice.

[Archmonger]

I've been doing some thinking on this. I think we might need to take a "zero rehydration" policy and always double render (HTTP followed by WS render).

Since the HTTP stack and websocket stack aren't always on the same process/thread, we can't store any data in-memory. Additionally, not all backends have have a built-in database framework... and we really don't want to build any database features directly into ReactPy.

So, to keep things consistent it's probably best to do the following:

• Template tag performs the Initial HTTP render using html_to_vdom
  • Exist solely to pre-populate the page.
  • This initial render is completely non-interactive, and the page will remain non-interactive until the re-rendering step below.
• ASAP re-render using websockets.
  • Developers will need to be mindful of a few details
    • There may be differences between the two renders, since hook values aren't stored + rehydrated.
    • use_scope/use_connection hooks will utilize a HTTP Carrier during this initial render.

If we really want to, we can enable rehydration for batteries-included frameworks such as Django. But in my opinion, having a the functional difference between the rehydration/non-rehydration between frameworks isn't a good idea. Even more so when considering that it would introduce more maintenance burdens.