Optimizing mounting to a serverside rendered HTML tree (DOM hydration)

[dineshselvantdm-zz]

trafficstars

Right now Mithril recalculates the VDOM while mounting over a server-side rendered HTML. Can it be optimized?

[pygy]

It is called DOM hydration, and it would need explicit support.

I was looking into it last autumn before being side tracked by the router.

[dead-claudia]

I also looked into it some in 0.2.x, and it's non-trivial to do in general. You literally have to build the initial tree differently, and it will inevitably run into complications with m.trust. You also have the secondary concern of whether the tree is even correct compared to the vnodes.

[ChrisGitIt]

Hi everybody,

i currently worked on a "hydration" feature for mithril. RIght now its properly working for me. There are just minor changes to mithril.js to make it work (mainly its passing my hydration function to "rendering" mithril methods. So for example m.route(document.body, '/', json_routes) ... becomes ... m.route(document.body, '/', json_routes, hydration_func)

I just experienced some smaller issues that i think are "unresolvable". For example, if your "create" a several following text nodes, it will ran into a hydration problem.

For example: m('div', [ $val1 ? 'OK': '', $val2 ? 'OK': '', ]) => theses renderings are problematic.

I thinks this is primarily the same issue as with using m.trust ... but as i write this ... i just realize that while having control about the server side rendering (https://github.com/StephanHoyer/mithril-isomorphic-example), it might be possible to enclose all subsequently following text nodes with a tag and this way having the possibility to fully rehydrate the dom on the client side.

Is anybody interested in my early alpha "rehydration" approach? My approach is a single function, currently 8 KB unzipped, commented, beautified. I might publish it, so my approach itself might be verified and extended.

Greats,

Chris

[dead-claudia]

@ChrisGitIt

I just experienced some smaller issues that i think are "unresolvable". For example, if your "create" a several following text nodes, it will ran into a hydration problem.

Try either joining Mithril text nodes where possible or just diffing the nodes without the backing components - that will resolve the diff.

Note that Mithril is missing a hook that would otherwise allow third-party integration with this: the ability to mount a root + component without rendering first. @pygy @tivac Would you all support such an addition, something like m.mount(root, component, {hydrated: true})?

[ChrisGitIt]

Hi isiahmeadows and thanks for your feedback!

Joining text nodes is very difficult. A component can return a text node, arrays can return text nodes ... to make my hydration function work, it needs two runs. The first run would be the thing that you described ({hydrated: true}), the second would be to actually assign the vnode.dom ... this actually was my first approach, and somehow i started all over ... i think i saw the opportunity to do it in one run and keep my code size small.

I would much prefer my solution right now despite the fact that two text nodes can occur ... i just have to be a little carefull writing my components and m() markup.

I think the {hydrated: true} will result in a much bigger mithril code base ... i mean like 1 or 2 more KB's, not much actually, but everybody who does not want hydration has to load 2 more KBs.

Passing a vnode accepting function as third argument is a perfect fit. It requires marginal change to mithril code base (just three functions need adjustment about the third parameter) and if you need hydration, you load and pass in the hydration function ...

Am i on the wrong track? Feedback appreciate!

[dead-claudia]

@ChrisGitIt

Joining text nodes is very difficult. A component can return a text node, arrays can return text nodes ... to make my hydration function work, it needs two runs. The first run would be the thing that you described ({hydrated: true}), the second would be to actually assign the vnode.dom ... this actually was my first approach, and somehow i started all over ... i think i saw the opportunity to do it in one run and keep my code size small.

Note the alternate (diffing the text nodes) would be much easier to do, at the cost of recreating text slices.

I think the {hydrated: true} will result in a much bigger mithril code base ... i mean like 1 or 2 more KB's, not much actually, but everybody who does not want hydration has to load 2 more KBs.

To clarify, that option would literally be just to skip this line, to enable third-party support (we've done similar in other areas to help mithril-node-render and a few others out).

[ChrisGitIt]

Hi isiahmeadows and thanks for your feedback!

I thought about your infos and i think it will unnecessarily add code to the mithril base.

Right now, it looks like all render/render.js -> createText -> createComponent etc. have to be rewritten to appreciate {hydrate:true}

My current soultion just needs marginal changes to mithril: /render/render.js function render(dom, vnodes) { ... if (dom.vnodes == null) dom.textContent = "" becomes

function render(dom, vnodes, hydrate) { ... hydrate(dom, vnodes, hooks, null);

/api/mount.js => replace two redrawService.render(root, *)

with redrawService.render(root, *, hydrate)

About the diffing text nodes: I'm currently experimenting with this. I think its possible to to properly hydrate text nodes. Its not actually joining the text nodes but to create extra text nodes and remove just the one that needs to be hydrated ... this might lead to a flicker ... i'm not totally convinced with this solution... there also might be a solution by using streams ... well, haven't dived much into them.

Any feedback appreciate!

Greats,

Chris

[leeoniya]

what reservations do you guys have against joining adjacent textnodes? i have not seen any issues with doing so, and it makes hydration pretty trivial [1].

[1] https://github.com/leeoniya/domvm/blob/2.x-dev/src/view/addons/attach.js

[ChrisGitIt]

Hi leeoniya!

And also thanks for your feedback! I just had a quick look at your provided trivial solution.

But i think the problem remains ... i think i have to recapitulate for myself:

If there is a call like m('div, ['A','B=',m(my_component)])) and var my_component = { view: function() { return Math.random() } } is given, the rendered root DOM Element would look like this=>

AB=0.23

When dom hydration takes place, m() call from before gives me a vnode like this { children: ['A', 'B=', my_component] tag: 'div' } and my root DOM Element "Node View(!!)" would look like this { nodeType: 1 nodeName: 'div' children: [{nodeType: 3, nodeValue: 'AB=0.23'}] } SOOOO ... i'm not sure if this leads to anywhere ...

When i hydrate through the VNODE, the vnode children will ALL be applied to the same DOM text node ... AND ... when there is a redraw, only the my_component would change ... and the change will directly applied to the vnode.dom attached dom node ... and this dom node is the text node with the content "AB=0.23" SOOOOO --- yeah, i think my thought were right --- > the updated vnode.dom.content would be 0.83 and not the expected AB=0.83.

OK, i'm done. I think this is a very simplified problem description, but the problems that arise while doing something like this are very tricky to track down and messes up the whole redraw cycle.

So my current work in progress solution is to create new text element WHILE hydrating, remove the suspicious "joined" DOM text node and append three dom text nodes childs to the DOM div node.

Any feedback appreciate! I'm also not sure if the problem might be easily resolved within the mithril core. Also not sure if the problem described is actually a problem. During developing my hydration function, it looks like this is a problem ... haven't tested it lately ...this might ruin several days of thinking and testing about resolving the problem ... but @isiahmeadows might have had the same problems ... so i'm confident, this IS a problem ;-)

Greats,

Chris

[pygy]

Alternatively, while hydrating, if you encounter several consecutive text vnodes, you can turn them into a fragment of textNodes and replace the textNode in the DOM with it.

Empty text vnodes between elements will also need special care.

Edit: Specifically:

m('p', '')
m('p', '', m('p'))
m('p', m('p'), '')
m('p', m('p'), '', m('p'))
m('p', m('p'), '', '', m('p'))

[dead-claudia]

@leeoniya I'm accounting for the case of fragments. Here's a contrived example of where joining text nodes would end up not matching Mithril's internal model, and Mithril doesn't currently have a layer between text nodes and its model of their slices.

const A = {
    oncreate({dom}) { /* vnode.dom is the text node's dom */ },
    view() { return ["foo"] },
}

const B = {
    view() { return [m(A), "bar"] },
}

[leeoniya]

you may be interested in @thysultan's feedback starting here:

https://github.com/leeoniya/domvm/issues/101#issuecomment-261228607

i don't know if the latest dio still works this way.

[thysultan]

It still does Hydrate.js#L54, though this only works if you have an internal data structure to represent text nodes, Element.js#L202.

[dead-claudia]

Mithril does have an internal data structure, but it'd require significant rewriting to avoid it.

I'm experimenting independently with a way of using slices to update text nodes and fragments while keeping a separate model for the actual DOM tree, so that the rendered internal model only sees fully normalized text nodes and an optimal tree. (It'd also make for easier hydration.)

(As for the status of this experiment, it's still local and closed-source until I actually get something functional and tested.)

[jmooradi]

Has any progress been made on this recently or anyway I can assist the active development?

[ChrisGitIt]

Hi jmooradi,

i'm currently using my implementation on a "larger site (+1000 pages) and so far it works really well. I think as soon as my project is done, i will publish my "mithril hydration" script.

[dead-claudia]

@jmooradi Not lately, but see here for a contributing FAQ.

@ChrisGitIt I'd love to see it. I'm personally curious how it would turn out.

[jmooradi]

@ChrisGitIt I'd also love to test it out before I try to take a shot at building it all myself :)

Would it make sense to have the server (mithril-node-render) return the vnode as serialized json along with the html output? The client can then unserialize and set the vnodes object on the root dom element before first render? Or is it better to parse the html into vnodes?

[dead-claudia]

@jmooradi

• Serialized JSON requires that we parse it out, and it pretty much ruins all the existing browser optimizations it could do with HTML (like building DOM as it's being downloaded).
• We have to build the vnodes separately from the DOM to begin with, but it's far cheaper to build than the DOM nodes.

Believe it or not, I've actually profiled and found that serialized templates + frag.cloneNode(true) is far faster than document.createElement, even if you only use it once. It loads faster by 10s to 100s of milliseconds for all but the smallest of trees.

[ChrisGitIt]

@jmooradi

I think there is no deeper meaning to use Json and send it to the client, because client side, there is mithril to handle all dom related. Mainly Javascript SSR (Server Side Rendering / Isomorphic) has only two, but important advantages:

• First Load Time
• Using the same libs on server and client

So my approach is just simply do the server side rendering via https://github.com/MithrilJS/mithril-node-render and then, hook in a little script that will read all DOM data and create a mithril vdom.

[jmooradi]

@ChrisGitIt I'm guessing when you build the vnode you have have to then also run a client render in order to bind any events set in view?

[ChrisGitIt]

@jmooradi: The client rendering is crucial to add events. It might be possible to add "simple" events like "onclick" on a much easier (and faster) way, but to get full mithril power, we need the full mithril vdom. So mainly i took the "Mithril first time vdom CREATION" and made it a ""Mithril first time vdom HYDRATION".

Actually, you COULD put mithril on top of every server side rendered webpage (m.mount(document.body, ...)), but this results on visible redraw (short flash of nothing). For my project, this was not acceptable. Also, the more DOM there is, the longer and more visible the redraw became. This is very noticeable on mobile devices!

I think i will upload my script and a example today. I will leave a note here when done! Any feedback or comments on my implementation is greatly appreciate!

[jmooradi]

Also on the subject of text nodes that I saw above, wouldn't splitting the text node during hydration work? You can compare the string to vdom to get the offset and split it to match what mithril expects. The implementation I'm thinking is that the vdom needs to be parsed normally during hydration in order the bind events but nothing is actually drawn to prevent flashing. https://developer.mozilla.org/en-US/docs/Web/API/Text/splitText

The lazy way of implementing this would to have the client completely trust that the server's html output matches the client's vdom.

[jmooradi]

Here my first draft at a hydrate script: https://gist.github.com/jmooradi/6f2a3d5ae279ddef76f3b42ed5c6f393

Its pretty lazy and makes a lot of assumptions that the server side rendered dom will be exactly the same as the vnodes (but I'll be adding some addition dom reading overtime). It pretty much is just the normal mithril createNodes but with all the dom creation parts stripped out, it might be a little buggy but I figured it could be a good starting place for discussion and feedback to get this feature integrated into mithril core.

Its used by calling hydrate(element, vnodes) before a render(element, vnodes)

[dead-claudia]

So I've got a thought for hydration:

• Use data-mithril-hydrate on the root DOM node with a concise bytecode to track component and fragment children correctly.
• Add a mithril/hydrate module with a single default export hydrate:
  • hydrate(root, vnodes) interprets data-mithril-hydrate on the root to generate the tree.
  • hydrate verifies the bytecode first before executing it, and throws a SyntaxError if it's invalid.
  • hydrate throws a ReferenceError if data-mithril-hydrate expects nodes that are missing and/or are of a different type, but it ignores extra nodes on elements only and it will split text nodes as it needs to.
  • If the view is different from the DOM when called, it rewrites that particular section by clearing the existing elements of the fragment and calling m.render.
• Update m.mount and m.route to accept a new, optional mountOptions parameter:
  • options.hydrate(root, vnodes) is used to generate the first vnode tree, defaulting to clearing root and then invoking m.render(root, vnodes).
• Update mithril-node-render to optionally generate a string with the above syntax to tack onto the root. So instead of const html = render(vnodes), you might be able to do const {html, hydrate} = render.hydratable(vnodes), where hydrate is a bytecode string intended for data-mithril-hydrate.

This would be really fast to interpret, because the hydration module wouldn't be diffing anything, just reviving from a single static string.

If implementation details interest you...

The bytecode would be a tree specified via a single string.

• The root would be C..., where:
  • N is an instruction, one of:
    • fC... = Add fragment
    • cIN = Add component instance
    • eL... = Add element
    • hLO = Add raw HTML with innerHTML start offset
    • n = Add null
    • tL = Add text slice
  • L is an integer string or fragment length
  • I is an integer ID
  • O is an integer offset
  • ... is a sequence of L instructions, with no whitespace or other characters separating them.
  • Integers are formatted as a modified unsigned LEB128 with 5 value bits instead of 7, and each resulting 6-bit number encoded using the following table to ensure it's all ASCII and a valid unquoted attribute string:
    • 0-9 = 0-9
    • 10-36 = A-Z
    • 37-61 = a-z
    • 62 = +, 63 = -

In addition to the grammar above, there's one other big rule:

• New child IDs must be no more than one more than the last highest child ID.

Since we also validate before we even attempt to interpret the string, this lets us figure out the number of component instances we need to revive, so we can allocate a fixed-size array of child components. This saves a lot on space and time on our part, and we can use Array.prototype.fill (with fallback) as a hint to engines that we just want to allocate a fixed array.

This format was specifically chosen to be very concise, highly compressible, and quickly interpreted.

• It's concise in that even non-trivial subtrees can be expressed in a relatively small number of bytes. Consider that the entire component subtree for the example below is only 44 bytes.
• It's highly compressible because the format is specifically designed to be regular, and the structure is intentionally very similar between types. Consider how even in the relatively contrived example below, there's already some repeating substructures in it, like e1t6 and e0.
• It's quickly interpretable because of two reasons: the grammar can be easily parsed and interpreted in streaming fashion and it aligns very well with the vnode structure. It's very amenable to a hybrid state machine- and pushdown automaton-based interpreter loop where the string's characters are iterated in the outer loop, and the verification process only needs an ID counter, a stack of lengths, and a few state variables.

It's also intentionally fully within the ASCII space and valid as an unquoted HTML attribute, for a few more size gains.

If you're curious how this'd look, here's a concrete example if you statically rendered @barneycarroll's lifecycle explorer (with added whitespace to show HTML structure):

<body data-mithril-hydrate="c0f2e1c1e7t4t1t1ne0f0e1t1e1c2e4e1t6e1t6e0e0">
	<div style="flex: 0 1 50%; overflow-y: auto;">
		<div style="border: 1px solid; padding: 0.5em; margin: 0.5em;">
			Node1 <hr>
			<button>+</button>
		</div>
	</div>
	<div style="flex: 1 1 50%; overflow-y: auto;">
		<button>Redraw</button>
		<button>Clear</button>
		<button>Break</button>
		<div></div>
	</div>
</body>

<!--
Here's the above `data-mithril-hydrate` attribute broken down:

c0 f2 - This is an anonymous component with 2 children
	e1 - The first `div` has a single child, `m(App.views.Node, {key: 1})` 
		c1 e7 - The component's instance has 7 children
			t4 - `Node`
			t1 - `1` (`key`)
			t1 - ` `
			n - Omitted `<button>x</button>`
			e0 - `<hr>`
			f0 - An empty fragment
			e1 t1 - `<button>+</button>`
	e1 - The second `div` has a single child, `m(App.views.Log)`
		c2 e4 - The component's instance has 4 children
			e1 t6 - `<button>Redraw</button>`
			e1 t6 - `<button>Clear</button>`
			e0 - `<button>`, contents set via `innerHTML`
			e0 - `<div></div>`, contents rendered in subtree
-->

Edit: @barneycarroll @pygy @StephanHoyer Thoughts?

[ChrisGitIt]

Hi @isiahmeadows,

i had now time to think about your approach for some time ... what problem does it solve? Is it for "Static" hypertext only? Where are the lifecyle methods? Why use the bytecode when each rendered hyperscript element need to get a dom attached anyway? I have to admit: I don't understand your approach. Maybe its best to have a real life example (proper component and lifecycle methods defined, then your hydrate approach explained in correlation etc.).

Wish you all had a merry Christmas!

Greets,

Chris

[dead-claudia]

The bytecode exists not for the DOM but to track the vnode state when rendered, particularly text, trusted, fragment, and component vnodes.

• The HTML parser does not generate adjacent text vnodes, only normalized ones that contain all text between two tags (or the start or end of a file). So adjacent text nodes like in m("div", "Count: ", count) would be lost.
• HTML and the DOM provide no facilities for live fragments, but we need to represent them somehow. And given the DOM might be rendered with a different version than the current vnode tree (or maybe even just a wrong attribute or state variable due to a user bug), I need something to diff the DOM against.
• Similarly, HTML provides no facilities for tracking components, and the obvious "solution" of custom elements would really just get in the way - I'd have to replace them with their children, preventing the browser from being able to calculate painting and stream it straight to screen.

It does duplicate the DOM structure a little, but I chose that over a data-mithril-hydrate on every element for its children (it's more compressible), and it acts as a convenient sanity check on the DOM itself. Also, the bytecode carries only "is this an element", not any details on tag name, attributes, or children, so it's only very mildly duplicative. On Wed, Dec 26, 2018 at 05:28 ChrisGitIt [email protected] wrote:

Hi @isiahmeadows https://github.com/isiahmeadows,

i had now time to think about your approach for some time ... what problem does it solve? Is it for "Static" hypertext only? Where are the lifecyle methods? Why use the bytecode when each rendered hyperscript element need to get a dom attached anyway? I have to admit: I don't understand your approach. Maybe its best to have a real life example (proper component and lifecycle methods defined, then your hydrate approach explained in correlation etc.).

Wish you all had a merry Christmas!

Greets,

Chris

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MithrilJS/mithril.js/issues/1838#issuecomment-449946090, or mute the thread https://github.com/notifications/unsubscribe-auth/AERrBMjAr5Xi93ToBrCBUbtsD_cRUKSkks5u809CgaJpZM4NN7bM .

[dead-claudia]

Update: I'll instead just use a list of comma-separated m.trust node counts for data-mithril-hydrate, with no spaces. It's optional and can be omitted if no such nodes exist in the tree. I'll simply throw a TypeError if things don't match, so it's still blazingly fast and relatively simple.

The rest can be inferred just as quickly from the DOM and resulting initial vnode tree as it could a bytecode string, and 99% of the temporary stuff could just be local variables. For example:

• When adopting text nodes, I could just first check node.nodeValue.startsWith(vnode.text) and throw an error based on that. Then I could just do vnode.dom = node; node = node.nodeValue.length === vnode.text.length ? null : node.splitText(vnode.text.length), since Text.prototype.splitText returns the new node.
• When iterating fragments and component instances, I don't have to add or remove anything - I just need to assert there exists a node to be consumed, and I could calculate vnode.domSize from the vnode tree and data-mithril-hydrate.

[dead-claudia]

I may implement both approaches and see which one gives me better performance. In either case, I won't diff anything: I'll throw a TypeError if anything ends up different from what I expected.

[gilbert]

Any updates on this? Very interested in getting both-side rendering in mithril.