dream Templaters comparison

This issue aims to complete this templater comparison. Progress can be tracked in this repository. This issue will be updated accrodingly

For now, I have compared how those libraries can be covered using bisect_ppx. All libraries have successfully been covered, but with differing results.

	eml	mlx	TyXML	TyXML let%html	dream-html
syntax preservation	no, transforms into internal representation	no, transforms into ocaml	yes	yes	yes
readability	no, internal representation is compressed	yes. Resulting ocaml is readable	yes	yes	yes
comments				only ocaml parts are covered

Dec 04 '24 14:12 katel0k

Great, thank you! Could you add little representative screenshots of what the HTML coverage report looks like for each of the templaters?

Dec 05 '24 18:12 aantron

Hello, sorry for the late response. Here are the results:

Tyxml with percent syntax

It uses a preprocessor, but it is still correctly processed with bisect. Without even losing original code.

Tyxml

It is plain ocaml, bisect ppx works fine.

MLX

Because it uses a preprocessor, original code expands into this form. Result of this expantion is plain ocaml, so bisect ppx correctly processes it. Original code can be read if you squint a bit, but original code would be preferred

Dream html

It is plain ocaml, thus bisect ppx works perfectly here.

Dream eml

Issue here is that code gets smushed by preprocessor in sections passed to some buffer implementation. In perspective, dream eml can be upgraded to include some hints for bisect-ppx, so that it wont include code for buffers. Also, coverage reports generated by bisect-ppx can't be scrolled horizontally, and because of that readability suffers additional damage.

Feb 09 '25 16:02 katel0k

I have also compared their perfomance. Benchmark was simple and consisted of generating html with various amounts of repeating values. Time was measured using builtin time function. Results are in a table and on a plot below. Columns in table represent amount of repeating values, values are in seconds. Plot is done in logartihmic scale, values in legend are coefficients of line approximations for time growth (my assumption is that algorithms work in O(n^k) where k is this coefficient).

	100	1000	10000	100000
MLX	0.000103	0.000846	0.017366	0.181574
TyXML	0.000555	0.006125,	0.044396	0.413093
TyXML%	0.000518	0.005058	0.043199	0.374945
Dream html	0.000029	0.001715	0.003285	0.035864
Dream eml	0.001772	0.025511	1.492746	5.599526 (with 20000)

Dream eml showed very poor perfomance(>3 times slower than tyxml and >60 times slower than dream html). It also scales worse than other frameworks, almost with a quadratic rate of growth. Because of its poor perfomance, last test was done with 2e4 values instead of 1e5 - the test was taking too much time.

Overall, speed-wise dream html is the best option, which can clearly be seen from the plot.

Result are published in this repository

Feb 09 '25 18:02 katel0k

@aantron, I'm sorry, test using dream eml was flawed. I used recursion like that in order to generate the list:

let repeating_value ~children = (
  %%
  <div>
    <%s children %>
  </div>
  %%
)

let rec _stress_render volume i =
  if i < volume then (repeating_value ~children:(string_of_int i)) ^ (_stress_render volume (i+1))
  else ""

let stress_render volume _ = (
  <html>
  <head>
    <title>Stress</title>
  </head>
  <body>
    <div>
      <%s! _stress_render volume 0 %>
    </div>
  </body>
  </html>
)

It is ineffective due to needing to do lots of heavy string concatenation. I have changed this code from that to simply mapping range of numbers into repeating values and concatenating them afterwards:

let range from till =
  let rec _range ind =
    if ind < till then ind::_range (ind + 1)
    else []
  in
  _range from

let stress_render volume _ = (
  <html>
  <head>
    <title>Stress</title>
  </head>
  <body>
    <div>
    <%s! String.concat "" ((List.map (fun ind -> (repeating_value ~children:(string_of_int ind))) (range 0 volume))) %>
    </div>
  </body>
  </html>
)

I have also done proper measurements, running several times and averaging afterwards. Updated results are below:

	100	1000	10000	100000
MLX	0.000117	0.000969	0.015393	0.139218
TyXML	0.000423	0.003726	0.026868	0.241985
TyXML%	0.000508	0.004809	0.033487	0.236966
Dream html	0.000041	0.001579	0.003616	0.034738
Dream eml	0.001917	0.005415	0.040749	0.281302

Feb 20 '25 18:02 katel0k