dune icon indicating copy to clipboard operation
dune copied to clipboard

Improve incrementality of jsoo build

Open hhugo opened this issue 2 years ago • 10 comments

Instead of compiling *.cma files to JavaScript. We can compile individual *.cmos to javascript and link all *.cmo.js together. This results in a much better incremental build as jsoo no longer recompile compilation units that were not modified.

Linking *.cmo.js into a cma.js should be much faster than any compilation with jsoo.

This can only be done for local libraries as "installed" ones are not required to provide individual cmo. This is fine.

This is following https://discuss.ocaml.org/t/replace-jsoo-link-with-cat-or-esbuild/11600/13?u=hhugo

hhugo avatar Mar 23 '23 14:03 hhugo

Does this affect dead code elimination?

rgrinberg avatar Mar 27 '23 18:03 rgrinberg

It should not affect dead code elimination.

hhugo avatar Mar 27 '23 19:03 hhugo

This PR requires a new release of jsoo so that js_of_ocaml link can understand the -a flag. We'll need to add some check to know if the flag is accepted.

hhugo avatar Mar 27 '23 19:03 hhugo

I see. I think I discussed doing something like this with @nojb as well to speed up native/bytecode builds and dead code elimination was the sour point as to why this didn't work.

rgrinberg avatar Mar 27 '23 19:03 rgrinberg

I see. I think I discussed doing something like this with @nojb as well to speed up native/bytecode builds and dead code elimination was the sour point as to why this didn't work.

I don't understand.. this PR is about making jsoo compilation similar to ocamlc. Maybe you're thinking about another optimization .. ?

hhugo avatar Mar 27 '23 19:03 hhugo

Oh I see. I didn't read the PR implementation. I thought you meant producing the final .js from the .cmo.js files while completely side-stepping .cma. I see that you're just speeding up the production of your .cma.js here.

rgrinberg avatar Mar 27 '23 19:03 rgrinberg

I see. If you wanted to not produce cma files and pass cmos directly, you could implement the "dead code" logic in dune directly. It's not very difficult (jsoo does it for its link command)

hhugo avatar Mar 27 '23 20:03 hhugo

I see. If you wanted to not produce cma files and pass cmos directly, you could implement the "dead code" logic in dune directly. It's not very difficult (jsoo does it for its link command)

It's actually more complicated for dune as you need to be able to deal with arbitrary compiler version.

hhugo avatar Mar 27 '23 23:03 hhugo

It's actually more complicated for dune as you need to be able to deal with arbitrary compiler version.

How come? Wouldn't this knowledge come from ocamlep?

rgrinberg avatar Mar 28 '23 05:03 rgrinberg

It's actually more complicated for dune as you need to be able to deal with arbitrary compiler version.

How come? Wouldn't this knowledge come from ocamlep?

I don't think you want to base the logic on ocamldep output. You should let ocamlc decide if a module depend on the implementation of another one. You probably want to just use reloc info inside cmo file format and replicate the small logic for what unit should be linked

hhugo avatar Mar 28 '23 10:03 hhugo