normalization-bench icon indicating copy to clipboard operation
normalization-bench copied to clipboard

Testing out Coq / native_compute with flambda

Open JasonGross opened this issue 5 years ago • 3 comments

It's worth noting that native_compute performs significantly worse than OCaml HOAS, though the two should be similar; probably native_compute does not use flambda optimization options at all.

Indeed, the Coq packages I've provided don't use flambda, but it's possible install OCaml via opam with flambda support / optimizations, and then build Coq (/ install Coq on opam?) with flambda, which should automatically translate to the native compiler. Coq's INSTALL.md suggests installing ocaml version 4.10.0+flambda.

@ejgallego and @maximedenes might be interested in this benchmark

JasonGross avatar May 19 '20 19:05 JasonGross

The native compiler will not call flambda with optimization, see this code in nativelibs

  let flambda_args = if Sys.(backend_type = Native) then ["-Oclassic"] else [] in

you would like to use something such as -O3 -funbox-closures ; this is quite expensive in terms of compilation time; native also uses some internals of OCaml that could make the above flags not work.

ejgallego avatar May 19 '20 20:05 ejgallego

Also, @AndrasKovacs it's not clear to me whether your times include compilation time or just runtime, for the compiled languages. The timing on native_compute includes both. In Coq 8.12, you can see a breakdown via Set NativeCompute Timing. In older versions of Coq, you could pass -debug to coqc, but you'd get somewhat useless times, as it would report user times rather than real times. See also https://github.com/coq/coq/pull/11963

JasonGross avatar May 19 '20 20:05 JasonGross

@JasonGross testing native_compute with expressions like (const n10 t8M) ran in about 70ms (perhaps there's some caching going on in aux files?), which suggests that compilation does not significantly change the results. This is something though which I should mention in the docs.

AndrasKovacs avatar May 19 '20 20:05 AndrasKovacs