eqaf icon indicating copy to clipboard operation
eqaf copied to clipboard

OCaml 5.00 support

Open kit-ty-kate opened this issue 2 years ago • 1 comments

Currently eqaf’s tests fail with OCaml 5.00:

# /usr/bin/ld: clock/libclock_stubs.a(clock_stubs.o): in function `clock_linux_get_time_byte':
# /home/opam/.opam/5.00/.opam-switch/build/eqaf.0.8/_build/default/clock/clock_stubs.c:25: undefined reference to `copy_int64'
# collect2: error: ld returned 1 exit status
# File "caml_startup", line 1:
# Error: Error during linking (exit code 1)
# Command [17] exited with code 2:
# $ (cd _build/default && /home/opam/.opam/5.00/bin/ocamlopt.opt -w -40 -g -o test/test_branch.exe clock/clock.cmxa -I clock /home/opam/.opam/5.00/lib/ocaml/unix.cmxa -I /home/opam/.opam/5.00/lib/ocaml lib/eqaf.cmxa test/.test_branch.eobjs/native/dune__exe__Test_branch.cmx)

kit-ty-kate avatar Feb 08 '22 11:02 kit-ty-kate

As I said, even if the project compile with #37, it seems clear that we can not assert expected behaviors on OCaml 5.00.0 - and it becomes unsafe to use it until I find the problem. This is my output on my server:

Random: [|57420;41069;54817;64253;63934;64230;4054;6207;34546;20369;14111;47443;31090;25122;16285;17015;|].
> Start to test Eqaf.equal (B¹).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
> Start to test String.equal (B²).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
Eqaf.equal: 2912.436296 ns/run.
String.equal: 348.712805 ns/run.
B¹ = 24.829750, B² = -343.558125.
1 trial(s) for Eqaf.equal.
> Start to test Eqaf.compare (B¹).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
> Start to test String.compare (B²).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
Eqaf.compare: 14306.884698 ns/run.
String.compare: 71.208393 ns/run.
B¹ = -19.870853, B² = -62.595406.
1 trial(s) for Eqaf.compare.
> Start to test Eqaf.exists_uint8 (B¹).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
> Start to test String.contains (B²).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
Eqaf.exists_uint8: 18056.756548 ns/run.
String.contains: 197.184362 ns/run.
Fail with B¹ = 117.183307, B² = 4488.679906.
> Start to compute Z.
> Start to test Eqaf.exists_uint8 (B¹).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Calculating Z.
> Start to test String.contains (B²).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Calculating Z.
Z¹ = 29.020354, Z² = -3141.109392.
1 trial(s) for Eqaf.exists.
> Start to test Eqaf.find_uint8 (B¹).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
> Start to test String.index (B²).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
Eqaf.find_uint8: 17523.285751 ns/run.
String.index: 193.517407 ns/run.
B¹ = 6.197215, B² = 4455.193795.
1 trial(s) for Eqaf.find_uint8.
> Start to test Eqaf.divmod (B¹).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
> Start to test Int32.unsigned_div,Int32.unsigned_rem (B²).
> Start benchmarks on [fn⁰].
[########################################] 100%
> Start benchmarks on [fn¹].
[########################################] 100%
> Merge results.
> Start linear regression.
Eqaf.divmod: 95.941046 ns/run.
Int32.unsigned_div,Int32.unsigned_rem: 40.803290 ns/run.
B¹ = -0.083765, B² = 0.349805.
1 trial(s) for Eqaf.divmod.

To explain these results and why it's bad: the B^1 value should be near to 0 (and say that we don't have a diff time for different inputs - which is expected). It seems that we have a little diff at some points. I'm not sure if the diff come from a noise or if it's inherent to what OCaml 5.00.0 produces (in assembly). So we must check by hands the assembly with OCaml 5.00.0 and be sure that:

  1. we don't have an aggressive optimization
  2. the produced assembly is constant time regardless contents of arguments

dinosaure avatar Feb 17 '22 10:02 dinosaure