openlibm icon indicating copy to clipboard operation
openlibm copied to clipboard

test: add musl-libc testsuite

Open inkydragon opened this issue 1 year ago • 4 comments

Do NOT merge with squash commits!

  • make coverage will run glibc and musl test, and then gen report.
    • Currently, it passes all musl tests under linux CI. Other platforms have not been tested yet. Wait for the following pr
  • make test will only run glibc test.
    • TODO: run musl test on win and mac CI
  • All modifications are marked with XXX.
  • Tests that are currently skipped are marked with TODO.
    • Some tests were moved to subfolders to skip them src\math\broken\*, src\math\not-impl\*

Since there are a lot of new files added, it is recommended to review the changes made after the addition of new tests one by one.


broken tests:

  • fenv
  • fma
  • fmaf
  • fmal
  • fpclassify
  • j1
  • j1f
  • jn
  • jnf
  • nearbyintl
  • nextafterl
  • nexttoward
  • nexttowardl
  • round
  • tgamma
  • yn
  • ynf

not impl funcs:

  • drem
  • dremf
  • exp10
  • exp10f
  • exp10l
  • pow10
  • pow10f
  • pow10l
  • scalb
  • scalbf

inkydragon avatar Feb 02 '24 21:02 inkydragon

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (4b83beb) 38.24% compared to head (4644b62) 68.96%. Report is 2 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master     #298       +/-   ##
===========================================
+ Coverage   38.24%   68.96%   +30.72%     
===========================================
  Files         233      233               
  Lines        6139     6142        +3     
  Branches     1607     1607               
===========================================
+ Hits         2348     4236     +1888     
+ Misses       3394     1596     -1798     
+ Partials      397      310       -87     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Feb 02 '24 21:02 codecov[bot]

Just thinking out aloud, if the musl libm passes all these tests - would it be best to pull out the libm from musl into a standalone library and update openlibm to that?

ViralBShah avatar Feb 13 '24 04:02 ViralBShah

pull out the libm from musl into a standalone library and update openlibm to that?

I like this idea. Moving a step further, we could take the libm related tests from libc-testsuite (for musl) and put them in a standalone project, maybe call it openlibm-test. Instead of adding all these things to openlibm.

Then, we have:

  • openlibm == musl fork
  • openlibm-test == libc-testsuite fork

@ViralBShah Another thing I am thinking about is after julia removes its dependency on openlibm (and then links to the system libm by default), would it be more useful to have a libm written by julia than to continue to improve the precision of openlibm?

inkydragon avatar Feb 24 '24 05:02 inkydragon

It's a good idea to have a separate repo for the tests. Julia almost has no dependency on openlibm, except in a few cases where LLVM on win32 needs it for a couple of things.

If the tests are in a separate repo, we can recreate a libm from Julia and then use the testsuite to test the Julia implementation.

ViralBShah avatar Feb 24 '24 05:02 ViralBShah