Augustus icon indicating copy to clipboard operation
Augustus copied to clipboard

Test reference results not portable across architectures

Open satta opened this issue 3 years ago • 3 comments

Debian runs the testsuite (make test) during the build process for its Augustus packages. While building packages for non-amd64 architectures (see list at https://buildd.debian.org/status/package.php?p=augustus), we noticed that builds were failing because the tests are run by default with --compare and results are apparently different on different architectures. So diffs fail, failing the tests and hence the builds.

I have worked around this for now by dropping --compare during the build, which fixes the immediate issue. However, I figured you might want to know about this potential issue anyway, in case you were expecting the results to be identical. Here are some example build logs, showing the diffs at the end:

  • arm64 https://buildd.debian.org/status/fetch.php?pkg=augustus&arch=arm64&ver=3.4.0%2Bdfsg2-1&stamp=1609204576&raw=0
  • armel (little endian) https://buildd.debian.org/status/fetch.php?pkg=augustus&arch=armel&ver=3.4.0%2Bdfsg2-1&stamp=1609207371&raw=0.4.0%2Bdfsg2-1&stamp=1609207492&raw=0
  • armhf (hard float) https://buildd.debian.org/status/fetch.php?pkg=augustus&arch=armhf&ver=3.4.0%2Bdfsg2-1&stamp=1609207492&raw=0
  • i386 https://buildd.debian.org/status/fetch.php?pkg=augustus&arch=i386&ver=3.4.0%2Bdfsg2-1&stamp=1609205337&raw=0
  • mips64el https://buildd.debian.org/status/fetch.php?pkg=augustus&arch=mips64el&ver=3.4.0%2Bdfsg2-1&stamp=1609210270&raw=0
  • mipsel https://buildd.debian.org/status/fetch.php?pkg=augustus&arch=mipsel&ver=3.4.0%2Bdfsg2-1&stamp=1609224926&raw=0
  • ppc64el https://buildd.debian.org/status/fetch.php?pkg=augustus&arch=ppc64el&ver=3.4.0%2Bdfsg2-1&stamp=1609204270&raw=0
  • s390x https://buildd.debian.org/status/fetch.php?pkg=augustus&arch=s390x&ver=3.4.0%2Bdfsg2-1&stamp=1609218440&raw=0

satta avatar Jan 03 '21 20:01 satta

It seems to be for different reasons. @dhonsel , can you try to eliminate some? The case below appears to reoccur several times. Maybe this this has to do with white space differences, that could be ignored.

*** 4209 ****
! UTR exon level |          0 |              0 |       -nan |       -nan |
--- 4209 ----
! UTR exon level |          0 |              0 |        nan |        nan |
***************
*** 4211 ****
! UTR base level |          0 |              0 |       -nan |       -nan |
--- 4211 ----
! UTR base level |          0 |              0 |        nan |        nan |

MarioStanke avatar Jan 05 '21 18:01 MarioStanke

The case below appears to reoccur several times. Maybe this this has to do with white space differences, that could be ignored.

My bet would be on different handling of float corner cases, as we have negative NaN in one case and positive NaN in the other.

satta avatar Jan 05 '21 19:01 satta

Of course...

MarioStanke avatar Jan 05 '21 21:01 MarioStanke