Conflict with numba

[seanrjohnson]

I wrote a package to run the Viterbi hmm-profile alignment algorithm on hmmer3 profiles. The package uses pyhmmer to parse the hmm files and get numpy arrays from the values. https://github.com/seanrjohnson/hmmer_compare/tree/numba

In pure python, the algorithm is very slow, so I experimented with speeding it up using numba, which leads to huge improvements in performance (50-100x in the few examples I tried).

However, it also gives me a strange warning from Numpy.

/home/sean/miniconda3/envs/hmmer_compare/lib/python3.10/site-packages/numpy/core/getlimits.py:500: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
  setattr(self, word, getattr(machar, word).flat[0])
/home/sean/miniconda3/envs/hmmer_compare/lib/python3.10/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
  return self._float_to_str(self.smallest_subnormal)
/home/sean/miniconda3/envs/hmmer_compare/lib/python3.10/site-packages/numpy/core/getlimits.py:500: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
  setattr(self, word, getattr(machar, word).flat[0])
/home/sean/miniconda3/envs/hmmer_compare/lib/python3.10/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
  return self._float_to_str(self.smallest_subnormal)

Using the snippet below, there are three ways I can get rid of that warning:

• Explicitly suppress the warning using warnings.filterwarnings
• Don't import pyhmmer
• Don't use the numba @jit decorator. (And therefore don't use Numba)

import warnings
#warnings.filterwarnings("ignore", category=UserWarning, module='numpy') ## Uncommenting this will suppress the warning
from numba import jit
import numba as nb
import numpy as np
import pyhmmer   ## commenting this will suppress the warning
from typing import Tuple


@jit(nb.types.Tuple((nb.float32,nb.uint64))(nb.float32, nb.float32, nb.uint64, nb.uint64),cache=True) ## commenting this will suppress the warning
def max2(sMM:float, sXY:float, layer1:int, layer2:int) -> Tuple[float, int]:
    if sMM > sXY:
        score = sMM
        bt = layer1
    else:
        score = sXY
        bt = layer2
    return score, bt

if __name__ == "__main__":
    print(max2(1,2,3,4))

I seem to get the same output as I did before switching to Numba, so I think for practical purposes, it's not actually a problem, I can just suppress the warning. It's just kind of confusing and annoying, so I thought I'd bring it to your attention in case there is an easy fix.

[zafar-hussain]

This is due to -ffast-math, (specifically -funsafe-math-optimizations) being enabled by some library

Here are the details, Someone’s Been Messing With My Subnormals!

After going through this article, Flushing subnormals to zero, one can decide whether to suppress the warnings or to find the culprit library.

One can test if passing "-fno-unsafe-math-optimizations" in compiler-args will handle this error, and what will be the performance penalty

I read somewhere that gcc-13 will have a proper solution, but can't find that article.

hope this helps,

cheers

[althonos]

Hi both, Indeed, this is an issue that has been creeping for some time, and which is visible even when just importing numpy after pyhmmer. The reason PyHMMER might be compiled with the -ffast-math flag is because it's using the flags provided by distutils, and several of the current releases of the Python interpreter used for compiling the wheels were compiled in -ffast-math mode already. I could manually patch the compile flags but didn't do it so far.

[althonos]

I think this is now fixed upstream in the Python binaries used to compile wheels for PyHMMER, and I have not seen this issue anymore with new releases.