Error with py3.10

[Glock21]

With Python 3.10 and freshly installed pybloomfiltermmap3, I can't create a Bloom Filter. I got an error on pybloomfilter.pyx (see below). Note that with py3.9 and py3.8 it works well.

Python 3.10.0 (default, Oct  5 2021, 01:39:54) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pybloomfilter
>>> fruit = pybloomfilter.BloomFilter(100000, 0.1, '/tmp/words.bloom')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "src/pybloomfilter.pyx", line 91, in pybloomfilter.BloomFilter.__cinit__
  File "src/pybloomfilter.pyx", line 126, in pybloomfilter.BloomFilter._create
TypeError: extend() takes exactly 1 positional argument (0 given)```

[prashnts]

Huh, good catch! I'll take a look soon. Thanks!

Now that we don't have travis, I guess we'd switch to Actions so we can have a matrix based tests.

[mhendrey]

I'm getting Segmentation fault (core dumped) with Python 3.10 (works with Python 3.9), but I'm a little confused on which version I'm running. I install v0.5.5, but inside python VERSION says (0, 5, 4)

$ pip install pybloomfiltermmap3==0.5.5 --no-cache-dir Collecting pybloomfiltermmap3==0.5.5 Downloading pybloomfiltermmap3-0.5.5.tar.gz (488 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 488.7/488.7 kB 2.3 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Using legacy 'setup.py install' for pybloomfiltermmap3, since package 'wheel' is not installed. Installing collected packages: pybloomfiltermmap3 Running setup.py install for pybloomfiltermmap3 ... done Successfully installed pybloomfiltermmap3-0.5.5

$ python

Python 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pybloomfilter
>>> pybloomfilter.__VERSION__
(0, 5, 4)
>>> bf = pybloomfilter.BloomFilter(20000, 0.01)
Segmentation fault (core dumped)

[ecly]

@prashnts this would be still be awesome to have resolved, as I think every version from 3.10 and onwards isn't working at the moment

[prashnts]

Thanks for the ping, yes this is in my long todo list, but more honest answer is that I forgot to check what is going wrong. I'll take a look into it asap, but if I can get some assistance in tracking down why it segfaults I'd really appreciate it!

[prashnts]

Curiously, when I try to create a filter in development mode it works perfectly in 3.10.2. So this could be due to packaging error. Could you try to do the same? Clone the repo and use python setup.py develop and then try to init a filter.

[jhoareau]

Hi @prashnts, when running it from this repo's master branch, I am getting the following error:

gcc: error: src/pybloomfilter.c: No such file or directory
gcc: fatal error: no input files

However, when using the tar.gz from PyPI, the build works, but I still get the segmentation fault. I tried on Python 3.10.9 on macOS, and Python 3.10.10 on Linux.

EDIT - I have checked the setup.py and found out that my environment was missing Cython. The build works when I install Cython. I can also confirm that building it from master instead of from PyPI seems to be working, with both setup.py build and setup.py develop.

[jhoareau]

I've built wheels for both Ubuntu 20.04 LTS and macOS for our own usage, waiting for a fixed build from this repository, using simply setup.py bdist_wheel: https://pypi.org/project/pybloomfiltermmap3py310/#files

[prashnts]

@jhoareau yes the pybloomfilter.c is autogenerated by Cython. Thanks for confirming it works when building from source.

I see that the new package is working from pypi -- can you point me to the modifications you made (or a link to your fork)?

--

I did take a look into this issue over the weekend and my conclusion is that the .c source generated and packaged in the pypi package is not compatible with 3.10 (due to reasons unknown right now, but the cython release notes point that calling conventions was changes).

If your change works we can in the mean time release a quick fix.

[jhoareau]

I did not fork, I just built from the master branch from this repository with the aforementioned command.

[mhendrey]

@prashnts I just pulled the repo (master branch) using a conda environment with python=3.10.2 and ran

python setup.py develop

This initially gave me an error (fatal error: src/pybloomfilter.c: No such file or directory). I then realized I had not installed cython into the new conda environment. I looked at your requirements-dev.txt which lists cython=0.29.13. When I tried to install that version, the libgcc-ng conflicted between the cython=0.29.13 (wanted version >= 7.3.0) and python=3.10.2 (wanted version >= 9.4.0).

So I installed cython=0.29.32 and then was able to get python setup.py develop to build and verified that I could instantiate a bloomfilter.

So maybe it's just a dependency issue?

[mhendrey]

@prashnts And I just saw your pybloomfiltermmappy310 in pypi. I pip installed that into a clean environment with python=3.10.9 and it worked just fine.

[prashnts]

Okay so I think I fixed it by regenerating the pybloomfilter.c in pypi distribution. If you can confirm it works by installing 0.5.7 that I just pushed I'd appreciate it! If not, please reopen the issue!

@mhendrey pybloomfiltermmappy310 is not mine actually. I confirm your findings as well that this is dependent on cython. I have updated the requirements file, thanks!

Regarding the version being (0, 5, 4)... yeah, I forgot to update it! It's updated now in latest version.

[mhendrey]

@prashnts Many thanks for getting this fixed. I successfully pip install version 0.5.7