fishers_exact_test icon indicating copy to clipboard operation
fishers_exact_test copied to clipboard

pip install fails in python 3.12

Open marco-mariotti opened this issue 1 year ago • 13 comments

Hi, very recently, in Linux only, pip install fails:

mamba create -yn test_pr_fisher python
mamba activate test_pr_fisher
pip install fisher

The error:

Collecting fisher
  Downloading fisher-0.1.14.tar.gz (22 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy (from fisher)
  Using cached numpy-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
Using cached numpy-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.0 MB)
Building wheels for collected packages: fisher
  Building wheel for fisher (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for fisher (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [118 lines of output]
      Failed to load config from /tmp/pip-install-1cwg4ln9/fisher_35c0d6f414c145cd88a7eebf3afda6d3/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      running bdist_wheel
      running build
      running build_py
      Failed to load config from /tmp/pip-install-1cwg4ln9/fisher_35c0d6f414c145cd88a7eebf3afda6d3/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      Failed to load config from /tmp/pip-install-1cwg4ln9/fisher_35c0d6f414c145cd88a7eebf3afda6d3/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      creating build
      creating build/lib.linux-x86_64-cpython-312
      creating build/lib.linux-x86_64-cpython-312/fisher
      copying fisher/tableprinter.py -> build/lib.linux-x86_64-cpython-312/fisher
      copying fisher/_version.py -> build/lib.linux-x86_64-cpython-312/fisher
      copying fisher/__init__.py -> build/lib.linux-x86_64-cpython-312/fisher
      running egg_info
      writing fisher.egg-info/PKG-INFO
      writing dependency_links to fisher.egg-info/dependency_links.txt
      writing requirements to fisher.egg-info/requires.txt
      writing top-level names to fisher.egg-info/top_level.txt
      reading manifest file 'fisher.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'README.rst'
      adding license file 'LICENSE'
      writing manifest file 'fisher.egg-info/SOURCES.txt'
      Failed to load config from /tmp/pip-install-1cwg4ln9/fisher_35c0d6f414c145cd88a7eebf3afda6d3/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      UPDATING build/lib.linux-x86_64-cpython-312/fisher/_version.py
      set build/lib.linux-x86_64-cpython-312/fisher/_version.py to '0.1.14'
      running build_ext
      /tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-1cwg4ln9/fisher_35c0d6f414c145cd88a7eebf3afda6d3/src/cfisher.pyx
        tree = Parsing.p_module(s, pxd, full_module_name)

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          return pvalue(k, n - k, K - k, N - K - n + k)


      @cython.boundscheck(False)
      def pvalue_npy(
             np.ndarray[np.uint_t] a_true,
                           ^
      ------------------------------------------------------------

      src/cfisher.pyx:129:21: Invalid type.
      Failed to load config from /tmp/pip-install-1cwg4ln9/fisher_35c0d6f414c145cd88a7eebf3afda6d3/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      Failed to load config from /tmp/pip-install-1cwg4ln9/fisher_35c0d6f414c145cd88a7eebf3afda6d3/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      Compiling src/cfisher.pyx because it changed.
      [1/1] Cythonizing src/cfisher.pyx
      Traceback (most recent call last):
        File "/home/mmariotti/miniforge3/envs/test_pr_fisher/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/mmariotti/miniforge3/envs/test_pr_fisher/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/mmariotti/miniforge3/envs/test_pr_fisher/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 415, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 67, in <module>
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/command/bdist_wheel.py", line 373, in run
          self.run_command("build")
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 1974, in run
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 93, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
          self.build_extensions()
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
          self._build_extensions_serial()
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 254, in build_extension
          _build_ext.build_extension(self, ext)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/Cython/Distutils/build_ext.py", line 130, in build_extension
          new_ext = cythonize(
                    ^^^^^^^^^^
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
          cythonize_one(*args)
        File "/tmp/pip-build-env-sx33scse/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: src/cfisher.pyx
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for fisher
Failed to build fisher
ERROR: Could not build wheels for fisher, which is required to install pyproject.toml-based projects

You can also see the error in this action fail in pyranges, a package that imports fisher: https://github.com/pyranges/pyranges_1.x/actions/runs/9593389507/job/26453746122

fisher version 0.1.14, python 3.12.4 (but same error on python 3.12.0)

The same version installed correctly some time ago, so I'm guessing it's something about its dependencies that changed (???)

btw thanks for your package!

marco-mariotti avatar Jun 20 '24 07:06 marco-mariotti

looks like this is due to numpy 2.0. We could change the type to np.uint32_t, I think. Would accept a PR

brentp avatar Jun 24 '24 17:06 brentp

here: https://github.com/brentp/fishers_exact_test/pull/49

marco-mariotti avatar Jun 26 '24 10:06 marco-mariotti

@brentp while this works when installed from github, and also surprisingly from pip, the jobs that requires fisher still fail at github and RTD. I see that the version number didn't change. So perhaps it's a problem of github etc still caching the old version?

Please bump version number and update pypi

thanks

marco-mariotti avatar Jul 07 '24 13:07 marco-mariotti

@brentp did you have a chance to look into this? Our repo workflows have been stuck on this small bug for a long time. If we can't fix this in fisher, we'll need to remove it from our dependencies.

marco-mariotti avatar Jul 18 '24 12:07 marco-mariotti

How do I do it 'manually'?

Ah, this works (which is what @marco-mariotti probably meant when he said, "this works when installed from github, and also surprisingly from pip):

pip install git+https://github.com/marco-mariotti/fishers_exact_test

dbolser avatar Mar 14 '25 11:03 dbolser

Hi, I'd accept a PR to fix this, but am not able to keep up with python packaging issues lately

brentp avatar Mar 14 '25 14:03 brentp

Thanks @brentp, as far as I can tell, the issue is 100% fixed. The problem is that pip finds the old, unfixed version. I'm honestly not sure what the process is for updating the version that pip uses... Seems like @marco-mariotti knows? He has provided a PR which fixes it for me.

dbolser avatar Mar 15 '25 11:03 dbolser

I think bumping the version number is all that is left to do. I couldn't understand how to do it in this package, since it relies on some automated system which I couldn't grasp. Marco Mariotti

On Sat, Mar 15, 2025, 12:49 Dan Bolser @.***> wrote:

Thanks @brentp https://github.com/brentp, as far as I can tell, the issue is 100% fixed. The problem is that pip finds the old, unfixed version. I'm honestly not sure what the process is for updating the version that pip uses... Seems like @marco-mariotti https://github.com/marco-mariotti knows? He has provided a PR which fixes it for me.

— Reply to this email directly, view it on GitHub https://github.com/brentp/fishers_exact_test/issues/48#issuecomment-2726471764, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACTXRIA3U3TEYYEOEN5H7IT2UQHV7AVCNFSM6AAAAABZANRN2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRWGQ3TCNZWGQ . You are receiving this because you were mentioned.Message ID: @.***> [image: dbolser]dbolser left a comment (brentp/fishers_exact_test#48) https://github.com/brentp/fishers_exact_test/issues/48#issuecomment-2726471764

Thanks @brentp https://github.com/brentp, as far as I can tell, the issue is 100% fixed. The problem is that pip finds the old, unfixed version. I'm honestly not sure what the process is for updating the version that pip uses... Seems like @marco-mariotti https://github.com/marco-mariotti knows? He has provided a PR which fixes it for me.

— Reply to this email directly, view it on GitHub https://github.com/brentp/fishers_exact_test/issues/48#issuecomment-2726471764, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACTXRIA3U3TEYYEOEN5H7IT2UQHV7AVCNFSM6AAAAABZANRN2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRWGQ3TCNZWGQ . You are receiving this because you were mentioned.Message ID: @.***>

marco-mariotti avatar Mar 15 '25 13:03 marco-mariotti

Image

https://github.com/brentp/fishers_exact_test/blob/master/.github/workflows/package.yml

I wonder why the package doesn't fail?

dbolser avatar Mar 20 '25 08:03 dbolser

Here is my update to package.yml, but not sure how to trigger it on my fork. https://github.com/brentp/fishers_exact_test/pull/52

dbolser avatar Mar 20 '25 08:03 dbolser

OK, now we need to debug the build 😅

As I said, I couldn't figure out how to trigger an action on my own fork of the repo, so I couldn't test the PR before pushing... I'm not an expert, but I'll poke around...

I think this is the latest run: https://github.com/brentp/fishers_exact_test/actions/runs/13970828167

dbolser avatar Mar 22 '25 12:03 dbolser

This is the latest build: https://github.com/brentp/fishers_exact_test/actions/runs/13974734435 I have it working except that the MacOS builds are not running

brentp avatar Mar 22 '25 13:03 brentp

GitHub want some money I guess?

On Sat, 22 Mar 2025 at 13:06, Brent Pedersen @.***> wrote:

This is the latest build: https://github.com/brentp/fishers_exact_test/actions/runs/13974734435 I have it working except that the MacOS builds are not running

— Reply to this email directly, view it on GitHub https://github.com/brentp/fishers_exact_test/issues/48#issuecomment-2745259770, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA7NE6MIWL7U33PCB4IJIT2VVN4NAVCNFSM6AAAAABZANRN2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONBVGI2TSNZXGA . You are receiving this because you commented.Message ID: @.***> [image: brentp]brentp left a comment (brentp/fishers_exact_test#48) https://github.com/brentp/fishers_exact_test/issues/48#issuecomment-2745259770

This is the latest build: https://github.com/brentp/fishers_exact_test/actions/runs/13974734435 I have it working except that the MacOS builds are not running

— Reply to this email directly, view it on GitHub https://github.com/brentp/fishers_exact_test/issues/48#issuecomment-2745259770, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA7NE6MIWL7U33PCB4IJIT2VVN4NAVCNFSM6AAAAABZANRN2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONBVGI2TSNZXGA . You are receiving this because you commented.Message ID: @.***>

dbolser avatar Mar 28 '25 12:03 dbolser