fitsio Impossible to pip install

Running pip install on the latest version crashes. python -m pip install fitsio --upgrade, but not on version 1.0.5.

Collecting fitsio
  Using cached https://files.pythonhosted.org/packages/9c/7d/99906853351108cd5abea387240b5b58109a91e349f0ae22e33c63969393/fitsio-1.1.1.tar.gz
Requirement already satisfied, skipping upgrade: numpy in <HOME>.local/lib/python3.7/site-packages (from fitsio) (1.18.2)
Building wheels for collected packages: fitsio
  Running setup.py bdist_wheel for fitsio ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-f3s2mfyj/fitsio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-f4wmuk84 --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/fitsio
  copying fitsio/header.py -> build/lib.linux-x86_64-3.7/fitsio
  copying fitsio/fitslib.py -> build/lib.linux-x86_64-3.7/fitsio
  copying fitsio/__init__.py -> build/lib.linux-x86_64-3.7/fitsio
  copying fitsio/test.py -> build/lib.linux-x86_64-3.7/fitsio
  copying fitsio/util.py -> build/lib.linux-x86_64-3.7/fitsio
  creating build/lib.linux-x86_64-3.7/fitsio/hdu
  copying fitsio/hdu/base.py -> build/lib.linux-x86_64-3.7/fitsio/hdu
  copying fitsio/hdu/image.py -> build/lib.linux-x86_64-3.7/fitsio/hdu
  copying fitsio/hdu/__init__.py -> build/lib.linux-x86_64-3.7/fitsio/hdu
  copying fitsio/hdu/table.py -> build/lib.linux-x86_64-3.7/fitsio/hdu
  running build_ext
  checking file build/temp.linux-x86_64-3.7/cfitsio3470/putcols.c
  Hunk #1 succeeded at 157 (offset -1 lines).
  patching file build/temp.linux-x86_64-3.7/cfitsio3470/putcols.c
  Hunk #1 succeeded at 157 (offset -1 lines).
  checking file build/temp.linux-x86_64-3.7/cfitsio3470/configure
  Hunk #1 succeeded at 711 with fuzz 2 (offset 3 lines).
  Hunk #2 succeeded at 1345 (offset 5 lines).
  Hunk #3 succeeded at 2337 (offset 84 lines).
  Hunk #4 succeeded at 5200 (offset 117 lines).
  patching file build/temp.linux-x86_64-3.7/cfitsio3470/configure
  Hunk #1 succeeded at 711 with fuzz 2 (offset 3 lines).
  Hunk #2 succeeded at 1345 (offset 5 lines).
  Hunk #3 succeeded at 2337 (offset 84 lines).
  Hunk #4 succeeded at 5200 (offset 117 lines).
  checking file build/temp.linux-x86_64-3.7/cfitsio3470/drvrnet.c
  patching file build/temp.linux-x86_64-3.7/cfitsio3470/drvrnet.c
  checking file build/temp.linux-x86_64-3.7/cfitsio3470/fitsio.h
  Hunk #1 succeeded at 806 (offset 9 lines).
  patching file build/temp.linux-x86_64-3.7/cfitsio3470/fitsio.h
  Hunk #1 succeeded at 806 (offset 9 lines).
  checking file build/temp.linux-x86_64-3.7/cfitsio3470/fitscore.c
  Hunk #1 succeeded at 186 (offset 4 lines).
  patching file build/temp.linux-x86_64-3.7/cfitsio3470/fitscore.c
  Hunk #1 succeeded at 186 (offset 4 lines).
  checking file build/temp.linux-x86_64-3.7/cfitsio3470/configure.in
  Hunk #1 succeeded at 52 (offset 10 lines).
  Hunk #2 succeeded at 610 (offset 21 lines).
  patching file build/temp.linux-x86_64-3.7/cfitsio3470/configure.in
  Hunk #1 succeeded at 52 (offset 10 lines).
  Hunk #2 succeeded at 610 (offset 21 lines).
  checking build system type... x86_64-pc-linux-gnu
  checking host system type... x86_64-pc-linux-gnu
  checking target system type... x86_64-pc-linux-gnu
  checking for gcc... x86_64-linux-gnu-gcc
  checking whether the C compiler works... yes
  checking for C compiler default output file name... a.out
  checking for suffix of executables...
  checking whether we are cross compiling... no
  checking for suffix of object files... o
  checking whether we are using the GNU C compiler... yes
  checking whether x86_64-linux-gnu-gcc accepts -g... yes
  checking for x86_64-linux-gnu-gcc option to accept ISO C89... none needed
  checking how to run the C preprocessor... x86_64-linux-gnu-gcc -E
  checking for grep that handles long lines and -e... /bin/grep
  checking for egrep... /bin/grep -E
  checking for ANSI C header files... yes
  checking for sys/types.h... yes
  checking for sys/stat.h... yes
  checking for stdlib.h... yes
  checking for string.h... yes
  checking for memory.h... yes
  checking for strings.h... yes
  checking for inttypes.h... yes
  checking for stdint.h... yes
  checking for unistd.h... yes
  checking for main in -lbz2... no
  configure: error: Unable to locate bz2 library needed when enabling bzip2 support; try specifying the path
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-f3s2mfyj/fitsio/setup.py", line 289, in <module>
      cmdclass={"build_ext": build_ext_subclass}
    File "<HOME>.local/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 188, in run
      self.run_command('build')
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-f3s2mfyj/fitsio/setup.py", line 72, in run
      build_ext.run(self)
    File "<HOME>.local/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run
      _build_ext.run(self)
    File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/tmp/pip-install-f3s2mfyj/fitsio/setup.py", line 100, in build_extensions
      RANLIB=self.compiler.ranlib,
    File "/tmp/pip-install-f3s2mfyj/fitsio/setup.py", line 227, in configure_cfitsio
      "could not configure cfitsio %s" % self.cfitsio_version)
  ValueError: could not configure cfitsio 3470
  
  ----------------------------------------
  Failed building wheel for fitsio
  Running setup.py clean for fitsio
Failed to build fitsio
Installing collected packages: fitsio
  Found existing installation: fitsio 1.0.5
    Uninstalling fitsio-1.0.5:
      Successfully uninstalled fitsio-1.0.5
  Running setup.py install for fitsio ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-f3s2mfyj/fitsio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-2i013flm/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/fitsio
    copying fitsio/header.py -> build/lib.linux-x86_64-3.7/fitsio
    copying fitsio/fitslib.py -> build/lib.linux-x86_64-3.7/fitsio
    copying fitsio/__init__.py -> build/lib.linux-x86_64-3.7/fitsio
    copying fitsio/test.py -> build/lib.linux-x86_64-3.7/fitsio
    copying fitsio/util.py -> build/lib.linux-x86_64-3.7/fitsio
    creating build/lib.linux-x86_64-3.7/fitsio/hdu
    copying fitsio/hdu/base.py -> build/lib.linux-x86_64-3.7/fitsio/hdu
    copying fitsio/hdu/image.py -> build/lib.linux-x86_64-3.7/fitsio/hdu
    copying fitsio/hdu/__init__.py -> build/lib.linux-x86_64-3.7/fitsio/hdu
    copying fitsio/hdu/table.py -> build/lib.linux-x86_64-3.7/fitsio/hdu
    running build_ext
    checking file build/temp.linux-x86_64-3.7/cfitsio3470/putcols.c
    Hunk #1 succeeded at 157 (offset -1 lines).
    patching file build/temp.linux-x86_64-3.7/cfitsio3470/putcols.c
    Hunk #1 succeeded at 157 (offset -1 lines).
    checking file build/temp.linux-x86_64-3.7/cfitsio3470/configure
    Hunk #1 succeeded at 711 with fuzz 2 (offset 3 lines).
    Hunk #2 succeeded at 1345 (offset 5 lines).
    Hunk #3 succeeded at 2337 (offset 84 lines).
    Hunk #4 succeeded at 5200 (offset 117 lines).
    patching file build/temp.linux-x86_64-3.7/cfitsio3470/configure
    Hunk #1 succeeded at 711 with fuzz 2 (offset 3 lines).
    Hunk #2 succeeded at 1345 (offset 5 lines).
    Hunk #3 succeeded at 2337 (offset 84 lines).
    Hunk #4 succeeded at 5200 (offset 117 lines).
    checking file build/temp.linux-x86_64-3.7/cfitsio3470/drvrnet.c
    patching file build/temp.linux-x86_64-3.7/cfitsio3470/drvrnet.c
    checking file build/temp.linux-x86_64-3.7/cfitsio3470/fitsio.h
    Hunk #1 succeeded at 806 (offset 9 lines).
    patching file build/temp.linux-x86_64-3.7/cfitsio3470/fitsio.h
    Hunk #1 succeeded at 806 (offset 9 lines).
    checking file build/temp.linux-x86_64-3.7/cfitsio3470/fitscore.c
    Hunk #1 succeeded at 186 (offset 4 lines).
    patching file build/temp.linux-x86_64-3.7/cfitsio3470/fitscore.c
    Hunk #1 succeeded at 186 (offset 4 lines).
    checking file build/temp.linux-x86_64-3.7/cfitsio3470/configure.in
    Hunk #1 succeeded at 52 (offset 10 lines).
    Hunk #2 succeeded at 610 (offset 21 lines).
    patching file build/temp.linux-x86_64-3.7/cfitsio3470/configure.in
    Hunk #1 succeeded at 52 (offset 10 lines).
    Hunk #2 succeeded at 610 (offset 21 lines).
    checking build system type... x86_64-pc-linux-gnu
    checking host system type... x86_64-pc-linux-gnu
    checking target system type... x86_64-pc-linux-gnu
    checking for gcc... x86_64-linux-gnu-gcc
    checking whether the C compiler works... yes
    checking for C compiler default output file name... a.out
    checking for suffix of executables...
    checking whether we are cross compiling... no
    checking for suffix of object files... o
    checking whether we are using the GNU C compiler... yes
    checking whether x86_64-linux-gnu-gcc accepts -g... yes
    checking for x86_64-linux-gnu-gcc option to accept ISO C89... none needed
    checking how to run the C preprocessor... x86_64-linux-gnu-gcc -E
    checking for grep that handles long lines and -e... /bin/grep
    checking for egrep... /bin/grep -E
    checking for ANSI C header files... yes
    checking for sys/types.h... yes
    checking for sys/stat.h... yes
    checking for stdlib.h... yes
    checking for string.h... yes
    checking for memory.h... yes
    checking for strings.h... yes
    checking for inttypes.h... yes
    checking for stdint.h... yes
    checking for unistd.h... yes
    checking for main in -lbz2... no
    configure: error: Unable to locate bz2 library needed when enabling bzip2 support; try specifying the path
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-f3s2mfyj/fitsio/setup.py", line 289, in <module>
        cmdclass={"build_ext": build_ext_subclass}
      File "<HOME>.local/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "<HOME>.local/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.7/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-f3s2mfyj/fitsio/setup.py", line 72, in run
        build_ext.run(self)
      File "<HOME>.local/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run
        _build_ext.run(self)
      File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/tmp/pip-install-f3s2mfyj/fitsio/setup.py", line 100, in build_extensions
        RANLIB=self.compiler.ranlib,
      File "/tmp/pip-install-f3s2mfyj/fitsio/setup.py", line 227, in configure_cfitsio
        "could not configure cfitsio %s" % self.cfitsio_version)
    ValueError: could not configure cfitsio 3470
    
    ----------------------------------------
  Rolling back uninstall of fitsio
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-f3s2mfyj/fitsio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-2i013flm/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-install-f3s2mfyj/fitsio/

Apr 12 '20 20:04 londumas

I didn't add the bz2 support, can't recall who did. Anyone have ideas?

Apr 14 '20 17:04 esheldon

Just to restate, this is an issue with bz2 support. A quick fix would be to make sure libbz2 is installed with headers. but we should be detecting that

Apr 27 '20 16:04 esheldon

I looked into this a bit more. I didn't add the bzip support.

It looks like currently installation of fitsio requires libbz2 to be installed (with headers), and this is true for pip installs as well. So if you want to use pip you will need to install it.

Note however that if you use conda it will pull in bzip for you

Apr 28 '20 15:04 esheldon

@esheldon, thanks for the answers. I would be nice to be able to use pip install. Wouls you agree?

Jul 18 '20 09:07 londumas

Pip install does work, but you need to install libbz2 first

Jul 19 '20 01:07 esheldon

I think we understand the problem here, that you need libbz2, it is a dependency. That dependency cannot be installed from pip. Is it not possible to install libbz2 on your system?

Nov 12 '20 13:11 esheldon

@esheldon, the issue first came from travis tests, that uses pip install.

Nov 12 '20 14:11 londumas

@esheldon, Indeed. I just tried and running sudo apt-get install libbz2-dev then python3 -m pip install fitsio --upgrade works for the latest fitsio. Is there a way to fix that so pip works?

Nov 18 '20 22:11 londumas

We'd have to vendor those libs to make pip work out of the box.

Nov 18 '20 22:11 beckermr

Do you mean you would have to buy them?

Nov 18 '20 22:11 londumas

No I mean we'd have to build them from source on the fly.

Nov 18 '20 23:11 beckermr

This is a very bad practice by the way. The proper fix is for you to install them as you did.

Nov 18 '20 23:11 beckermr

For anyone else stumbling across this and using Anaconda, this works:

conda install -c conda-forge fitsio

Ref: https://anaconda.org/conda-forge/fitsio

Aug 11 '21 18:08 demitri