File not found error when pip installing python-lzo on mac Sierra

[emson]

I'm trying to install python-lzo on mac OS Sierra. I have used brew to install the c lzo library:

brew update
brew install lzo

I then try and install python-lzo using a virtual environment and pip:

$ virtualenv ./env
$ . ./env/bin/activate
$ pip install python-lzo

Which gives me the following output:

Collecting python-lzo
  Using cached python-lzo-1.11.tar.gz
Building wheels for collected packages: python-lzo
  Running setup.py bdist_wheel for python-lzo: started
  Running setup.py bdist_wheel for python-lzo: finished with status 'error'
  Complete output from command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-build-wDRYjx/python-lzo/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/tmp_ueOY4pip-wheel- --python-tag cp27:
  /usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
    warnings.warn(msg)
  running bdist_wheel
  running build
  running build_ext
  building 'lzo' extension
  creating build
  creating build/temp.macosx-10.11-x86_64-2.7
  clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
  lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
  #include <lzo1x.h>
           ^
  1 error generated.
  error: command 'clang' failed with exit status 1
  
  ----------------------------------------
  Running setup.py clean for python-lzo
Failed to build python-lzo
Installing collected packages: python-lzo
  Running setup.py install for python-lzo: started
    Running setup.py install for python-lzo: finished with status 'error'
    Complete output from command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-build-wDRYjx/python-lzo/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-8RUWxo-record/install-record.txt --single-version-externally-managed --compile:
    /usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
      warnings.warn(msg)
    running install
    running build
    running build_ext
    building 'lzo' extension
    creating build
    creating build/temp.macosx-10.11-x86_64-2.7
    clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
    lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
    #include <lzo1x.h>
             ^
    1 error generated.
    error: command 'clang' failed with exit status 1
    
    ----------------------------------------

The lzo1x.h file does exist here: /usr/local/opt/lzo/include/lzo/ or with the brew command:

$ ls $(brew --prefix lzo)/include/lzo/

I have then tried to set the CFLAGS envronment variables, but still have the same error:

$ export EXTRA_CFLAGS=-I/usr/local/opt/lzo/include && export EXTRA_CXXFLAGS=-I/usr/local/opt/lzo/include && export EXTRA_LDFLAGS=-L/usr/local/opt/lzo/lib && pip install

And in desperation:

$ export EXTRA_CFLAGS=-I/usr/local/opt/lzo/include/lzo && export EXTRA_CXXFLAGS=-I/usr/local/opt/lzo/include/lzo && export EXTRA_LDFLAGS=-L/usr/local/opt/lzo/lib && pip install python-lzo

Still the same error message, any ideas? Many thanks

[emson]

I asked the same question on Stackoverflow and managed to get an answer that worked: http://stackoverflow.com/questions/40424097/file-not-found-error-when-pip-installing-python-lzo

In summary the solution is:

export C_INCLUDE_PATH=$(brew --cellar lzo)/2.09/include/lzo:$(brew --cellar lzo)/2.09/include/
export LIBRARY_PATH=/usr/local/lib
pip install python-lzo

Also there was an interesting suggestion about the "include paths"

Having a look at the setup.py file for python-lzo it seems there's a bit of weirdness going on, in particular I'm concerned that on Windows both include paths are being added, which suggests the lzo source distribution isn't very well arranged. But there doesn't seem to be a designed way of achieving this on macOS with a brew install, so I think the include part of it at least is best solved with the extra environment variable.

Hope this helps, many thanks.

[aegisstrike]

It works! That's really do me a favour. Thank you so much!

[Alex--C]

Thanks for that solution! Note that lzo has been updated to 2.10, therefore the export has to be changed to

export C_INCLUDE_PATH=$(brew --cellar lzo)/2.10/include/lzo:$(brew --cellar lzo)/2.10/include/
export LIBRARY_PATH=/usr/local/lib
pip install python-lzo

[galaxy001]

Just C_INCLUDE_PATH=/usr/local/Cellar/lzo/2.10/include/lzo pip2 install python-lzo is enough.

[AnFour]

thx so much!

[chahatupreti]

@Alex--C , unfortunately this solution did not work for me, and I still get the error -

lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found #include <lzo1x.h> ^~~~~~~~~ 1 error generated. error: command 'gcc' failed with exit status 1

I am running high Sierra on Mac

[jd-boyd]

@chahatupreti Can you confirm where lzo1x.h is to be found on your Mac?

[chahatupreti]

thanks @jd-boyd for the response. I searched for the file but i don't think it exists on my Mac. I assumed since it is a pip install, I wouldn't need the file on my system, and that it will be downloaded from the internet, but I could be wrong about that.

[jd-boyd]

@chahatupreti for python packages with external C dependencies, normally they don't automatically install them. For instance, pillow does not download and install libpng or libtiff for you, you have to install them separately. On Linux this is usually super easy, but not so much elsewhere. I've been looking into alternative ways to handle this.

[Alex--C]

@chahatupreti then you probably did not install lzo. If you already have brew just run

brew update brew install lzo

If you don't have brew, check https://brew.sh first. Sorry for the poor formatting, I'm currently on mobile.

[rijieli]

For macOS 11.6, and M1 chip, the following step work for me.

# Install lzo homebrew 
arch -arm64 brew install lzo

# Configure include path and lib path
export C_INCLUDE_PATH=/opt/homebrew/Cellar/lzo/2.10/include/lzo:/opt/homebrew/Cellar/lzo/2.10/include/
export LIBRARY_PATH=/opt/homebrew/lib

# Install
pip3 install python-lzo

Installing collected packages: python-lzo
  Running setup.py install for python-lzo ... done
Successfully installed python-lzo-1.12