Document installation for OSX (cannot import name constants)

[nitinbodke]

I am trying to install Graphite. I am stuck upon cairo installation. I came across "http://stackoverflow.com/questions/11491268/install-pycairo-in-virtualenv". I have installed cairocffi in my virtual environment.

My environment is: "OS X: 10.8.5 Python: 2.7.2 Virtual Env: pip list cairocffi (0.5.1) ceres (0.10.0) cffi (0.8.1) Django (1.5.5) django-tagging (0.3.1) pip (1.5) pycparser (2.10) setuptools (2.0.2) Twisted (11.1.0) txAMQP (0.6.2) whisper (0.9.12) wsgiref (0.1.2) zope.interface (4.0.5)"

Cairo: I have installed cairo using macports. "cairo @1.12.16_2+x11 (active)"

I started graphite with following sequence of commands:

1. source /opt/graphite/bin/activate
2. sudo /opt/graphite/bin/carbon-cache.py start
3. sudo ./bin/run-graphite-devel-server.py /opt/graphite

http://0.0.0.0:8080/ is displaying Graphite home page. But http://0.0.0.0:8080/render produces following traceback:

Traceback (most recent call last):
  File "/opt/graphite/lib/python2.7/site-packages/django/core/handlers/base.py", line 103, in get_response
    resolver_match = resolver.resolve(request.path_info)
  File "/opt/graphite/lib/python2.7/site-packages/django/core/urlresolvers.py", line 321, in resolve
    sub_match = pattern.resolve(new_path)
  File "/opt/graphite/lib/python2.7/site-packages/django/core/urlresolvers.py", line 321, in resolve
    sub_match = pattern.resolve(new_path)
  File "/opt/graphite/lib/python2.7/site-packages/django/core/urlresolvers.py", line 223, in resolve
    return ResolverMatch(self.callback, args, kwargs, self.name)
  File "/opt/graphite/lib/python2.7/site-packages/django/core/urlresolvers.py", line 230, in callback
    self._callback = get_callable(self._callback_str)
  File "/opt/graphite/lib/python2.7/site-packages/django/utils/functional.py", line 31, in wrapper
    result = func(*args)
  File "/opt/graphite/lib/python2.7/site-packages/django/core/urlresolvers.py", line 97, in get_callable
    mod = import_module(mod_name)
  File "/opt/graphite/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/opt/graphite/webapp/graphite/render/views.py", line 36, in <module>
    from graphite.render.evaluator import evaluateTarget
  File "/opt/graphite/webapp/graphite/render/evaluator.py", line 47, in <module>
    from graphite.render.functions import SeriesFunctions
  File "/opt/graphite/webapp/graphite/render/functions.py", line 33, in <module>
    from graphite.render.glyph import format_units
  File "/opt/graphite/webapp/graphite/render/glyph.py", line 15, in <module>
    import cairocffi as cairo
  File "/opt/graphite/lib/python2.7/site-packages/cairocffi/__init__.py", line 16, in <module>
    from . import constants
ImportError: cannot import name constants

[SimonSapin]

What files does /opt/graphite/lib/python2.7/site-packages/cairocffi contain? Does it not contain constants.py? Try this (assuming you used sudo to create the virtualenv. Otherwise remove sudo from these lines.)

source /opt/graphite/bin/activate
sudo pip uninstall cairocffi
sudo pip install git+https://github.com/SimonSapin/cairocffi.git

What’s the entire output of the above? Does it fix the issue?

[nitinbodke]

Yes, it do contain constants.py. I followed above steps, but that didn't help. Below is the version that gets installed with above command. "cairocffi (0.5.1)" It's the same as earlier one.

[SimonSapin]

Ok. Add these lines to /opt/graphite/lib/python2.7/site-packages/cairocffi/__init__.py, before from . import constants

print(sys.path)
print(__path__)

You should now get two additional lines of output, before the traceback. What do they contain?

[nitinbodke]

['/opt/graphite/webapp', '/opt/graphite/webapp', '/opt/graphite/bin', '/opt/graphite/lib/python27.zip', '/opt/graphite/lib/python2.7', '/opt/graphite/lib/python2.7/plat-darwin', '/opt/graphite/lib/python2.7/plat-mac', '/opt/graphite/lib/python2.7/plat-mac/lib-scriptpackages', '/opt/graphite/Extras/lib/python', '/opt/graphite/lib/python2.7/lib-tk', '/opt/graphite/lib/python2.7/lib-old', '/opt/graphite/lib/python2.7/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/opt/graphite/lib/python2.7/site-packages', '/opt/graphite/webapp/graphite/thirdparty']
['/opt/graphite/lib/python2.7/site-packages/cairocffi']"

[SimonSapin]

I’m sorry, I have no idea what’s going on there… Everything looks like it should work.

Does this work, when Graphite is not involved?

/opt/graphite/bin/python -c 'import cairocffi'

[nitinbodke]

Something wrong with my environment. Yesterdays fix is not working. Let me go through it.

(graphite)$ python
Python 2.7.2 (default, Oct 11 2012, 20:14:37) 
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cffi
>>> import cairocffi
['', '/opt/graphite/lib/python27.zip', '/opt/graphite/lib/python2.7', '/opt/graphite/lib/python2.7/plat-darwin', '/opt/graphite/lib/python2.7/plat-mac', '/opt/graphite/lib/python2.7/plat-mac/lib-scriptpackages', '/opt/graphite/Extras/lib/python', '/opt/graphite/lib/python2.7/lib-tk', '/opt/graphite/lib/python2.7/lib-old', '/opt/graphite/lib/python2.7/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/opt/graphite/lib/python2.7/site-packages']
['/opt/graphite/lib/python2.7/site-packages/cairocffi']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/graphite/lib/python2.7/site-packages/cairocffi/__init__.py", line 41, in <module>
    cairo = dlopen(ffi, 'libcairo.so.2', 'libcairo-2.dll', 'cairo', 'libcairo-2', 'libcairo.2.dylib')
  File "/opt/graphite/lib/python2.7/site-packages/cairocffi/__init__.py", line 36, in dlopen
    return ffi.dlopen(names[0])  # pragma: no cover
  File "/opt/graphite/lib/python2.7/site-packages/cffi/api.py", line 117, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/opt/graphite/lib/python2.7/site-packages/cffi/api.py", line 399, in _make_ffi_library
    backendlib = backend.load_library(name, flags)
OSError: cannot load library libcairo.so.2: dlopen(libcairo.so.2, 2): image not found
>>> import cffi; cffi.FFI().dlopen('libcairo.2.dylib')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/graphite/lib/python2.7/site-packages/cffi/api.py", line 117, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/opt/graphite/lib/python2.7/site-packages/cffi/api.py", line 399, in _make_ffi_library
    backendlib = backend.load_library(name, flags)
OSError: cannot load library libcairo.2.dylib: dlopen(libcairo.2.dylib, 2): image not found
>>> 

[hanneshapke]

@nitinbodke Did you solve the environment problem at the end? I am facing the same problem now and have no idea where to continue fixing the problem. Any advice is highly appreciated.

Thank you.

[SimonSapin]

@hanneshapke Please describe exact step to setup your environment and reproduce the bug, and copy any traceback or other output here. Also do as in https://github.com/SimonSapin/cairocffi/issues/29#issuecomment-33896736 and copy the output here.

[hanneshapke]

Hi @SimonSapin thank you for offering your help

I am trying to install Weasyprint via pip on OSX10.7 and all requirements via brew.

HOMEBREW_VERSION: 0.9.5
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CELLAR: /usr/local/Cellar
OS X: 10.7.5-x86_64
Xcode: 4.1
Python: /usr/local/bin/python => /usr/local/Cellar/python/2.7.2/bin/python

I have installed the required packages via

sudo brew install cairo pango gdk-pixbuf libxml2 libxslt libffi

Everything installs well and also the pip install works fine.

However, when I try to execute weasyprint --help I get the follow error with the copied trace back:

$weasyprint --help
Traceback (most recent call last):
  File "/usr/local/bin/weasyprint", line 8, in <module>
    load_entry_point('WeasyPrint==0.21', 'console_scripts', 'weasyprint')()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 318, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2221, in load_entry_point
    return ep.load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 1954, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/Library/Python/2.7/site-packages/weasyprint/__init__.py", line 312, in <module>
    from .css import PARSER, preprocess_stylesheet
  File "/Library/Python/2.7/site-packages/weasyprint/css/__init__.py", line 30, in <module>
    from . import computed_values
  File "/Library/Python/2.7/site-packages/weasyprint/css/computed_values.py", line 18, in <module>
    from .. import text
  File "/Library/Python/2.7/site-packages/weasyprint/text.py", line 18, in <module>
    import cairocffi as cairo
  File "/Library/Python/2.7/site-packages/cairocffi/__init__.py", line 39, in <module>
    cairo = dlopen(ffi, 'libcairo.so.2', 'libcairo-2.dll', 'cairo', 'libcairo-2')
  File "/Library/Python/2.7/site-packages/cairocffi/__init__.py", line 34, in dlopen
    return ffi.dlopen(names[0])  # pragma: no cover
  File "/Library/Python/2.7/site-packages/cffi/api.py", line 119, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/Library/Python/2.7/site-packages/cffi/api.py", line 412, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/Library/Python/2.7/site-packages/cffi/api.py", line 401, in _load_backend_lib
    return backend.load_library(name, flags)
OSError: cannot load library libcairo.so.2: dlopen(libcairo.so.2, 2): image not found

The reinstall of cairocffi with sudo pip uninstall cairocffi didn't make an difference.

The print out of sys.path and __path__ is listed below

['/usr/local/bin', '/Library/Frameworks/SQLite3.framework/Versions/3/Python/2.7', '/Library/Frameworks/GDAL.framework/Versions/1.9/Python/2.7/site-packages', '/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', '/Library/Python/2.7/site-packages']

['/Library/Python/2.7/site-packages/cairocffi']

The test import of cairocffi brings up this error:

$ python -c 'import cairocffi'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named cairocffi

However cairocffi is installed

pip install cairocffi
Requirement already satisfied (use --upgrade to upgrade): cairocffi in /Library/Python/2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): cffi>=0.6 in /Library/Python/2.7/site-packages (from cairocffi)
Requirement already satisfied (use --upgrade to upgrade): pycparser in /Library/Python/2.7/site-packages (from cffi>=0.6->cairocffi)

Please let me know if you need further tracebacks

[hanneshapke]

sudo brew link --force cairo also did knot solve it for me.

[SimonSapin]

@hanneshapke this sounds like #28. I just pushed v0.5.3 to PyPI with a possible fix. Get it with pip install --upgrade cairocffi. Does it fix the issue for you?

[hanneshapke]

Unfortunately not, problem persists. After the pip install --upgrade cairocffi, the error message is still

OSError: cannot load library libcairo.so.2: dlopen(libcairo.so.2, 2): image not found

I notice something interesting. I read through #28 and noticed the same problem that the imports don't work.

>>> import cffi; cffi.FFI().dlopen('cairo');
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named cffi

Could this maybe the issue?

[SimonSapin]

These look like two separate issues. Are you certain you’re using the same Python install and virtual environment (if any) in both cases?

[hanneshapke]

I think so. When I exit Python and run pip install cffi for example, I get the following message

pip install cffi
Requirement already satisfied (use --upgrade to upgrade): cffi in /Library/Python/2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): pycparser in /Library/Python/2.7/site-packages (from cffi)
Cleaning up...

Pretty sure these are the same environments.

[SimonSapin]

What’s the output of pip --version, which pip, which python, and python -c 'import sys; print(sys.prefix)'?

[hanneshapke]

pip --version pip 1.1 from /Library/Python/2.7/site-packages/pip-1.1-py2.7.egg (python 2.7)

which pip /usr/local/bin/pip

which python /usr/local/bin/python

python -c 'import sys; print(sys.prefix) /usr/local/bin/../Cellar/python/2.7.2

Does it mean, two different python versions are involved?

[SimonSapin]

I’m not sure. What about python -c 'import sys; print('\n'.join(sys.path)) ?

[hanneshapke]

python -c 'import sys; print('\n'.join(sys.path))

/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/pip-1.0.2-py2.7.egg
/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/PIL-1.1.6-py2.7-macosx-10.4-x86_64.egg
/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/distribute-0.6.25-py2.7.egg
/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/virtualenv-1.7.1.2-py2.7.egg
/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/readline-6.2.4.1-py2.7-macosx-10.4-x86_64.egg
/usr/local/Cellar/python/2.7.2/lib/python27.zip
/usr/local/Cellar/python/2.7.2/lib/python2.7
/usr/local/Cellar/python/2.7.2/lib/python2.7/plat-darwin
/usr/local/Cellar/python/2.7.2/lib/python2.7/plat-mac
/usr/local/Cellar/python/2.7.2/lib/python2.7/plat-mac/lib-scriptpackages
/usr/local/Cellar/python/2.7.2/lib/python2.7/lib-tk
/usr/local/Cellar/python/2.7.2/lib/python2.7/lib-old
/usr/local/Cellar/python/2.7.2/lib/python2.7/lib-dynload
/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages

What does it tell you?

[SimonSapin]

Ok, this does look like two different Python versions (one in /Library/Python/2.7/, one in /usr/local/Cellar/python/2.7.2/lib/python2.7/) and does explain why you get different error messages.

The first line of the file indicated by which weasyprint should be a "shebang" that starts with #! and continues with the path to the Python interpreter that’s used. I bet it’s not /usr/local/bin/python.

That explain away the ImportError: No module named cffi error: that Python install has nothing of what you installed with pip.

However, that leaves the OSError: cannot load library libcairo.so.2: dlopen(libcairo.so.2, 2): image not found error. If you still get that with 0.5.3, then I have no idea how to begin to debug it :/

[hanneshapke]

Hi Simon,

Thank you for your detailed help! To your second point first: I just checked pip freeze and it seems I have the 0.5.3 version installed.

$ pip freeze | grep 'cffi'
cairocffi==0.5.3
cffi==0.8.2

Would it make sense to try installing all packages in a fresh virtualenv to avoid any conflicts?

To your second point. Your guess was right. The first line of /usr/local/bin/weasyprint is #!/usr/bin/python, not /usr/local/bin/python. Is this cause by the PYTHONPATH?

[SimonSapin]

I guess one of your Pythons was shipped by Apple and the other one installed by Homebrew. But I don’t know, I don’t use OS X myself. If someone can tell me what dlopen incantation I need to reliably load dynamic libraries on OS X, I’ll be happy to fix this, but I just don’t know what to do here.

[mpaolino]

It looks like cffi might not be able to load dylibs. Can you confirm that? I know ctypes can, it's slower but maybe a fallback to ctypes can be a solution in this case.

[mpaolino]

Correction, cffi does support dylibs. This email seems relevant https://groups.google.com/forum/#!searchin/python-cffi/OSX$20in$20non-system$20Python/python-cffi/ZSj7xd7qAFQ/nJv33dDeFk8J

[SimonSapin]

Thanks for the link @mpaolino

@hanneshapke Try to find out which directory contains the *cairo*.dylib file installed by Homebroew, and start WeasyPrint with the path to that directory in a DYLD_FALLBACK_LIBRARY_PATH environment variable. Eg:

DYLD_FALLBACK_LIBRARY_PATH=/usr/local/Cellar/something/something/lib weasyprint --help

[SimonSapin]

Although I thought that’s what sudo brew link --force cairo was supposed to do.

[Thomas-Schatz]

I installed cairocffi from pip on MacOS 10.6.8, using a python-anaconda distribution and got the same error as the original poster:

import cairocffi
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/Users/thomas/anaconda/lib/python2.7/site-packages/cairocffi/__init__.py", line 16, in <module>
from . import constants
ImportError: cannot import name constants

Changing

from . import constants

to

import constants

in the init.py of cairocffi fixed the issue for me.

[SimonSapin]

import constants

This makes no sense to me. If someone can explain the root cause of this issue I’m happy to fix it, but right now I have no idea what’s going on.

[Thomas-Schatz]

In the version of cairocffi that I got from pip, the code in init.py never uses any of the cairo constants defined in constants.py directly, instead it uses

constants.NAME_OF_THE_CONSTANT

This can work if the module itself has been imported in the current namespace under the name constants, as is the case with:

import constants

but it cannot work if the constants contained in the module have been imported in the current namespace directly as is the case with:

from constants import *

[SimonSapin]

Note that cairocffi/__init__.py contains both of these:

from . import constants
from .constants import *

(Although the latter is at the end of the file.)

The leading dot is important: it indicate explicit relative imports. The equivalent absolute imports are:

from cairocffi import constants
from cairocffi.constants import *

import constants however indicates a top-level constants module that is not in the cairocffi package. Unless it’s an implicit relative import, but these are evil and do not exist in Python 3.

[Thomas-Schatz]

Indeed, sorry, I read the code too fast. I experimented a little more and noticed something that might help resolve the issue. I only get an Import error:

ImportError: cannot import name constants

when I previously got and fixed (by adding the required path to LD_LIBRARY_PATH) an OS error:

OSError: cannot load library libcairo.2.dylib: dlopen(libcairo.2.dylib, 2): image not found

Maybe there is something in the code that is executed before the call to dlopen that isn't interrupted properly if the program fails on the dlopen call and prevents python from executing properly the imports afterwards?