Can't install latest pycairo (1.20.0) on OSX 10.15

[gijzelaerr]

hi! When I try to pip install pycairo on OS X 10.15 I get the error:

  [WARNING] This wheel needs a higher macOS version than is set in MACOSX_DEPLOYMENT_TARGET variable.  To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 11_0 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:

an older version (1.19.1) installs fine. Seems that 1.20.0 depends on the unreleased OSX beta?

more logs:

  Copying pycairo.egg-info to build/bdist.macosx-10.14.6-x86_64/wheel/pycairo-1.20.0-py3.8.egg-info
  running install_scripts
  [WARNING] This wheel needs a higher macOS version than is set in MACOSX_DEPLOYMENT_TARGET variable.  To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 11_0 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:
  build/bdist.macosx-10.14.6-x86_64/wheel/cairo/_cairo.cpython-38-darwin.soTraceback (most recent call last):
    File "/Users/gijs/Work/surfnet/python-eduvpn-client/venv/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/Users/gijs/Work/surfnet/python-eduvpn-client/venv/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/gijs/Work/surfnet/python-eduvpn-client/venv/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 204, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/private/var/folders/3x/3x7fz5857gbc0ygf80vy0cs00000gq/T/pip-build-env-5f39welu/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 216, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/private/var/folders/3x/3x7fz5857gbc0ygf80vy0cs00000gq/T/pip-build-env-5f39welu/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 202, in _build_with_temp_dir
      self.run_setup()
    File "/private/var/folders/3x/3x7fz5857gbc0ygf80vy0cs00000gq/T/pip-build-env-5f39welu/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 253, in run_setup
      super(_BuildMetaLegacyBackend,
    File "/private/var/folders/3x/3x7fz5857gbc0ygf80vy0cs00000gq/T/pip-build-env-5f39welu/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 145, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 568, in <module>
      main()
    File "setup.py", line 535, in main
      setup(
    File "/private/var/folders/3x/3x7fz5857gbc0ygf80vy0cs00000gq/T/pip-build-env-5f39welu/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/private/var/folders/3x/3x7fz5857gbc0ygf80vy0cs00000gq/T/pip-build-env-5f39welu/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 328, in run
      impl_tag, abi_tag, plat_tag = self.get_tag()
    File "/private/var/folders/3x/3x7fz5857gbc0ygf80vy0cs00000gq/T/pip-build-env-5f39welu/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 278, in get_tag
      assert tag in supported_tags, "would build wheel with unsupported tag {}".format(tag)
  AssertionError: would build wheel with unsupported tag ('cp38', 'cp38', 'macosx_11_0_x86_64')
  ----------------------------------------
  ERROR: Failed building wheel for pycairo

[lazka]

There is nothing macos specific in pycairo. Maybe your pip is confused, try pip cache purge

[gijzelaerr]

seems to have the same result: https://gist.github.com/gijzelaerr/57d58be34d9f98fb3dcc1e82242b6312

[lazka]

thanks. the one thing that changed in 1.20 is that I opted pycairo into "PEP 517" by adding a pyproject.toml file to the package. I guess this triggers some bug in pip your case.

No idea right now, sorry.

[lazka]

There are also some arm64 warnings in there. are you on arm64?

[gijzelaerr]

nope, just a normal x86_64 OSX 10.15. I guess the osx toolchain is already preparing the binaries for the upcoming switch to arm arch.

[lazka]

ok. The only related error I can find is https://old.reddit.com/r/vscode/comments/iwoh5d/help_error_in_installing_pylint_in_virtual/ which isn't that helpful.

Maybe updating python in homebrew would help (the errors suggest it was installed with an older macos version).

[lazka]

And consider filing a bug with wheel: https://github.com/pypa/wheel Maybe they know more.

[gijzelaerr]

Interesting, If I use python 3.9.0 using pyenv the problem goes away. So yes, I guess it is some odd issue in combination with the vanilla python 3.8 bundled with OS X 10.15 and wheel / pip.

[justvanrossum]

I'm curious: why are there no macOS wheels for the latest release on PyPI?

[stuaxo]

@justvanrossum probably some extra work needs doing here - WHL support was getting added, at around the same time travis decided to start charging + everything had to move to github actions.

[infirit]

If I remember correctly there will be a wheel with the next release of pycairo.

[naveen521kk]

Not for macOS though? Maybe if you want macOS wheels, it is easy to build(at least I can help) it but you would need to convince @lazka for that.

[stuaxo]

@naveen521kk If you want help reviewing a PR I'm up for that, though even more time starved than last time :)

I've been away from this long enough to have forgotten what the status of WHLs is here :-/

[lazka]

The problem is that wheels would break pygobject (then there would be two different libcairo in the same process), so I'm not that enthusiastic about it. But I guess pycairo has more users...

[justvanrossum]

No wheels, no painless installation, no happy users. I've honestly been avoiding pycairo because it's such a pain to get installed reliably across platforms. Not every Python user has (the right) development tools installed to build from the sources. (In my circles most do not.)

[stuaxo]

This is something at the edge of my knowledge, is it that if a library is contained in a whl and the system with the same name there is a clash and an issue of which to choose ?

Could we do some mangling of the name when it comes to the one we put in the whl, so we at least have some control over which we choose ?

[naveen521kk]

@lazka Maybe we can upload wheel to the releases section and not to PyPi. User's can install from there if needed and I don't think pip will ever find it. So I think there wont be any problem with pygobject though.

@stuaxo Python handles it. First, priority for loading the library is for the .libs in the wheel. Atleast that is so on unixish OS. This isn't so for windows though, but the first one loaded will be in the same directory, so the wheels do work on windows.

Also, lazka, does pygobject link with pycairo or directly with cairo itself? Because if we statically link the produced output and if pygobject simply links with pycairo there won't be any problem I think.

[naveen521kk]

Again, @lazka does pygobject links with pycairo directly or with Cairo? If we link statically with cairo will pygobject just use the one from pycairo?

[lazka]

pygobject links against both pycairo and cairo.

[naveen521kk]

pygobject queries for pkg-config right? So what if we create those accordingly? such that it doesn't link directly to cairo. and we create wheels where we link to cairo statically?

[lazka]

pygobject gets cairo pointers from gtk and passes them to pycairo and vice versa. I don't think pkg-config is relevant here.

[naveen521kk]

Ok, then nothing can be done to make it work with the wheels provided right? Then I think wheels can be build but not uploaded to PyPI, instead just kept in the releases section, and have a mention of it in the installation section.

[justvanrossum]

Then I think wheels can be build but not uploaded to PyPI, instead just kept in the releases section, and have a mention of it in the installation section.

Then projects can't depend on pycairo automatically, making their CI cumbersome, and making it impossible to install such a project seamlessly with pypi.

I have no clue about the problems that cause this and how to solve them, I'd just like to point out the obvious: that this would be a very sad situation for pycairo indeed.

[naveen521kk]

Maybe upload wheels to testpypi? I could understand the problems with pygobject, yes I don't know a way to fix it though. I at first thought it doesn't link to Cairo directly, instead, it just links to pycairo, which inturn will link to Cairo. If that's the case if we have wheels statically linked to Cairo, then pygobject doesn't need to link with Cairo, instead just linking with pycairo will suffice and the .pc file pycairo generates will also be changed according. But I don't think that is the case as @lazka told. Having two different versions of Cairo installed would create a lot of problems... Rather I would like to have it only in specific condition only when the users need it, the wheels should be used. Or maybe a new pypi package for wheels, but I still don't know which should be the best.

On the other side on Windows, this wasn't a problem at all. pygobject can't be directly installed there, it needs MSYS2 project, so there were no problems creating a wheel.

[stuaxo]

This isn't the only library that's had the issue of wanting to use libraries that may clash with system libraries.

There was some discussion around this for auditwheel here https://discuss.python.org/t/delocate-auditwheel-but-for-windows/2589/8

Dealocate looks like a tool that aims to solve clashes with system libraries on mac, maybe it can solve that part of the problem for us: https://pypi.org/project/delocate/

EDIT: Since we mentioned windows, there is a similar tool: https://pypi.org/project/delvewheel/ [/EDIT]

I have to admit to getting a little confused every time I re-visit this ticket, it might be useful to make a table of the different scenarios.

[naveen521kk]

I don't think that solves the problem of two different Cairo instance and the compatibility between the two.

[stuaxo]

To get on the same page, what are the different versions that may clash ?

Lets assume homebrew the user is on OSX with homebrew + we didn't know about this issue and had WHLs for pycairo and pygobject.

Is this a reasonable list of the cairo libs that could clash ?

cairo installed via homebrew cairo that was installed as part of the pycairo whl cairo that was installed as part of the pygobject whl

(And lets assume they are all different, so either different versions or different compilation options)

[naveen521kk]

Well, what pygobject does is pick the pointer from gtk and give it to pycairo. The pointer can be incompatible if gtk was compiled under different Cairo(either version or compilation options) and could cause Segmentation Faults. And this is the main reason why @lazka is hesitant to build wheels.

[stuaxo]

I'm totally onboard with being cautious.

Is this for any use of pygobject, or only when doing something like drawing with cairo in Gtk ?

What's the smallest amount of code that will show us things are broken, is it simply import gi or do we need to do something more?

I might be able to give you remote access to a mac to try and create the breaking scenarios and see if deallocate can help.

[naveen521kk]

Maybe rather than simply an import statement run the example at https://pygobject.readthedocs.io/en/latest/guide/cairo_integration.html