astral-sh
`uv pip install open3d` installs python3 package but has python2 init file
Hi, when I tried to use uv to install open3d (and other packages), I encountered a weird issue which I got a python2 init.py file while the whole package is in python3. Basically I can fix the package by copying this file to lib/python3.11/site-packages/open3d/__init__.py.
I couldn't reproduce this result consistently, so it is really hard to debug :\ Wonder if anyone has encountered similar issue before? Thanks!
I would love to help but would need something that's more easily reproducible.
I would love to help but would need something that's more easily reproducible.
Thank you! The following is a repro snippet
mamba create \
--prefix $HOME/.conda/envs/test_uv2 \
--channel conda-forge \
--channel anaconda \
--channel pytorch \
--yes \
python==3.11.6 cmake cython libopenblas libprotobuf libunwind "pip>=22.2" psycopg2 py-opencv==4.7.0 pybind11
conda activate $HOME/.conda/envs/test_uv2
pip install uv
UV_INDEX_STRATEGY="unsafe-best-match" UV_NO_BUILD_ISOLATION="1" uv pip install jobqueue==3.0.4 msgpack numpy==1.23.5 open3d opencv-contrib-python-headless==4.6.0.66 pyspark==3.5.0 pythreejs python-geohash PyOpenGL-accelerate PyTurboJPEG rtree sbp scipy==1.10.0 sentry_sdk==1.30.0 smplx[all] splines==0.3.0 streamlit==1.23.0 "sympy>=1.1.1" syrupy tb-nightly==2.5.0a20210402 tox typing-inspect twine visdom==0.1.8.3 warp-lang
uv version
❯ uv --version
uv 0.2.37
mamba version
❯ mamba --version
mamba 1.4.2
conda 23.3.1
os
❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.2 LTS
Release: 22.04
Codename: jammy
❯ uname -a
Linux se02ln002 5.15.0-75-generic #82-Ubuntu SMP Tue Jun 6 23:10:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Thanks! I'll take a look (but probably not today -- prepping for a release).
Thank you! The error message is:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/robert1003/.conda/envs/test_uv2/lib/python3.11/site-packages/open3d/__init__.py", line 9, in <module>
from open3d.linux import *
File "/home/robert1003/.conda/envs/test_uv2/lib/python3.11/site-packages/open3d/linux/__init__.py", line 7, in <module>
globals().update(importlib.import_module('open3d.linux.open3d').__dict__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/robert1003/.conda/envs/test_uv2/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: /home/robert1003/.conda/envs/test_uv2/lib/python3.11/site-packages/open3d/linux/open3d.so: undefined symbol: _Py_ZeroStruct
Another weird thing I observe is that this is not reproducible on all machine for me, which make me suspect that there might be some kind of race condition?
I cannot reproduce it in another Ubuntu 22.04 machine
❯ uname -a
Linux se01ln001 5.19.0-45-generic #46~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jun 7 15:06:04 UTC 20 x86_64 x86_64 x86_64 GNU/Linux
I can reproduce it on the following 20.04 machine
❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04 LTS
Release: 20.04
Codename: focal
❯ uname -a
Linux ln001 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
but cannot reproduce the error on the following 20.04 machine:
❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal
❯ uname -a
Linux login-dh1-r517u02 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
What is your open3d version inside /home/robert1003/.conda/envs/test_uv2/lib/python3.11/site-packages when this happens? Is it 0.3.0 (what you linked to) since https://pypi.org/project/open3d/0.3.0/ does not seem to exist in the pypi (maybe some other registry?).
I also wonder if you're hitting some kind of edge case that ends up getting your the oldest open3d available, e.g. something like (but not exactly the same) the case described in https://github.com/astral-sh/uv/issues/6141
Do you have logs using e.g. RUST_LOG=trace or --verbose when the error occurs that you could share?
the version is 0.18.0:
❯ pip list | grep open3d
pip is now uv; running uv pip...
open3d 0.18.0
open3d-python 0.3.0.0
❯ which python3
/home/robert1003/.conda/envs/test_uv2/bin/python3
❯ python3 -c 'import open3d as o3d'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/robert1003/.conda/envs/test_uv2/lib/python3.11/site-packages/open3d/__init__.py", line 9, in <module>
from open3d.linux import *
File "/home/robert1003/.conda/envs/test_uv2/lib/python3.11/site-packages/open3d/linux/__init__.py", line 7, in <module>
globals().update(importlib.import_module('open3d.linux.open3d').__dict__)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/robert1003/.conda/envs/test_uv2/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: /home/robert1003/.conda/envs/test_uv2/lib/python3.11/site-packages/open3d/linux/open3d.so: undefined symbol: _Py_ZeroStruct
yeah I am very confused that it fetched that specific init file, it took me a while to find where is that file from...
Do you have logs using e.g. RUST_LOG=trace or --verbose when the error occurs that you could share?
I will add this flag and retry again
I also wonder if you're hitting some kind of edge case that ends up getting your the oldest open3d available, e.g. something like (but not exactly the same) the case described in https://github.com/astral-sh/uv/issues/6141
Interesting... will take a look
❯ pip list | grep open3d
pip is now uv; running uv pip...
open3d 0.18.0
open3d-python 0.3.0.0
Ok, this helped a lot.
It seems having both open3d==0.18.0 and open3d-python=0.3.0.0 is incompatible as they overwrite each other's namespaces. Depending on the order of installation, you could end up with files from one or the other in the same site-packages/open3d location.
Here's on open3d-python=0.3.0.0 on PyPI. The sdist shows me it uses open3d namespace with a very old build of open3d (2018).
It's possible one of your dependencies is introducing open3d-python, worth finding out which one it is. Avoid installing open3d-python as it's incompatible with open3d==0.18.0 and unmaintained since 2019. It's either one or the other 😅
Thank you for figuring this out! It appears that smplx[all] is the one pulling in open3d-python. I just removed that and it works. Thank you for your help!