Building 23.02 with conda results in CLI tools not working

[eric-with-a-c]

Description of Issue

Building USD-23.02 with a conda environment appears to succeed and usdview works. However, some of the command line tools, such as usdcat, do not work. When running usdcat the following error is returned:

dyld[43830]: symbol not found in flat namespace (__Py_NoneStruct)
zsh: abort      PYTHONPATH=$PYTHONPATH:/opt/local/USD-23.02-mini/lib/python PATH=

Below are the steps I took to reproduce the issue with a minimal build of USD-23.02. Taking the same steps but with USD-22.11 results in the CLI tools working as expected.

Steps to Reproduce

1. conda create -n usd_build python=3.9.7 -y
2. conda activate usd_build
3. pip install --no-cache numpy jinja2
4. python $PWD/USD-23.02/build_scripts/build_usd.py --no-imaging -v /opt/local/USD-23.02-mini --build-args USD,"-DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON"
5. PYTHONPATH=$PYTHONPATH:/opt/local/USD-23.02-mini/lib/python /opt/local/USD-23.02-mini/bin/usdcat bushes.usd

System Information (OS, Hardware)

Intel MacOS 12.6.3

[sunyab]

Filed as internal issue #USD-8178

[meshula]

Hi @eric-with-a-c, I encountered this just now. I'm not yet sure why this problem has occurred, but as a workaround until there's more insight into the problem, this worked for me, maybe you could try it and report back?

diff --git a/cmake/defaults/Packages.cmake b/cmake/defaults/Packages.cmake
index 2bb1f30d0fc..7c2c71149b3 100644
--- a/cmake/defaults/Packages.cmake
+++ b/cmake/defaults/Packages.cmake
@@ -100,11 +100,11 @@ if(PXR_ENABLE_PYTHON_SUPPORT)

         # This option indicates that we don't want to explicitly link to the
         # python libraries. See BUILDING.md for details.
-        if(PXR_PY_UNDEFINED_DYNAMIC_LOOKUP AND NOT WIN32)
-            set(PYTHON_LIBRARIES "")
-        else()
-            set(PYTHON_LIBRARIES "${package}::Python")
-        endif()
+        #if(PXR_PY_UNDEFINED_DYNAMIC_LOOKUP AND NOT WIN32)
+        #    set(PYTHON_LIBRARIES "")
+        #else()
+        set(PYTHON_LIBRARIES "${package}::Python")
+        #endif()
     endmacro()

     # USD builds only work with Python3

[eric-with-a-c]

Hello @meshula, I just gave the workaround you mentioned a try. The build succeeded and the CLI tools work too. Thanks so much for figuring out what was causing the issue!

[meshula]

Thanks for the confirmation! We can use this issue as a breadcrumb for people until we discover the root case for the regression with conda.

[sjcengine]

Hi @meshula,

I have this issue too, usdview worked but usdcat got the above error.

I followed your fix and the usdcat works but usdview now comes up with:

------------------------------ python3 terminated ------------------------------
python3 crashed. FATAL ERROR: Failed axiom: ' Py_IsInitialized() '
in operator() at line 163 of /Users/--/Desktop/Repos/USDenv/USD/pxr/base/tf/pyTracing.cpp
writing crash report to [ H337-VH0Y3N709C:/var/folders/rq/27cnp9zs5s3808zhq53l8jpm0000gp/T//st_python3.15432 ] ... done.
--------------------------------------------------------------------------------

Just a note, I'm on an M2 MBP.

Hopefully I can get both to work!

Best, Simon

[meshula]

Could you attach that crash report?

[siva-nagendra]

Hey guys, I have the same issue.

Below are the steps to reproduce:

1. conda create --name usd python=3.10.13
2. conda activate usd
3. pip install numpy PyOpenGL PySide6
4. python OpenUSD/build_scripts/build_usd.py /opt/local/OpenUSD
5. usdcat --usdFormat usda "/Users/siva/decor_jar01_mesh.usdc"

I get the below error after the last command: dyld[43403]: symbol not found in flat namespace '__Py_NoneStruct' [1] 43403 abort usdcat --usdFormat usda

I am using an M2 Macbook Pro 2023.

Any help is much appreciated.

Regards, Siva

[meshula]

@siva-nagendra Could you otool -l usdcat and attach the output here as a text file?

[siva-nagendra]

@meshula Thank you for taking a look.

Attaching the otool -l usdcat output as a text file.

otool_log.txt

[meshula]

Ok, can you also do it for libboost_python310.dylib? (I'm checking the rpaths to see if they are resolving python from a funny place) (Also, last I checked, conda static links python, so I'm checking to see if there's a dynamic python coming from somewhere else)

[siva-nagendra]

Here is the output from otool -l /Users/siva/miniconda3/envs/usd/lib/libboost_python310.dylib: otool_libboost_log.txt

(Boost wasn't installed previously in this env. Before I ran otool, I installed boost in my conda env by conda install -c conda-forge boost.)

May be not related but I tried usdcat after installing boost, it errors the same way. dyld[50782]: symbol not found in flat namespace '__Py_NoneStruct' [1] 50782 abort /opt/local/OpenUSD/bin/usdcat --usdFormat usda

[meshula]

Thanks, nothing unusual there.

The problem is summarized here in a cpython issue: https://github.com/python/cpython/issues/97524#issuecomment-1458855301 Wenzel's solution for pybind11 is very heavy https://github.com/wjakob/nanobind/commit/8e777d2dcffbe22fbebc9064909094dc52905b0d

It seems consistent with that investigation that adding this cmake setting SHOULD work, without modifying the build scripts or cmake scripts at all. As far as I can tell our cmake code is set up properly.

-DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON

Did you set it?

[jonlm]

Hi, I just ran into this issue as well. Based on reading through this thread and trying out a few different things, here is what I found:

• If you build openusd against a statically-linked python installation on macos then by default you get PXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON. In this scenario you can successfully run python programs that import usd packages, but any standalone usd binary in the build fails with the original "__Py_NoneStruct" error.

• If you force PXR_PY_UNDEFINED_DYNAMIC_LOOKUP=OFF, the reverse happens: standalone binaries work, but python programs fail with the "Py_IsInitialized()" error mentioned above.

Is it expected that usd would be able to link against a static python and support both python packages and standalone binaries, all while usd itself is built as a shared lib? Intuitively I kind of get why only one or the other would work. If libpython is static then it can't be included if usd libraries are imported into python. But it must be included if usd is running without python. It seems like this could only work if standalone binaries were built/linked in a different manner than the python packages.

(I've tried to read through the cpython/pybind11 threads linked above but they went way over my head.)

[meshula]

Thanks for the notes. What a puzzle...

[jeffzhkw]

Can confirm in 24.05, the problem with dyld[50782]: symbol not found in flat namespace '__Py_NoneStruct' still exists. My workaround is to use venv rather than conda.

[flipphillips]

This is quite fascinating. Just ran into this myself and have made same observations re: DYNAMIC_LOOKUP .... will try venv, but am equally boggled.

[jonlm]

Continuing from my comments above, I believe the problem is due to the fact that python can call into USD via the pxr python package and USD can call into python regardless of whether it is loaded inside of python. This is due to one particular feature: plugins of type python, i.e. any plugInfo.json that specifies "Type": "python". No such plugins ship with USD itself, apart from some tests. (Luca Scheller has an example of one.) Nonetheless, it makes sense that USD can't be linked with static python and support being dynamically loaded into a python host at the same time since you couldn't have a python module carrying its own libpython.a, barring some kind of arrangement whereby it uses its own separate python interpreter (if that's even possible). This is why you end up with the results in my post where either binaries work or the python package works, but not both. Everything "just works" with a dynamic libpython because USD can be the parent and/or child of python without having to decide at link time if it's going to drag along its own interpreter.

It seems like this is not a macOS-specific issue, but only coming up on mac because of conda using static python for whatever reason on that platform. Or perhaps this is some mac-specific linker thing and I'm wrong on all accounts. :)

If my hypothesis is correct, I think it would be nice if there were a build flag that said "I want to build the USD python modules for static python but I'm willing to forgo python USD plugins," which should in theory make it possible for everything to work. Or maybe it just disables USD python plugins by default if python is static.