[BUG] ImportError of GLIBCXX in Catalyst frontend

Issue description

I am experiencing issues when importing Catalyst that seem to depend on where and how the library is imported. It might be specific to my installation (I had to hack around a bit to get it to pick up clang), but I thought I should report because the failure seems erratic. I think it is related to something in the frontend.

• Expected behavior: (What you expect to happen)

Importing Catalyst from both shell, interpreter, and Jupyter notebook should work without errors.

• Actual behavior: (What actually happens)

When I fire up an interpreter, I can do

>>> from catalyst import qjit

or

>>> import catalyst

without issue.

If I have a Python script, and begin it with

from catalyst import qjit

it works when this is the only line in the script, but in other contexts (my Shor implementation), I receive the following Traceback:

Traceback (most recent call last):
  File "/home/olivia/Code/shortalyst-dev/full_jit_working.py", line 10, in <module>
    from catalyst import cond, measure, qjit, for_loop, while_loop
  File "/home/olivia/Code/catalyst/frontend/catalyst/__init__.py", line 64, in <module>
    from catalyst.compilation_pipelines import QJIT, CompileOptions, qjit
  File "/home/olivia/Code/catalyst/frontend/catalyst/compilation_pipelines.py", line 40, in <module>
    from catalyst.compiler import CompileOptions, Compiler
  File "/home/olivia/Code/catalyst/frontend/catalyst/compiler.py", line 30, in <module>
    from mlir_quantum.compiler_driver import run_compiler_driver
ImportError: /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so)

However, adding a line above this import that does import catalyst solves the problem

In a Jupyter Notebook, neither version works, and I get a similar but more detailed error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[2], line 1
----> 1 import catalyst
      3 from catalyst import qjit

File ~/Code/catalyst/frontend/catalyst/__init__.py:64
     58 sys.modules["mlir_quantum._mlir_libs._quantumDialects.quantum"] = types.ModuleType(
     59     "mlir_quantum._mlir_libs._quantumDialects.quantum"
     60 )
     63 from catalyst.ag_utils import AutoGraphError, autograph_source
---> 64 from catalyst.compilation_pipelines import QJIT, CompileOptions, qjit
     65 from catalyst.pennylane_extensions import (
     66     adjoint,
     67     cond,
   (...)
     74     while_loop,
     75 )
     76 from catalyst.utils.exceptions import CompileError

File ~/Code/catalyst/frontend/catalyst/compilation_pipelines.py:40
     38 import catalyst
     39 from catalyst.ag_utils import run_autograph
---> 40 from catalyst.compiler import CompileOptions, Compiler
     41 from catalyst.jax_tracer import trace_to_mlir
     42 from catalyst.pennylane_extensions import QFunc

File ~/Code/catalyst/frontend/catalyst/compiler.py:30
     27 from io import TextIOWrapper
     28 from typing import Any, List, Optional
---> 30 from mlir_quantum.compiler_driver import run_compiler_driver
     32 from catalyst._configuration import INSTALLED
     33 from catalyst.utils.exceptions import CompileError

ImportError: /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so)

• Reproduces how often: (What percentage of the time does it reproduce?)

Erratic (see description above)

• System information: (post the output of import pennylane as qml; qml.about())

Running on Ubuntu 22.04.

Catalyst is installed from source off of main branch commit 422dc14879071039a9a64eb781627b6b441174de.

Clang++ is version 14.0.0-1ubuntu1.1.

Output of qml.about() (packages installed from Catalyst's requirements.txt file):

Name: PennyLane
Version: 0.32.0
Summary: PennyLane is a Python quantum machine learning library by Xanadu Inc.
Home-page: https://github.com/PennyLaneAI/pennylane
Author: 
Author-email: 
License: Apache License 2.0
Location: /home/olivia/.conda/envs/catalyst/lib/python3.11/site-packages
Requires: appdirs, autograd, autoray, cachetools, networkx, numpy, pennylane-lightning, requests, rustworkx, scipy, semantic-version, toml, typing-extensions
Required-by: pennylane-catalyst, PennyLane-Lightning

Platform info:           Linux-6.2.0-33-generic-x86_64-with-glibc2.35
Python version:          3.11.0
Numpy version:           1.23.5
Scipy version:           1.10.0
Installed devices:
- default.gaussian (PennyLane-0.32.0)
- default.mixed (PennyLane-0.32.0)
- default.qubit (PennyLane-0.32.0)
- default.qubit.autograd (PennyLane-0.32.0)
- default.qubit.jax (PennyLane-0.32.0)
- default.qubit.tf (PennyLane-0.32.0)
- default.qubit.torch (PennyLane-0.32.0)
- default.qutrit (PennyLane-0.32.0)
- null.qubit (PennyLane-0.32.0)
- lightning.qubit (PennyLane-Lightning-0.32.0)

Source code and tracebacks

N/A

Additional information

Let me know what else would be useful!

Thanks for reporting this error @glassnotes! It looks to be related to the new compiler driver integration.

Thanks for the quick reply, @erick-xanadu !

Some more info for you: upon further testing, in a script, if I import packages in the following order,

import catalyst
import jax.numpy as jnp

I do not get an error, however if I reverse the two lines, I get the error listed above. I've just updated both jax and jaxlib to be versions 0.4.14.

Hi @glassnotes, Thanks for reporting this issue.

This comes from the fact that you have multiple versions of libstdc++ on your machine. The compiler_driver library of Catalyst was apparently built with a different version of this library. You can find it by running ldd /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so. The path to libstdc++.so should be something like /lib/x86_64-linux-gnu/libstdc++.so.6.

The conda version of libstdc++ doesn't include GLIBCXX_3.4.30 while the gcc package (in /lib/libstdc++.so.6) has it! The compiler_driver is linked against the gcc package but it doesn't find the lib when it's called from a conda venv.

To fix this issue, you can replace the conda version of this library by this one,

cp /lib/x86_64-linux-gnu/libstdc++.so.6 /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6

Please let us know if this also fixes the issue or you face any other problems with installing Catalyst.

The question is, how do we prevent this from happening more generally? 🤔

Copying the library in does work, but feels a little sketchy :sweat_smile:

I ended up with multiple versions of the library during the installation process. In particular, despite having clang installed on my machine, I initially received the output

CMake Error at /home/olivia/.conda/envs/catalyst/lib/python3.11/site-packages/cmake/data/share/cmake-3.27/Modules/CMakeTestCXXCompiler.cmake:60 (message):
  The C++ compiler

    "/usr/bin/clang++"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: '/home/olivia/Code/catalyst/runtime/build/CMakeFiles/CMakeScratch/TryCompile-k56B71'
    
    Run Build Command(s): /usr/bin/ninja -v cmTC_54cbb
    [1/2] /usr/bin/clang++    -MD -MT CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -MF CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o.d -o CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -c /home/olivia/Code/catalyst/runtime/build/CMakeFiles/CMakeScratch/TryCompile-k56B71/testCXXCompiler.cxx
    [2/2] : && /usr/bin/clang++   CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -o cmTC_54cbb   && :
    FAILED: cmTC_54cbb 
    : && /usr/bin/clang++   CMakeFiles/cmTC_54cbb.dir/testCXXCompiler.cxx.o -o cmTC_54cbb   && :
    /usr/bin/ld: cannot find -lstdc++: No such file or directory
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    ninja: build stopped: subcommand failed.
    
  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:3 (project)

when trying to run make all in the top-level Catalyst directory. This led me to installing libstdc++-12-dev, which fixed the previous problem, but resulted in the new one in the issue description.

For what it's worth, the "not being able to compile a simple test program" happened on more than one of my machines, and I had installed previous versions by manually adjusting the Makefiles to use g++ instead of clang; but today I thought I'd try and fix things to work out-of-the-box...

Thanks for checking! I can feel the pain :cry: we're discussing the issue and should be able to come up with a better solution.

Such feedback helps us gain a deeper understanding of edge cases, and we're committed to working on robust solutions that make the process of installing Catalyst from source even smoother.

Still investigating this, but we might need to recommend users not use conda when building from source.

Hi @glassnotes, we actually have an update on this because the problem came up again. The issue is that conda ships with a version of the C++ standard library (libstdc++.so.6) that is older than what the Catalyst package was compiled with. In addition, when a conda environment is active it will put its own libraries ahead of the system libraries, so even though most modern distributions (say Ubuntu 20+) will have the right libraries conda injects an older one that is incompatible.

The solution is quite simple in this case, updating the relevant conda package with:

conda install -c conda-forge libstdcxx-ng

Hope this helps if you ever run into this issue again.

Just wanted to confirm, but it looks like a conda environment was active when you received the error:

ImportError: /home/olivia/.conda/envs/catalyst/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/olivia/Code/catalyst/frontend/catalyst/../../mlir/build/python_packages/quantum/mlir_quantum/compiler_driver.so)

is that correct?

Deactivating the environment would also solve the error from what I saw during testing.