no attribute 'CppyyLegacy'

[lgiacome]

Hello,

In my research group we have a code using cppyy (https://gitlab.cern.ch/IRIS/IW2D). Most of our users run their simulations on the CERN lxplus cluster, but recently we have had issues with cppyy there.

A minimal example of what we do is the following:

import cppyy
cppyy.load_library('gsl')

But we get the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 cppyy.load_library('gsl')

File ~/miniconda3/envs/cppyy_env/lib/python3.12/site-packages/cppyy/__init__.py:275, in load_library(name)
    273 def load_library(name):
    274     """Explicitly load a shared library."""
--> 275     with _stderr_capture() as err:
    276         gSystem = gbl.gSystem
    277         if name[:3] != 'lib':

File ~/miniconda3/envs/cppyy_env/lib/python3.12/site-packages/cppyy/__init__.py:214, in _stderr_capture.__init__(self)
    213 def __init__(self):
--> 214    self._capture = not gbl.CppyyLegacy.gDebug and True or False
    215    self.err = ""

AttributeError: <namespace cppyy.gbl at 0x55eb1ef22460> has no attribute 'CppyyLegacy'. Full details:
  type object '' has no attribute 'CppyyLegacy'
  'CppyyLegacy' is not a known C++ class
  'CppyyLegacy' is not a known C++ template
  'CppyyLegacy' is not a known C++ enum

We install cppyy with conda as follows:

wget https://repo.anaconda.com/miniconda/Miniconda3-py310_24.7.1-0-Linux-x86_64.sh
bash Miniconda3-py310_24.7.1-0-Linux-x86_64.sh
source miniconda3/bin/activate
conda create -n cppyy_env
conda install -c conda-forge gsl cppyy

Interestingly if I try doing exactly the same on a Ubuntu machine everything works fine.

Probably this is not bug in cppyy, but I am wondering if the error is something well-known and if it points clearly to some set-up issues on lxplus.

[wlav]

Since yours is a CERN project, my first question would be whether you have ROOT somewhere on that box. Point being that ROOT's cppyy is mostly the same, but does not contain CppyyLegacy (aka ROOT).

[lgiacome]

Many thanks for your answer!

It is true that ROOT seems to be installed by default on lxplus (i.e. there is a pre-installed /usr/bin/root executable), but I would have hoped that installing a conda environment I am shielded from what is installed by default..

When I install cppyy with conda, does it use any specific environment variable to check if ROOT is installed? I am wondering if I could modify these variables to not see this root executable, if it is the issue.

[wlav]

Not sure, I'm not too familiar with the conda release nor the ROOT install on lxplus. However, the PCH (which would containt CppyyLegacy) is build using rootcling, which may have been picked up the ROOT rootcling from /usr/bin (yes, it really shouldn't be as the conda path should preceed it). Or maybe $ROOTSYS is still used somewhere to find the ROOT/meta headers (as opposed to finding the ones installed using conda). I'd figure that if ROOT is pre-installed on lxplus, then $ROOTSYS is preset as well? (I just checked; it's not for me.)

If you want to be sure, you can use strace to see which files are being read in, which will show whether anything from /usr is picked up.

[lgiacome]

Hi, Thanks for checking. $ROOTSYS is not set for me either.

If I do strace python -c 'import cppyy' I can indeed see that it picks up stuff like /usr/lib64/root/libCling.so.6.32.0 so clearly you are pointing at the right issue. Now I don't really understand if I can somehow fool pip/conda to think that there is no ROOT installed, but I am also wondering if it could be possible to add an install flag saying to force the complete installation of cppyy. Would that be doable?

[lgiacome]

By the way, we tried also installing cppyy with pip (and also pipping directly on the source code) and we get the same error, so it is not specific to conda.

[wlav]

Ah yes, of course: the libraries are in your case picked up through LD_LIBRARY_PATH and the cppyy_backend ones aren't in there, but are available only through the Python package path. This is the loader code:

 # normal load, allowing for user overrides of LD_LIBRARY_PATH
    try:
        return ctypes.CDLL(bkname, ctypes.RTLD_GLOBAL), errors
    except OSError as e:
        errors.add(str(e))

 # failed ... load dependencies explicitly
    try:
        pkgpath = os.path.dirname(bkname)
        if not pkgpath:
            pkgpath = os.path.dirname(__file__)
        elif os.path.basename(pkgpath) in ['lib', 'bin']:
            pkgpath = os.path.dirname(pkgpath)
        for dep in ['libCoreLegacy', 'libThreadLegacy', 'libRIOLegacy', 'libCling']:
            for loc in ['lib', 'bin']:
                fpath = os.path.join(pkgpath, loc, dep+soext)
                if os.path.exists(fpath):
                    ldtype = ctypes.RTLD_GLOBAL
                    if dep == 'libCling': ldtype = ctypes.RTLD_LOCAL
                    ctypes.CDLL(fpath, ldtype)
                    break
        return ctypes.CDLL(os.path.join(pkgpath, 'lib', bkname), ctypes.RTLD_GLOBAL), errors
    except OSError as e:
        errors.add(str(e))

So yes, this is a feature: you can override locations, which is what happens here. Adding the package path to LD_LIBRARY_PATH would fix this, but that's not part of the conda setup.

That doesn't explain the non-existence of CppyyLegacy per se, but I would guess that the libCling.so from the base install uses modules by default, with the PCH disabled, and it finds those modules alongside in /usr/lib64/root, which points to that installs headers and thus no CppyyLegacy. If so, then picking up the correct libCling.so would fix that.

Note that cppyy in ROOT 6.32.0 is pretty close to master (this was a recent effort, with now the biggest difference being that ROOT's uses LLVM16 instead of LLVM13 in conda/pip, which is most likely a plus). You could just not install cppyy in conda if on lxplus, but pick up the ROOT one instead.

[lgiacome]

Hi,

Thanks a lot for all the suggestions and explanations.

I checked and if I set LD_LIBRARY_PATH to $CONDA_PREFIX/lib/python3.11/lib/cppyy_backend/lib, which is where libCLing.so is located, the problem disappears and I find cppyy.gbl.CppyyLegacy. In principle this fixes the issue 🥳

Regarding your last comment, which sounds very interesting, in practice how would you suggest to pick the cppyy from ROOT?

[wlav]

It's pre-installed on lxplus, with ROOT. Only caveats seem to be that you need to set CLING_STANDARD_PCH=none (this is not a good idea otherwise, but it'll use modules with ROOT instead of the PCH, so needed there) and CPPYY_API_PATH=/usr/include/root to allow it to find its own headers.

This works, without any further setup (clean login), for me:

[lxplus949] ~ % CLING_STANDARD_PCH=none CPPYY_API_PATH=/usr/include/root python3
Python 3.9.18 (main, Aug 23 2024, 00:00:00) 
[GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cppyy
>>> 

I appreciate that things will fall apart when using any different version of Python, so if py3.9 won't do, then setting LD_LIBRARY_PATH is the better way.

Alternatively:

[lxplus949] ~ % python3                                               
Python 3.9.18 (main, Aug 23 2024, 00:00:00) 
[GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ROOT
>>> import cppyy
>>> 

but that'll load some more stuff (e.g. the ROOT pythonizations).

[wlav]

Closing b/c it seems completed. Feel free to reopen or start a new issue if there are more questions.

[wiso]

Hello, I hit exactly the same problem. The issue has been closed, but I don't see a solution.

Summary: this error appears on systems when ROOT is installed, even if cppyy is installed inside virtual envs or conda (which should guarantee more isolation).

To reproduce:

import cppyy
cppyy.include("anything.cpp")

cppyy.include("anything.cpp")
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    cppyy.include("anything.cpp")
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/turra/ttt/lib64/python3.13/site-packages/cppyy/__init__.py", line 281, in include
    with _stderr_capture() as err:
         ~~~~~~~~~~~~~~~^^
  File "/home/turra/ttt/lib64/python3.13/site-packages/cppyy/__init__.py", line 192, in __init__
    self._capture = not gbl.CppyyLegacy.gDebug and True or False
                        ^^^^^^^^^^^^^^^
AttributeError: <namespace cppyy.gbl at 0x558896135da0> has no attribute 'CppyyLegacy'. Full details:
  type object '' has no attribute 'CppyyLegacy'
  'CppyyLegacy' is not a known C++ class
  'CppyyLegacy' is not a known C++ template
  'CppyyLegacy' is not a known C++ enum

this error doesn't appear if ROOT is not installed (e.g. Docker)

I don't know the context, but in addition, it is not clear to me what something and True and something or False means. Except for the cast, they should both be equivalent to something.

Skipping the line and setting self._capture to False seems to fix the problem.

I don't know why but it seems cppyy.gbl is including, in addition to what defined in anything.cpp, all the ROOT classes: not sure why.

[wlav]

I have no other answer than it is clearly picking up ROOT's version of cppyy. Yes, this should not happen with a virtual environment, but clearly it does. I can go over the possible things to check, but everything is already listed above.

Note that the ROOT team has been working to sync its version of cppyy with master, so it should be fine to just use ROOT's one if you can't disentangle the environments.

[wiso]

why you are saying it is picking the ROOT's version of cppyy? From the stack trace it is using modules from /home/turra/ttt, which is the virtual env.

[wlav]

Well, ROOT doesn't have CppyyLegacy, master does, so that diagnosis is clear. As for the stack trace, it only show the top-level cppyy module, but the bindings live in libcppyy.so and the problem is more likely in the 3rd module, namely libcppyy_backend.so and what pulls in (in particular, either the PCH or the headers).

libcppyy.so is found through PYTHONPATH, libcppyy_backend.so is found through LD_LIBRARY_PATH or otherwise in the cppyy_backend module through PYTHONPATH. (Note the order.) The PCH and headers are found in the latter module, but other parts of the environment may affect Cling's search path, could be set by ROOT, and could pre-empt Cling's searching, nothing to do with the venv.

I reiterate that the ROOT team has been working hard on compatibility. I recommend using only one or the other.