Replacing cling with clang-repl

[a-turko]

Hi, what do you think about replacing cling with clang-repl? From what I read (Cling transitions to LLVM's clang-repl) ) it seems that clang-repl could be a good substitute. This way cppyy would work with the newest clang.

Is something missing from clang-repl that prohibits the change? If not, what would be your estimate of the effort needed for the transition?

[wlav]

Same folks behind both clang-repl and Cling. They're also working on this change, using the more generic libinterop, with currently 207 out of 504 cppyy tests passing. See here for more information.

[a-turko]

Do you have an open task on that front that would take between 1 and 4 weeks for someone unfamiliar with the codebase?

[wlav]

I'm sure there are such tasks; adding @sudo-panda and @vgvassilev who should be better able to answer. Also: https://github.com/compiler-research/CppInterOp

[sudo-panda]

Hi @a-turko, thanks for your interest in the project.

You can have a look at these issues:

• https://github.com/compiler-research/cppyy-backend/issues
• https://github.com/compiler-research/CPyCppyy/issues
• https://github.com/compiler-research/CppInterOp/issues

The good first issues are marked there. For most good first issues, familiarity with the codebase is not required.

Feel free to ask us if you need help setting up the dev environment. I suggest checking out the CI on cppyy for up-to-date steps to set it up.

[a-turko]

I was following the CI config for cppyy for clang-repl and encountered an error when building CppInterOp: CppInterOp/include/clang/Interpreter/Compatibility.h:56:10: fatal error: clang/Interpreter/Value.h: No such file or directory 56 | #include "clang/Interpreter/Value.h"

As far as I can tell this file was never present in CppInterOp, but has been added (under the same path) to llvm, but only in newer releases. However, CI uses llvm16, where this files does not exist. I have consulted the instructions in the CppInterOp repository, but the result was the same.

Have you also run into similar issues? Could you please point me to a possible place where I could have misconfigured something?

[vgvassilev]

I was following the CI config for cppyy for clang-repl and encountered an error when building CppInterOp: CppInterOp/include/clang/Interpreter/Compatibility.h:56:10: fatal error: clang/Interpreter/Value.h: No such file or directory 56 | #include "clang/Interpreter/Value.h"

As far as I can tell this file was never present in CppInterOp, but has been added (under the same path) to llvm, but only in newer releases. However, CI uses llvm16, where this files does not exist. I have consulted the instructions in the CppInterOp repository, but the result was the same.

Have you also run into similar issues? Could you please point me to a possible place where I could have misconfigured something?

You are right. That file will appear in llvm17. For now, we have patched it on top of llvm16 by backporting the patch. You can see in the CI how we apply these patches.

[a-turko]

Thank you. I see the compgen -G "../patches/llvm/clang16-*.patch" > /dev/null && find ../patches/llvm/clang16-*.patch -printf "%f\n" && git apply ../patches/llvm/clang16-*.patch command, but could you tell me where do you get the patches (../patches/llvm directory) from?

[sudo-panda]

Thank you. I see the compgen -G "../patches/llvm/clang16-*.patch" > /dev/null && find ../patches/llvm/clang16-*.patch -printf "%f\n" && git apply ../patches/llvm/clang16-*.patch command, but could you tell me where do you get the patches (../patches/llvm directory) from?

You can find them in the CppInterOp repo: https://github.com/compiler-research/CppInterOp/tree/main/patches/llvm

[a-turko]

Thank you, now the installation went smoothly to the end. However, the command python -c "import cppy" failed with the following:

In file included from <<< inputs >>>:1:
input_line_2:1:10: fatal error: 'iostream' file not found
#include <iostream>
         ^~~~~~~~~~
python: /home/aturko/src/cling-front/llvm-project/clang/include/clang/AST/DeclBase.h:1302: clang::NamedDecl* clang::DeclListNode::iterator::operator*() const: Assertion `Ptr && "dereferencing end() iterator"' failed.
Aborted (core dumped)

Have you also also run into this problem?

Here are the paths that iostream is searched for, but not found:

openat(AT_FDCWD, "/home/aturko/src/cling-front/llvm-project/build/lib/clang/16/../../..//include/iostream", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/aturko/src/cling-front/cppyy-backend/python/cppyy_backend//include/iostream", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/iostream", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/aturko/src/cling-front/llvm-project/build/lib/clang/16/include/iostream", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/local/include/iostream", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/include/x86_64-linux-gnu/iostream", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/include/iostream", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

@sudo-panda, do you have an idea what may be wrong here?

[vgvassilev]

We somehow did not find the path to the stl headers. Could you take a look at how the CI does it?

[a-turko]

I got it when following CI instructions. I'm not so sure about the directory structure though: should all dependencies be cloned directly into the cppyy repository (cppyy directory), or should they just have a commom parent, i.e. all repositories in the same directory?

[vgvassilev]

@a-turko, I believe if you cut and paste carefully these https://github.com/compiler-research/cppyy-backend/blob/4e244e486fb761dfa5559c95929d2d3b23535d0f/.github/workflows/ci.yml#L109-L258 it should work.

Generally, the error tells you that for some reason it cannot find <iostream>. Can you paste the output of echo "" | c++ -fsyntax-only -xc++ - -v?