ci: test pypy upstream

[henryiii]

Description

Adding a nox test for PyPy upstream before their next release. I think this works on 3.7 and 3.8, though we should see if we still need our workarounds, and fails on PyPy 3.9, but haven't really looked into it more than just adding the tests. One test was regularly xpassing for me. CC @mattip.

Suggested changelog entry:

* Support testing development versions of PyPy (UNIX only) via nox

[henryiii]

Hmmm, CMake failing to find PyPy 3.9 but reporting 3.9 is a valid version was not what I was expecting. That works correctly locally.

[henryiii]

This is the local error:

nox > cmake --build .nox/pypy_upstream-3-9/tmp --target pytest
[0/2] Re-checking globbed directories...
[0/1] cd /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/tests && /Users...tests/test_union.py /Users/henryschreiner/git/software/pybind11/tests/test_virtual_functions.py
libc++abi: terminating with uncaught exception of type pybind11::error_already_set: IndentationError: ('unexpected indent', ('<string>', 2, 1, '        class pybind11_static_property(property):\n', 2))

This actually seems correct, the code has a leading indent. Not sure why PyPy < 3.9 likes it, actually.

[Skylion007]

Okay, so we should definitely make a PR to fix that leading indent then.

[henryiii]

With that fix, Python tests pass, then the CMake tests fails with:

======== CMake output     ======
Found pybind11: /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/mock_install/include (found version "2.10.0dev1")
Found pybind11 v2.10.0 dev1: /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/mock_install/include;/Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/pypy-c-jit-104891-17e3c4e34398-osx64/include/pypy3.9/
Configuring done
Generating done
Build files have been written to: /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/tests/test_cmake_build/installed_function
======== End CMake output ======
Change Dir: /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/tests/test_cmake_build/installed_function

Run Build Command(s):/Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/bin/ninja check_installed_function && [1/2] Linking CXX shared library test_cmake_build.pypy39-pp73-darwin.so
FAILED: test_cmake_build.pypy39-pp73-darwin.so
: && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -O3 -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -dynamiclib -Wl,-headerpad_max_install_names  -o test_cmake_build.pypy39-pp73-darwin.so -install_name @rpath/test_cmake_build.pypy39-pp73-darwin.so CMakeFiles/test_installed_function.dir/Users/henryschreiner/git/software/pybind11/tests/test_cmake_build/main.cpp.o  /usr/local/lib/libpypy3-c.dylib && :
Undefined symbols for architecture x86_64:
  "_PyPyCMethod_New", referenced from:
      pybind11::cpp_function::initialize_generic(std::__1::unique_ptr<pybind11::detail::function_record, pybind11::cpp_function::InitializingFunctionRecordDeleter>&&, char const*, std::type_info const* const*, unsigned long) in main.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

[Skylion007]

Nice, PyPy, 3.7 and 3.8 are working now.

[henryiii]

3.7 and 3.8 were always working. It's just 3.9 that's broken - via CMake on CI and by _PyPyCMethod_New missing (have no idea what that is) locally - only in one of the three CMake compile tests, I think.

[mattip]

As far as I can tell, PyPy3.9 has the include file pypy_decl.h with #define PyCMethod_New PyPyCMethod_New, and CPython has the include file methodobject.h with PyAPI_FUNC(PyObject *) PyCMethod_New(PyMethodDef *, PyObject *, .... I wonder where _PyPyCMethod_New with the underline is coming from?

[mattip]

I think CMAKE is not finding the include files nor the library for PyPy3.9.

[henryiii]

Yes, but it is finding it for 3.8 & 3.7, AND it is finding it locally on my Mac for 3.9, even though everything should be identical - same pypy download, same cmake, etc...

[mattip]

it is finding it locally on my Mac for 3.9,

Do the tests pass?

[henryiii]

The normal tests pass, then the build tests fail with the error listed above.

[mattip]

What would be the easiest way to run those specific tests locally?

[mattip]

Got ~it~ how to run tests. Locally on Ubuntu 20.04, pipx run nox -s 'pypy_upstream(3.9)' from this branch fails the test_python_alreadyset_in_destructor test.

Edit: clarify which "it" I got.

[mattip]

I am not seeing any failure with _PyPyCMethod_New.

[henryiii]

Let me revert the last commit, that should get past that test.

[henryiii]

Ahaha, I bet this is what's breaking CMake: https://github.com/scikit-build/scikit-build/pull/673 (at least it's a start)

[DarkWingMcQuack]

This is the local error:

nox > cmake --build .nox/pypy_upstream-3-9/tmp --target pytest
[0/2] Re-checking globbed directories...
[0/1] cd /Users/henryschreiner/git/software/pybind11/.nox/pypy_upstream-3-9/tmp/tests && /Users...tests/test_union.py /Users/henryschreiner/git/software/pybind11/tests/test_virtual_functions.py
libc++abi: terminating with uncaught exception of type pybind11::error_already_set: IndentationError: ('unexpected indent', ('<string>', 2, 1, '        class pybind11_static_property(property):\n', 2))

This actually seems correct, the code has a leading indent. Not sure why PyPy < 3.9 likes it, actually.

I have this bug in a project currently which i try porting to pypy. Is there any fix for this currently? Using python 3.8 is not an option unfortunatly for me. This is the only place where i found this error, so i thought i would just ask here

[mattip]

@DarkWingMcQuack the indent is fixed on HEAD. You can install pybind11 from git until the fix is released: pip install git+https://github.com/pybind/pybind11.git.

[henryiii]

It should also be available in the v2.9 branch. Hopefully we can cut a new patch release soon.

[henryiii]

The library name change in 3.9 is also likely breaking things, though, possibly including all older versions of CMake's FindPython support. I want to look into that before making a release. Might not be fixable from our end, though, for CMake.

[mattip]

The library name change in 3.9 is also likely breaking things

bummer.

[nachovizzo]

I'm also seeing this error on a CI build, for Python 3.9

[root@bbf72c22780d ~]# "${PYBIN}/pytest" --capture=sys /io/
================================================ test session starts ================================================
platform linux -- Python 3.9.10[pypy-7.3.8-final], pytest-7.1.1, pluggy-1.0.0
rootdir: /io
collecting ... terminate called after throwing an instance of 'pybind11::error_already_set'
  what():  IndentationError: ('unexpected indent', ('<string>', 2, 1, '        class pybind11_static_property(property):\n', 2))
Fatal Python error: Aborted

[mattip]

@nachovizzo The indent is fixed on HEAD. You can install pybind11 from git until the fix is released: pip install git+https://github.com/pybind/pybind11.git.

[nachovizzo]

@nachovizzo The indent is fixed on HEAD. You can install pybind11 from git until the fix is released: pip install git+https://github.com/pybind/pybind11.git.

Great, already try this and fix the problem. Looking forward to a new release ;)