pybind11 icon indicating copy to clipboard operation
pybind11 copied to clipboard

[BUG]: Determine if the interpreter is running

Open cliffburdick opened this issue 3 years ago • 1 comments

Required prerequisites

  • [X] Make sure you've read the documentation. Your issue may be addressed there.
  • [X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
  • [X] Consider asking first in the Gitter chat room or in a Discussion.

Problem description

Hi, I have an application that uses gtest+pybind11 to test some API calls in a library, but that library also needs to use pybind11 in some calls outside of the test framework. I used to have the scoped_interpreter created in the gtest main right before the tests start, and it would go out of scope when they finished. The problem is when using the library when not in test mode I need certain functions to start the interpreter. If I have them do it right now, I get an error that an interpreter is already running from the test suite when I run it. I also tried to make the scoped_interpreter a static class member for my pybind11 wrapper, and that almost works, except it segfaults when the application exits:

#0  0x00007fffc2dbd675 in ?? () from /usr/lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff7ccf813 in PyBytes_FromString () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#2  0x00007fff0b2075f4 in __pyx_pw_13cupy_backends_4cuda_3api_6driver_15CUDADriverError_1__init__(_object*, _object*, _object*) () from /usr/local/bin/miniconda/lib/python3.8/site-packages/cupy_backends/cuda/api/driver.cpython-38-x86_64-linux-gnu.so
#3  0x00007ffff7cd44df in _PyObject_MakeTpCall () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#4  0x00007ffff7cd5a2c in method_vectorcall () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#5  0x00007ffff7cd648f in PyVectorcall_Call () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#6  0x00007ffff7cfc9ac in slot_tp_init.lto_priv () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#7  0x00007ffff7cf7917 in type_call.lto_priv () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#8  0x00007fff0b200f9f in __Pyx__PyObject_CallOneArg(_object*, _object*) () from /usr/local/bin/miniconda/lib/python3.8/site-packages/cupy_backends/cuda/api/driver.cpython-38-x86_64-linux-gnu.so
#9  0x00007fff0b2014d6 in __pyx_f_13cupy_backends_4cuda_3api_6driver_check_status(int, int) [clone .isra.21] [clone .part.22] () from /usr/local/bin/miniconda/lib/python3.8/site-packages/cupy_backends/cuda/api/driver.cpython-38-x86_64-linux-gnu.so
#10 0x00007fff0b20348f in __pyx_f_13cupy_backends_4cuda_3api_6driver_moduleUnload(long, int) () from /usr/local/bin/miniconda/lib/python3.8/site-packages/cupy_backends/cuda/api/driver.cpython-38-x86_64-linux-gnu.so
#11 0x00007fff0b38bc90 in __pyx_tp_dealloc_4cupy_4cuda_8function_Module(_object*) () from /usr/local/bin/miniconda/lib/python3.8/site-packages/cupy/cuda/function.cpython-38-x86_64-linux-gnu.so
#12 0x00007fff0b38a82e in __pyx_tp_dealloc_4cupy_4cuda_8function_Function(_object*) () from /usr/local/bin/miniconda/lib/python3.8/site-packages/cupy/cuda/function.cpython-38-x86_64-linux-gnu.so
#13 0x00007ffff7de77b6 in free_keys_object () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#14 0x00007ffff7de7938 in dict_clear.lto_priv () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#15 0x00007fff19a9bc96 in __pyx_pw_4cupy_5_util_3clear_memo(_object*, _object*) () from /usr/local/bin/miniconda/lib/python3.8/site-packages/cupy/_util.cpython-38-x86_64-linux-gnu.so
#16 0x00007ffff7e56d04 in cfunction_vectorcall_NOARGS () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#17 0x00007ffff7cd648f in PyVectorcall_Call () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#18 0x00007ffff7e3ce84 in atexit_callfuncs () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#19 0x00007ffff7e04fb3 in Py_FinalizeEx.part.28 () from /usr/local/bin/miniconda/lib/libpython3.8.so.1.0
#20 0x0000555555de0da6 in pybind11::finalize_interpreter () at /repro/MatX/build/_deps/pybind11-src/include/pybind11/embed.h:160
#21 0x0000555555de0eb2 in pybind11::scoped_interpreter::~scoped_interpreter (this=0x55555d8d2311 <matx::detail::MatXPybind::Init()::gil>, __in_chrg=<optimized out>) at /repro/MatX/build/_deps/pybind11-src/include/pybind11/embed.h:194
#22 0x00007fffc2c7ba27 in ?? () from /usr/lib/x86_64-linux-gnu/libc.so.6
#23 0x00007fffc2c7bbe0 in exit () from /usr/lib/x86_64-linux-gnu/libc.so.6
#24 0x00007fffc2c590ba in __libc_start_main () from /usr/lib/x86_64-linux-gnu/libc.so.6

Is there a way to determine if the interpreter is already running so I don't start it again? I also tried catching the exception thrown, but that still prints the error that the interpreter is already running as well. Thanks!

Reproducible example code

No response

cliffburdick avatar Feb 06 '22 04:02 cliffburdick

check out the definition of the function initalize_interpreter you can see how pybind11 detects if interpreter's running, by calling this:

if ( Py_IsInitialized() == 0 ) {
  //interpreter is not running, do you work
}

chunlai67 avatar Sep 01 '22 13:09 chunlai67