[BUG]: Crashed with importing ruptures package.

[leonleeann]

Required prerequisites

• [X] Make sure you've read the documentation. Your issue may be addressed there.
• [X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• [ ] Consider asking first in the Gitter chat room or in a Discussion.

Problem description

I'm using pybind11 to build a c++ main program that importing a python package ruptures at runtime.

We can simplly install it by pip3 install ruptures. I tried ruptures-1.1.6 and 1.1.7, results are same.

Once I call py::finalize_interpreter(); in c++, the program crashs. Error output:

free(): invalid pointer

Reproducible example code

#include <pybind11/embed.h>
#include <pybind11/pybind11.h>

namespace py = pybind11;

int main() {
// 	py::scoped_interpreter guard {};
	py::initialize_interpreter();

	auto mod_py = py::module::import( "ruptures" );

	py::finalize_interpreter();
	return EXIT_SUCCESS;
};

[henryiii]

I'm guessing this is https://github.com/pybind/pybind11/issues/4105.

[henryiii]

This code is broken in both 2.9.2 and 2.10.0. If I remove the initialize_interpreter/finalize_interpreter and instead use the scooted_interpreture guard, it works fine in both 2.9.2 and 2.10.0.

[rwgk]

You can fix the reproducer in the PR description like this:

#include <pybind11/embed.h>
#include <pybind11/pybind11.h>

namespace py = pybind11;

int main() {
//      py::scoped_interpreter guard {};
        py::initialize_interpreter();

        {
            auto mod_py = py::module::import( "ruptures" );
        }

        py::finalize_interpreter();
        return EXIT_SUCCESS;
}

With the original reproducer, mod_py is destructed only after the interpreter is finalized, causing a SEGFAULT in Python/pystate.c _Py_GetConfig() in my environment (I used gdb to see that). (The free(): invalid pointer you saw is probably just another symptom of undefined behavior.)