[BUG]: Function with Param of type `const Eigen::VectorXd&` causes "windows fatal exception: access violation" on Windows

[huweiATgithub]

Required prerequisites

• [X] Make sure you've read the documentation. Your issue may be addressed there.
• [X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• [X] Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

v2.10.4

Problem description

When calling function with parameter const Eigen::VectorXd&, it fails with "windows fatal exception: access violation" on Windows, compiled with VS2022, Python 3.12, and Pybind v2.10.4.

However, the example works well with Python 3.10 + VS2019 + Pybind v2.10.4. Besides, it also works well in Python 3.12 + VS2022 + Pybind current stable branch.

I know from the Documentation that it recommend using Eigen::Ref for reference. Using Eigen::Ref, Python 3.12 + VS2022 + Pybind v2.10.4 also works.

That said, using either Pybind current stable branch or Eigen::Ref can make it work.

I have been struggling on this issue for a while.

I want to understand how things go wrong:

• Why Eigen::Ref works, but const Eigen::VectorXd& not?
• Why Pybind current stable branch works?

Reproducible example code

#include <pybind11/operators.h>
#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include <pybind11/eigen.h>
#include <pybind11/stl.h>
#include <vector>
#include <Eigen/Eigen>

namespace py = pybind11;


class ArrayDynamic {
 using Vector =Eigen::VectorXd;

 private:
  Vector data_;

 public:
  EIGEN_MAKE_ALIGNED_OPERATOR_NEW

  ArrayDynamic() = default;
  virtual ~ArrayDynamic() = default;

  explicit ArrayDynamic(size_t dim) { data_.resize(dim); }
  auto data() const -> const Vector& { return data_; }
  auto data() -> Vector& { return data_; }
};


PYBIND11_MODULE(promote, m)
{
    auto math = m.def_submodule("math");
    py::class_<ArrayDynamic>(math, "ArrayDynamic")
        .def(py::init<size_t>(), "Create an array of given size.", py::arg("dim"))
        .def("data", py::overload_cast<>(&ArrayDynamic::data), py::return_value_policy::reference_internal, "internal")
        .def( "set_data",
            [](ArrayDynamic& array,
               const Eigen::VectorXd& data)
            {
                array.data() = data;
            },
            "Set data.");
}

For a full example, see https://github.com/huweiATgithub/pybind_fail_example

### Is this a regression? Put the last known working version here if it is.

Not a regression

[sethraymond]

I recently ran into this issue on linux where my python code was segfaulting due to accessing a nullptr. I determined that, in my case, there was an incompatibility with an older version of pybind (v2.11.1 was what I was using) and numpy>=2.0.0, which was installed by default on my newer Python version. I discovered this using Python 3.12, but the reproduction steps are easier in Python 3.11.

Not that this is something to fix in the older version of pybind, but hopefully someone in the future who's wondering why their pybind11 is seg faulting when attempting to access an Eigen object will stumble upon this and see what they need to upgrade/downgrade to get it working again.

Minimal repro steps:

pybind_numpy_incompatibility_debugging.tar.gz

(Contents of the tarball): module.cc

#include <pybind11/pybind11.h>
#include <pybind11/eigen.h>
#include <pybind11/stl.h>

namespace py = pybind11;
using namespace pybind11::literals;

bool myfunc(Eigen::Vector3f v) {
        py::print("foo", "flush"_a = true);
        return true;
}

PYBIND11_MODULE(my_module, m) {
        m.def("myfunc", &myfunc, py::arg("v"));
}

CMakeLists.txt

project(my_module)
find_package(Eigen3 REQUIRED)
find_package(pybind11 REQUIRED)
pybind11_add_module(my_module module.cc)
include_directories(${EIGEN3_INCLUDE_DIRS})

test.py

import my_module
my_module.myfunc([0.0, 0.0, 0.0])

---

old-pybind-old-numpy.Dockerfile:

FROM python:3.11

RUN apt update -y && apt install -y cmake libeigen3-dev
RUN git clone --depth 1 --branch v2.11.1 https://github.com/pybind/pybind11.git \
    && cd pybind11 \
    && cmake . \
    && make install -j

RUN pip install -U setuptools
RUN pip install numpy==1.24.4

WORKDIR /test
COPY module.cc /test
COPY CMakeLists.txt /test
COPY test.py /test
RUN cmake . && make

CMD [ "python", "test.py" ]

old-pybind-new-numpy.Dockerfile

FROM python:3.11

RUN apt update -y && apt install -y cmake libeigen3-dev
RUN git clone --depth 1 --branch v2.11.1 https://github.com/pybind/pybind11.git \
    && cd pybind11 \
    && cmake . \
    && make install -j

RUN pip install numpy==2.0.0

WORKDIR /test
COPY module.cc /test
COPY CMakeLists.txt /test
COPY test.py /test
RUN cmake . && make

CMD [ "python", "test.py" ]

new-pybind-old-numpy.Dockerfile

FROM python:3.11

RUN apt update -y && apt install -y cmake libeigen3-dev
RUN git clone --depth 1 --branch v2.11.1 https://github.com/pybind/pybind11.git \
    && cd pybind11 \
    && cmake . \
    && make install -j

RUN pip install -U setuptools
RUN pip install numpy==1.24.4

WORKDIR /test
COPY module.cc /test
COPY CMakeLists.txt /test
COPY test.py /test
RUN cmake . && make

CMD [ "python", "test.py" ]

new-pybind-new-numpy.Dockerfile

FROM python:3.11

RUN apt update -y && apt install -y cmake libeigen3-dev
RUN git clone --depth 1 --branch v2.13.6 https://github.com/pybind/pybind11.git \
    && cd pybind11 \
    && cmake . \
    && make install -j

RUN pip install numpy==2.0.0

WORKDIR /test
COPY module.cc /test
COPY CMakeLists.txt /test
COPY test.py /test
RUN cmake . && make

CMD [ "python", "test.py" ]

---

test.sh

#!/bin/bash
permutations=("old-pybind-old-numpy" "old-pybind-new-numpy" "new-pybind-old-numpy" "new-pybind-new-numpy")

for permutation in "${permutations[@]}"; do
    docker build . -f "${permutation}.Dockerfile" -t "${permutation}" > /dev/null 2>&1
    docker run --rm -it "$permutation"
    return_value=$?
    echo "${permutation}: ${return_value}"
done

Output:

foo
old-pybind-old-numpy: 0
old-pybind-new-numpy: 139
foo
new-pybind-old-numpy: 0
foo
new-pybind-new-numpy: 0