Remove pybind11 submodule

[dokempf]

In the transition to numpy 2, we had multiple issues (#360 + one other) that occurred because we were pinned to an old version of pybind11. It may be time to get rid of the submodule and use it as a Python dependency instead.