Build issue with newer torch pybind11 cast.h - workaround inside

[ilikenwf]

This may be out of the scope of current development, but the workaround of patching cast.h as described in this issue allows builds for me

https://github.com/pybind/pybind11/issues/4606

I realize we can't really go modifying that file ourselves however manually it is acceptable - maybe worthy of being in a readme somewhere.