pybind11 cheatsheet documentation

[ezyang]

I have found the pybind11 documentation to be not so great for quickly trying to figure out "how the heck do I get from type A to type B". So I wrote my own notes. I'm interesting in getting these back into the official docs, but I am not sure what is the most appropriate form.

Here goes.

---

The important types:

• py::handle - equivalent to PyObject* (no automatic refcounting)
• py::object - does automatic refcounting. Subclass of py::handle.

Conversions

PyObject* ↔ py::handle

This is a trivial conversion. These two types are basically totally equivalent.

PyObject *x;

py::handle h = x; // implicit conversion OK
x = h.ptr(); // no implicit conversion to raw pointer

py::handle (or subclass) → py::object (or subclass)

The syntax here works for both py::object, and any of its subclasses (e.g., py::str, etc) http://pybind11.readthedocs.io/en/stable/reference.html#_CPPv26object

All of these are unchecked. Do an isinstance check beforehand if necessary. (Note that if you try to do anything with the py::object subclass, pybind11 will probably throw an exception at that point, so you probably aren't completely hosed. But casts to py::foo things are not checked at the point you do the cast!)

Common gotcha: you MUST pass a template argument to py::cast if you're casting from a PyObject*, it will silently do the wrong thing if you don't.

py::handle h;

// Copying
auto o = py::reinterpret_borrow<py::object>(h); // h can be PyObject*
  // or
auto o = x.cast<py::object>();
  // or
auto o = py::cast<py::object>(h); // h can be PyObject*
// equivalent to:
//    Py_INCREF(x);
//    THPObjectPtr o(x);

// Moving
auto o = py::reinterpret_steal<py::object>(h); // h can be PyObject*
// equivalent to:
//    THPObjectPtr o(x);

py::object (or superclass) → py::handle

Stealing is pretty useful if you're trying to invoke a C API function that steals its argument.

py::object o;

// Borrows from o
py::handle h2 = o;
// refcount NOT increased; h2 is only live as long as o is

// Steals from o
py::handle h = o.release();
// o no longer valid

NB: a “copy” from object into handle doesn't make sense, because handle doesn't know how to manage its own memory.

NB2: these conversions are implemented by the implicit copy-constructor on py::handle,static casting the input py::object as a const py::handle& (in case you're like me and tried to find where in the code this conversion was implemented).

py::object → C++

• Checked conversion!
• Reference: http://pybind11.readthedocs.io/en/stable/reference.html#_CPPv2NK6handle4castEv
• Declared at: https://github.com/pybind/pybind11/blob/master/include/pybind11/pytypes.h#L171
• Source: https://github.com/pybind/pybind11/blob/master/include/pybind11/cast.h#L1613
• You can also use py::cast<CppType> (explicit template argument needed): https://github.com/pybind/pybind11/blob/master/include/pybind11/cast.h#L1590
• Note that these are willing to do implicit conversions if necessary to get to the correct type; e.g., if you cast a py::float_ into an int, it will get truncated; if you cast an py::int_ into a std::string, it will get rendered as a string.

py::handle h; // or py::object
CppType x = h.cast<CppType>(); // may throw cast_error
  // or
CppType x = py::cast<CppType>(h);
// x is a new copy unrelated to h

C++ → py::object

• Undocumented in Reference
• Source: https://github.com/pybind/pybind11/blob/master/include/pybind11/cast.h#L1602
• Return value policies apply: http://pybind11.readthedocs.io/en/stable/advanced/functions.html#return-value-policies
• Do NOT use a template argument, that is for a Python to C++ type conversion

CppType x;

py::object o = py::cast(x);
  // or
py::object o = py::cast(x, return_value_policy::automatic);
// o is a new copy, but with a different policy it could be different

How do I...

// b = isinstance(x, ty)
py::handle x;
py::handle ty;
bool b = py::isinstance(x, ty);

// b = isinstance(x, str)
py::handle x;
bool b = py::isinstance<py::str>(x);

// b = isinstance(x, tuple)
py::handle x;
bool b = py::isinstance<py::tuple>(x);

// s = str(x)
py::handle x;
py::str s(x); // use the constructor

// b = hasattr(x, "something")
py::handle x;
bool b = py::hasattr(x, "something");

[lemairecarl]

Thanks a lot! I really struggled with "C++ → py::object", which as mentioned here, is undocumented. It should at least be in the Reference section of the docs, if not in the type conversion section.

[ebetica]

It's actually documented here, albeit a bit briefly.

[eacousineau]

The more intricate bits of ref counting, casting, etc., came up on Gitter here. If I have the time, I may try to incorporate this and a few other things in a PR for the docs (adding it to the currently terse docs).

[EricCousineau-TRI]

Slowly circling back, but relates #2362 -- documenting pytypes a bit more, perhaps with examples

\cc @YannickJadoul @henryiii @rwgk

[YannickJadoul]

Agreed that the docs on these concepts should be expanded! Lots of these things can probably be added to https://pybind11.readthedocs.io/en/stable/advanced/pycpp/object.html? I would argue to try and fit them into the existing way the docs are structured and laid out, though:

• C++ <-> Python can be in the "Casting back and forth" section (this is currently missing e.g. py::cast<CustomClass>(object)). The examples and explanation from this issue should also and we should also clarify the difference between py::cast<CustomClass> and py::cast<CustomClass&> and py::cast<CustomClass*>.
• Conversion between different Python wrappers (handle, object, and more specific classes) could then be added to a "Converting between Python types", or so? It might be interesting to still split this up into handle vs. object on the one hand, and the more specific object subtypes on the other? What's missing from the docs and the cheat sheet above is also the difference between converting an object to a type and just reinterpreting one.