implicit conversions

[hiaselhans]

Thanks for the great work you do on PyO3.

Pybind11 has a feature called 'implicit conversions' https://pybind11.readthedocs.io/en/stable/advanced/classes.html#implicit-conversions

This can be quite handy, let me give an example:

#[pyclass]
#[derive(Clone, Copy)]
pub struct Vector3D {
    pub x: f64
    pub y: f64
    pub z: f64
}

#[pymethods]
impl Vector3D {
    #new
    pub fn __new__(v: [f64, 3]) -> Self {
        Self {x: v[0], y: v[1], z: v[2]}
    }
}


#[pyfunction]
fn do_something(vec: Vector3D) -> f64 {
    vec.x * 2
}

>>> from py_rust_module import *

>>> do_something([1,2,3])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: argument 'vec': 'list' object cannot be converted to 'Vector3D'

>>> do_something(Vector3D([1,2,3]))
2

In Pybind11 Vector3D could be declared as implicitly convertible from list and do_something([1,2,3]) would be valid and implicitly casting the list to a Vector3D object.

Is this something that could be done in pyo3? It is quite handy and can save a lot of verbosity...

[davidhewitt]

Hi, thanks for a great question. I'm aware of the pybind11 implicit conversions and agree they can be very powerful. It's a good topic of discussion whether they should be implemented in PyO3.

One thing to note is that Rust as a language chose not to have any implicit conversions. So if you view packages created by PyO3 as exposing Rust APIs then it may not be relevant to consider implicit conversions. Similarly pybind11 supports C++ function overloading, but I don't think overloading would be a good fit for PyO3 at all.

In my opinion Python as a language is also moving away from duck-typed functions to static typing. So the fit for implicit conversions on the Python side is perhaps less than it was when pybind11 was designed.

When I last considered it myself I didn't think we need this right now and was undecided on whether I'd want it in the long term. If a majority of users would like to have this feature, we can of course consider it.

[davidhewitt]

It's perhaps worth pointing out that without implicit conversions built into the PyO3 framework, you can always write functions which take &PyAny (or enums with #[derive(FromPyObject)]) and implement this sort of behavior yourself. To users of your package there would be no difference. As an implementor it would be opt-in at each function rather than global, which has both advantages and drawbacks.

[hiaselhans]

One thing to note is that Rust as a language chose not to have any implicit conversions. So if you view packages created by PyO3 as exposing Rust APIs then it may not be relevant to consider implicit conversions. Similarly pybind11 supports C++ function overloading, but I don't think overloading would be a good fit for PyO3 at all.

i agree on function overloading not being a good fit.

In my opinion Python as a language is also moving away from duck-typed functions to static typing. So the fit for implicit conversions on the Python side is perhaps less than it was when pybind11 was designed.

on the other side, libraries like pydantic are popular as never before and they are indeed casting types implicitly when possible

When I last considered it myself I didn't think we need this right now and was undecided on whether I'd want it in the long term. If a majority of users would like to have this feature, we can of course consider it.

Maybe we could have a macro to annotate implicit conversion? Changing the function signature to take pyobjectany would always require calling functions with a python object, even from within rust?

[davidhewitt]

Sorry for the delay.

Maybe we could have a macro to annotate implicit conversion? Changing the function signature to take pyobjectany would always require calling functions with a python object, even from within rust?

So there are two ways a crate other than PyO3 could already support this:

• Create a wrapper Implicit<T> which implements a custom FromPyObject implementation which does whatever implicit conversions you want to define, and use that as function arguments.
• Create a function implicit_extract to convert &PyAny -> T with any logic you want, and use #[pyo3(from_py_with = "implicit_extract")] annotation on the argument you want to enable this for.

So at the moment I don't see a need to invest in this for the PyO3 core. I propose we revisit in the future if a crate is published with this functionality and proves to be extremely popular.

[hameer-spire]

I can see one other solution: Make a pyfunction generic over types, but then, via macros or similar, add implementations to it which behave as overloads. Does that sound good?