pyo3 icon indicating copy to clipboard operation
pyo3 copied to clipboard

Add `IntoPyObject` & `FromPyObject` for `Arc<T>`

Open bschoenmaeckers opened this issue 9 months ago • 4 comments

Closes #4887.

This PR adds IntoPyObject and FromPyObject for Arc<T> by forwarding the conversion to &T. Since we can’t access T, we work around the naming of associated types by adding extra type parameters. This fixes the conversion error when using #[pyclass(get)] with Arc<T> fields.

bschoenmaeckers avatar Mar 17 '25 13:03 bschoenmaeckers

I wonder whether the additional type parameters have any impact on type inference. Did you perhaps tested/noticed anything in that regard? Maybe with the derive macro?

I didn't experience any weird inference behaviour when writing the existing test. The new macro tests behaves fine so I think we are good there as well.

Is it sufficiently clear that a roundtrip with Arc will not give the same Arc? Or should we document that somewhere?

I am not sure if it needs explicit documentation, as all roundtrip conversions create new objects, why would this one differ? That said documentation doesn't hurt nobody.

bschoenmaeckers avatar Mar 17 '25 20:03 bschoenmaeckers

One alternative possibility here is that we change the pyclass internals so that instead of only being able to store T they can also store a range of different containers which deref to T.

I imagine mostly that would be Arc and Box but maybe also more complicated designs where the T is actually stored as data on a separate python object and the data is just borrowed from there.

davidhewitt avatar Mar 17 '25 21:03 davidhewitt

I imagine mostly that would be Arc and Box but maybe also more complicated designs where the T is actually stored as data on a separate python object and the data is just borrowed from there.

I do not fully understand your second case. Can you give me a example of a class definition?

bschoenmaeckers avatar Mar 18 '25 11:03 bschoenmaeckers

I do not fully understand your second case. Can you give me a example of a class definition?

I imagine something like

#[pyclass]  
struct MemoryView{
    owner: Py<SomethingThatHasVecU8>,
    ptr: *mut u8,
}

Anyway, I share @Icxolu thoughts around roundtripping and ambiguity, and am inclined to not have this. If a user really wants a getter to an Arc field they can write a getter method and will be forced to consider the semantics they want.

mejrs avatar Apr 13 '25 21:04 mejrs

To follow up here - I was thinking that to allow Arc<T> conversions we might want to pair it with a #[pyclass(storage)] option which is able to store arcs in a way which roundtrips, e.g. #[pyclass(storage = Arc<Self>)].

davidhewitt avatar Aug 01 '25 14:08 davidhewitt

#[pyclass(storage)] being my not fully fleshed out idea from https://github.com/PyO3/pyo3/pull/5233#issuecomment-3089194417

davidhewitt avatar Aug 01 '25 15:08 davidhewitt