Voultapher
Safe API for mapping between types whose owner is Arc<T>
A frequent use case I have is multiple self_cell types which all have an owner of Arc<T>, and I'd like to be able to "map" between them. Here's an example of this:
use std::sync::Arc;
struct X(());
struct T1<'a>(&'a ());
#[derive(Debug)]
struct T2<'a>(&'a ());
self_cell::self_cell! {
struct O1 {
owner: Arc<X>,
#[covariant]
dependent: T1,
}
}
self_cell::self_cell! {
struct O2 {
owner: Arc<X>,
#[covariant]
dependent: T2,
}
}
unsafe fn relifetime<'a, 'b>(t: &'a T1<'a>) -> &'b T1<'b> {
std::mem::transmute(t)
}
fn map(o: &O1, f: impl for<'this> FnOnce(&'this X, &T1<'this>) -> T2<'this>) -> O2 {
O2::new(Arc::clone(&o.borrow_owner()), |x| {
f(&x, unsafe { relifetime(o.borrow_dependent()) })
})
}
fn main() {
let o1 = O1::new(Arc::new(X(())), |x| T1(&x.0));
let o2 = map(&o1, |_, t1| T2(t1.0));
drop(o1);
o2.with_dependent(|_, v| println!("{:?}", v));
}
Given an O1, I'd like to be able to safely create an O2 which points at the same data.
The implementation of map here works, but it'd be great if there was some way this functionality could be generalized as a part of self-cell, so consumers didn't need to carry unsafe code.
I see the use-case, however there are at least two major blockers that I don't have solutions for of the top of my head:
- The API allows borrowing the container, not only the inner part, i.e. take a reference to the
Arcitself and not the inner thing, we would need some kind of generic mechanism to disallow that. - We would need to somehow identify types with stable pointers, there is stable_deref_trait but it is
unsafeto implement. Back when I designedself_cellI considered using that trait, but I want an API that is completely safe-to-use, no exceptions. So that trait a non-starter :/
Maybe there is some other way to model the problem. Could you please provide some real examples of the kind of mapping that is done from one dependent type to the next. Also do you need a completely new value, or is it enough to transform the existing one maybe something conceptually like fn map(self, new_dependent_ctor) -> SelfCell<Owner, NewDependentType>.
Wait does this work for you?
use std::sync::Arc;
struct X(());
struct T1<'a>(&'a ());
#[derive(Debug)]
struct T2<'a>(&'a ());
self_cell::self_cell! {
struct O1 {
owner: Arc<X>,
#[covariant]
dependent: T1,
}
}
self_cell::self_cell! {
struct O2 {
owner: O1,
#[covariant]
dependent: T2,
}
}
fn map(o: O1, f: impl for<'this> std::ops::FnOnce(&'this X, &T1<'this>) -> T2<'this>) -> O2 {
O2::new(o, |x| f(&x.borrow_owner(), x.borrow_dependent()))
}
fn main() {
let o1 = O1::new(Arc::new(X(())), |x| T1(&x.0));
let o2 = map(o1, |_, t1| T2(t1.0));
o2.with_dependent(|_, v| println!("{:?}", v));
}
Sure, here's three real examples (search in the files for their callers) that follow this pattern:
- https://github.com/pyca/cryptography/blob/5f19fad7be68f75a4522ec88624114306f35294d/src/rust/src/x509/ocsp_resp.rs#L444-L467
- https://github.com/pyca/cryptography/blob/5f19fad7be68f75a4522ec88624114306f35294d/src/rust/src/x509/ocsp_resp.rs#L468-L489
- https://github.com/pyca/cryptography/blob/5f19fad7be68f75a4522ec88624114306f35294d/src/rust/src/x509/crl.rs#L466-L487
What about the nesting I showed, would that solve your use-cases? You could even drop the Arc and just directly use the interior type.
Hmm, so the challenge is that it doesn't work for O2 to have an owned O1, as it means cloning more than just the Arc.
Sorry, I meant "it doesn't address my use case", not that it doesn't function.
Can you please create a minimal example that shows the behavior you need but can't be done by nesting.
This issue is not that nesting doesn't work, it's that there'd be a performance cost.
On Sat, Apr 6, 2024 at 1:35 PM Lukas Bergdoll @.***> wrote:
Can you please create a minimal example that shows the behavior you need but can't be done by nesting.
— Reply to this email directly, view it on GitHub https://github.com/Voultapher/self_cell/issues/55#issuecomment-2041146585, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAGBACURHQLOVAYXCU5SDY4AW6FAVCNFSM6AAAAABF2BRP7WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGE2DMNJYGU . You are receiving this because you authored the thread.Message ID: @.***>
-- All that is necessary for evil to succeed is for good people to do nothing.
Sure, take the following use case:
type V<'a> = Vec<&'a str>;
type S<'a> = &'a str;
self_cell::self_cell! {
struct O1 {
owner: Arc<str>,
#[covariant]
dependent: V,
}
}
self_cell::self_cell! {
struct O2 {
owner: Arc<str>,
#[covariant]
dependent: S,
}
}
fn main() {
// ...
map(o1, |owner, dependent| O2::new(owner, dependent[0]));
// ...
}
If the owner in O2 were O1, instead of Arc<str>, then creating one would requiring cloning the Vec<&str>, instead of simply being a reference count increment.
The suggested map function:
fn map(o: O1, f: impl for<'this> std::ops::FnOnce(&'this X, &T1<'this>) -> T2<'this>) -> O2 {
O2::new(o, |x| f(&x.borrow_owner(), x.borrow_dependent()))
}
moves o1, so there is no copy at all, not even a ref-increment. Or do you need o1 to be around afterwards? I assumed no because you drop it immediately afterwards. Also by nesting it's not lost, you can still access it via the owner.
Yes, unfortunately I need to keep the original alive, so for my use case it doesn't work for map to take O1 by value.
The value of O1 would still be around though, it's the owner of O2. Calling .borrow_owner() would yield a &O1. How is O1 used afterwards?
In my use cases, O1 is somewhere else on the heap, with other references to it, so it's not possible to move it.
Ok, back to square one. Unless you have an idea how to circumvent the two major roadblocks I explained in the initial response, I'm out of ideas short of re-architect your code so you can move O1.
Hmm, I'm not sure I understand the first problem, can you explain a bit more?
For the second (unsafe trait required), yes I think this is a challenge. Either you have to hard-code self_cell to only support a few types, or I think any trait has to be unsafe.
Take this example:
type Dependent<'a> = &'a Arc<String>;
self_cell!(
struct SelfCell {
owner: Arc<String>,
#[covarint]
dependent: Dependent,
}
);
The reference to the inner part of Arc stay stable, however the dependent type references the Arc directly, and that doesn't stay stable. I'm not aware of a mechanism to generically expose that relationship. Does that help?
Ah yes, I think you need to only pass the map() callback the result of
deref(), you can't give it a ref to the owner itself.
On Sun, Apr 14, 2024 at 1:12 PM Lukas Bergdoll @.***> wrote:
Take this example:
type Dependent<'a> = &'a Arc<String>; self_cell!( struct SelfCell { owner: Arc<String>,
#[covarint] dependent: Dependent, });The reference to the inner part of Arc stay stable, however the dependent type references the Arc directly, and that doesn't stay stable. I'm not aware of a mechanism to generically expose that relationship. Does that help?
— Reply to this email directly, view it on GitHub https://github.com/Voultapher/self_cell/issues/55#issuecomment-2054124634, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAGBCIBTHXY4GH667JT4LY5K2ILAVCNFSM6AAAAABF2BRP7WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJUGEZDINRTGQ . You are receiving this because you authored the thread.Message ID: @.***>
-- All that is necessary for evil to succeed is for good people to do nothing.
Given the constraints that I want to keep, namely not giving certain types special treatment, and zero unsafe-to-use parts of the API, I'm not seeing a way forward with this issue.
I'm proposing to close it.