rfcs
rfcs copied to clipboard
rust-lang
Partial borrowing (for fun and profit)
Consider the following struct and impl for 2D coordinates:
struct Point {
x: f64,
y: f64
}
impl Point {
pub fn x_mut(&mut self) -> &mut f64 {
&mut self.x
}
pub fn y_mut(&mut self) -> &mut f64 {
&mut self.y
}
}
fn main() {
let mut point = Point { x: 1.0, y: 2.0 };
// ...
}
Even in this simple example, using point.x and point.y gives us more flexibility than using point.mut_x() and point.mut_y(). Compare following examples that double the components of a point:
{
// Legal:
point.x *= 2.0;
point.y *= 2.0;
}
{
// Legal:
let x_mut = &mut point.x;
let y_mut = &mut point.y;
*x_mut *= 2.0;
*y_mut *= 2.0;
}
{
// Legal:
let ref mut x_ref = point.x;
let ref mut y_ref = point.y;
*x_ref *= 2.0;
*y_ref *= 2.0;
}
{
// Legal:
*point.x_mut() *= 2.0;
*point.y_mut() *= 2.0;
}
{
// Illegal:
let x_mut = point.x_mut();
let y_mut = point.y_mut();
// ^~~~~
// error: cannot borrow `point` as mutable more than once at a time
*x_mut *= 2.0;
*y_mut *= 2.0;
}
The lifetime elision rules make it pretty clear why this happens: x_mut() returns a mutable borrow that has to live at least as long as the mutable borrow of self (written as fn x_mut<'a>(&'a self) -> &'a f64).
In order to 'fix' the above example, there needs to be a way to say that a borrow only has to live as long as the mutable borrow of self.x. Here's a modification to the above impl block for a possible syntax to express partial borrows using a where clause:
impl Point {
pub fn x_mut<'a>(&mut self)
-> &'a f64
where 'a: &mut self.x
{
&mut self.x
}
pub fn y_mut<'a>(&mut self)
-> &'a f64
where 'a: &mut self.y
{
&mut self.y
}
}
While the above examples aren't particularly encouraging, allowing for this type of change could be very powerful for abstracting away implementation details. Here's a slightly better use case that builds off of the same idea of representing a Point:
struct Vec2(f64, f64) // Represents a vector with 2 coordinates
impl Vec2 {
// General vector operations
// ...
}
// Represents a coordinate on a 2D plane.
// Note that this change to `Point` would require no change to the above
// example that uses `Point::x_mut` and `Point::y_mut` (the details of
// the structure are effectively hidden from the client; construction could
// be moved into `Point::new` as well)
struct Point(Vec2);
impl Point {
pub fn x_mut<'a>(&mut self)
-> &'a mut f64
where 'a: &mut self.0.0
{
&mut self.0.0
}
pub fn y_mut<'a>(&mut self)
-> &'a mut f64
where 'a: &mut self.0.1
{
&mut self.0.1
}
}
For a more involved example of an API that requires mutable borrows, see this gist that describes a possible zero-cost OpenGL wrapper.
There has been some discussion about partial borrows in the past, but it hasn't really evolved into anything yet, and it seems like the idea hasn't been rejected either.
An alternate syntax (adds a new keyword (that could easily just be contextual), but in my opinion makes more sense and is clearer):
impl Point {
pub fn x_mut(&mut self borrowing x) -> &mut f64 {
&mut self.x
}
}
// With a non-self parameter:
pub fn point_get_x(x: &Point borrowing x) -> &f64 {
&x.x
}
Edit: another syntax that doesn’t require any syntactic additions to the language:
impl Point {
pub fn x_mut(&mut Point { ref mut x, .. }: &mut Self) -> &mut f64 {
x
}
}
pub fn point_get_x(&Point { ref x, .. }: &Point) -> &f64 {
x
}
Although in addition to requiring extra borrowck logic, it would require considering any ‘static’ method that has a Self, &Self, &mut Self, or Box<Self> parameter to be a method that can be called with the x.y() notation (a feature that has been proposed in the past as part of UFCS).
@P1start I think either of those syntaxes would work fine as well! But, there would need to be a way to annotate explicit lifetimes for either syntax as well; something like this:
pub fn x_mut<'a>(&mut self borrowing 'a x) -> &'a mut f64 { ... }
or this:
pub fn x_mut<'a>(&mut Point { 'a ref mut x, .. }: &mut Self) -> &mut f64 { ... }
...which would be necessary where lifetime elision would be ambiguous (such as a bare function that takes multiple &mut and returns a &mut).
I want a green roof on the bikeshed:
impl Point {
pub fn x_mut<'a>(self: &'a mut Self::x) -> &'a mut f64 {
self.x
}
}
for borrowing multiple fields you'd use Self::{x, y}.
Also this needs a lint that denies specifying all fields.
I don't like the idea of a type signature requiring knowledge of private fields to understand, and if the field is public then of course there's not much point. The use case of borrowing multiple fields at the same time can be addressed at a slight cost to ergonomics by just having a single method return both, for example fn x_y_mut(&mut self) -> (&mut f64, &mut f64).
Yes, exposing names of private fields doesn't sit well. There's another way. Composite lifetimes allow separate borrows of individual parts of a type. Two mutable borrows with composite lifetimes '(a, _) and '(_, b) of a struct are possible if no field has both 'a and 'b assigned in the struct declaration. Here, _ is a free lifetime variable.
Example:
(I think the syntax Self::x is incompatible with lowercase associated types.)
struct Point<ref '(a, b)> where ref x: 'a, ref y: 'b {
x: f64,
y: f64
}
impl Point {
pub fn x_mut<'a>(&'(a,) mut self) -> &'a mut f64 {
&mut self.x
}
pub fn y_mut<'b>(&'(_, b) mut self) -> &'b mut f64 {
&mut self.y
}
}
// Typical lifetime parameters work.
struct PointRef<'a> {
borrowed: &'a Point
}
// type signature.
fn draw<'a>(point: PointRef<'a>) {}
// equivalent type signature, for some 'c, 'd.
fn draw<'c, 'd>(point: PointRef<'(c, d)>) {}
I really, really like this proposal. I had some cases where I could choose to either rewrite the same piece of code a lot of times (because it's borrowing self) or clone the values before mutation (which is costy, especially if LLVM can't optimize it away).
However, the syntaxes proposed here seems noisy and non-obvious. The syntax should be something that's easy to spot and write.
I like the idea of @P1start's second proposal for a syntax because it seems consistent with the rust syntax. However, I find it hard to include lifetimes in that proposal, as ref does not enable annotation of lifetimes. Introducing lifetimes for ref breaks the consistency.
@Ticki What does your use case look like? Does the fn x_y_mut(&mut self) -> (&mut f64, &mut f64) pattern not work?
Just a vote for wanting this in some form. It'd be nice to be able to get immutable references to the keys in a hashmap while also being able to get mutable references to the values later.
pub fn point_get_x(&Point { ref x, .. }: &Point) -> &f64 {
I've mentioned, e.g. pub fn point_get_x(p: &Point { x }) -> &f64 { &p.x } before, on reddit.
What makes this attractive to me is that it's refinement to include only specific fields, which would interact not only with borrowing, but many other language features, such as partial moves, ..default syntax and variant types.
Here is some motivation fuel from a contributor to the sdl2 crate: https://cobrand.github.io/rust/sdl2/2017/05/07/the-balance-between-soundness-cost-useability.html
I think this interacts well with #1546.
Also, there is a cute color for shed door if you permit multiple self arguments, like :
trait Foo {
position: usize,
slice: &[u8],
fn tweak(&mut self.position, &'a self.slice) -> &'a [u8];
}
It does not play so nicely with non-self arguments, but maybe it could be short hand for the green roof suggested by @oli-obk. It works with some sort of destructing function call syntax too :
fn foo((ref x,ref 'a mut y) : (usize,[u8])) -> &'a [u8] { .. }
I really would love to see this as well. But for partial self borrowing to happen I guess someone™ needs to write up an RFC, summarizing everything in this issue.
Since we probably don't want to expose the names of private fields in a public API, there should be a way to declare abstract "borrow regions" for your type as part of the public API. You can then privately declare which fields belong to which region. Multiple fields can belong to one region.
Here is an (arbitrary) example:
// Note that this is just pseudo-syntax, I'm not proposing any particular syntax.
// This example uses a new keyword called `region`.
struct Vec<T> {
/// Data belonging to the `Vec`.
pub region data;
/// Length of the `Vec`.
pub region length;
#[region(data)] ptr: *mut T,
#[region(length)] len: i32,
}
impl<T> Vec<T> {
// Needs mutable access to `data`, but doesn't mutate `length`.
fn as_mut_slice(®ion(mut data, length) self) -> &mut [T] {
do_something_with(self.ptr, self.len)
}
// Only needs to access the `length` region.
fn len(®ion(length) self) -> usize {
self.len
}
// This one borrows everything, so no need for special region annotation.
// This means it borrows all regions.
fn retain<F>(&mut self, f: F) where F: FnMut(&T) -> bool {
// ...
}
}
This would allow disjoint borrows, while still not exposing private fields in the public API or error messages.
fn main() {
let mut v = vec![1, 2, 3];
// This is fine because the borrow regions for `as_mut_slice` and `len` don't conflict.
for num in v.as_mut_slice() {
println!("{} out of {}", num, v.len());
}
// Not valid:
v.retain(|num| num == v.len());
// Error: Cannot borrow region `length` of `v` as immutable, because it is also borrowed as mutable.
// v.retain(|&num| num == v.len());
// - ^^^^^^ - - mutable borrow of `length` ends here
// | | |
// | | borrow occurs due to use of `v` in closure
// | immutable borrow of `length` occurs here
// mutable borrow of `length` occurs here
}
Oh, and this would also work nicely for traits. The trait could declare the regions it has, and the implementing type then can map that to its own private fields.
The fields in traits will resolve some of these issues. However they can't be used in case the struct/trait involves multi-field invariants, which are maintained by controlling their modification. An example of this would be data/len of Vec<T>
While the region syntax might be a bit of an overkill for this scenario, I personally like it a lot. Here is another example with it that includes traits as well.
I encountered the need in a scenario where the user impling their own type and they were holding immutable borrow for one field while wanting to call a method on self, which would have modified other disjoint fields.
struct Sphere {
// Field belonging to two regions would allow callers to acquire two mutable
// references to the field by using two different mutable regions.
// Since we don't want this, we don't need to support defining fields in multiple
// regions. This allows us to use scopes instead of attributes here.
pub region position { x: f64, y: f64, z: f64 }
pub region size { radius: f64 }
pub region physics { weight : f64 }
}
impl Foo {
// Borrows everything.
fn copy_from(&mut self, other: &Sphere) { ... }
// Mutably borrows position.
fn move(&mut self borrows { &mut position }, x: f64, y: f64, z: f64) { ... }
// Immutable borrow of size.
fn get_size(&self borrows { &size }) -> f64 { ... }
// Bikeshedding: If we are using 'regions', any syntax that relies on field
// destructuring probably won't work.
}
When it comes to traits, these could be implemented using specific regions. There's no reason borrowing for Display would need to borrowck fields that are not used for display.
// Subtracting a point only alters the sphere position.
pub trait SubAssign<Point3D> for Sphere using { mut position } {
// 'self' is valid only on fields under 'position' here.
fn sub_assign(&mut self, rhs: Point3D) {
self.x -= rhs.x;
self.y -= rhs.y;
self.z -= rhs.z;
}
}
Also traits could possibly describe their own regions.
trait Render {
region geometry;
region texture;
...
}
// Render is implemented on Sphere using position and size.
// Borrowing 'physics' region is still valid while the sphere is being rendered.
impl Render for Sphere using { position, size } {
// Rendered geometry inludes both position and size.
region geometry = { position, size }
// Sphere doesn't contain texture data. It'll use static values
// for any method that would need textures.
region texture = {}
...
}
// Acquiring reference to the trait will automatically borrow the used regions.
let r : &Render = &sphere;
// OK:
// increase_weight mut borrows only 'physics' region, which isn't used by 'Render' impl:
increase_weight( &mut sphere );
// FAIL:
// grow mut borrows physics and size. This fails, as &Render is holding borrow to size region.
grow( &mut sphere );
I think regions are not a good idea.
They do not convey anything about the reasons for being laid out in a particular way and they are at best tangent to describing which parts of the struct need to be borrowable separately as opposed to which parts lay close in whatever mental model.
Overall I think the information about which part of a struct to borrow should be specified or inferred at the point at which it is borrowed.
We now have arbitrary self types in traits. I think whatever syntax we use should approximate or mimic it. Personally, I lean towards the following:
impl Point {
pub fn x_mut(self: Point { ref mut x, .. }) -> &mut f64 {
&mut self.x
}
pub fn y_mut(self: Point { ref mut y, ..}) -> &mut f64 {
&mut self.y
}
}
I doubt Point could/should destrtucture in type position like that, but you could destrtucture in argument position like:
pub fn x_swap_y<T>(
Point { ref mut x, .. }: Point<T>,
Point { ref mut y, .. }: Point<T>
) {
mem::swap(x,y);
}
impl Point<f64> {
pub fn x_mut(Point { ref mut x, .. } = &mut self) -> &mut f64 { x }
pub fn y_mut(Point { ref mut y, ..} = &mut self) -> &mut f64 { y }
}
In particular, these methods have no self argument because the calling convention destrtuctures it for them.
One should decide that a series of destructuring borrows works this way too though.
The thing is we already have an appropriate syntax for having a trait with a different receiver than Self, which is used for Pin in stable already: https://github.com/rust-lang/rust/issues/44874
I believe straying too much from that already established syntax would be a bad idea.
We always have binding : Type everywhere else, including arbitrary self types, so anything else departs much more from the existing syntax and parser.
@estebank I agree, we should build upon existing syntax. @burdges But why use an equal sign there?
Wouldn't this be the most natural syntax?
impl Point {
fn x_mut(Point { ref mut x, .. }: &mut Self) -> &mut i32 { x }
fn y_mut(Point { ref mut y, .. }: &mut Self) -> &mut i32 { y }
}
Considering that this already works:
fn f(_: &mut i32, _: &mut i32) {}
let p = &mut Point { x: 1, y: 2 };
f(
{ let Point { ref mut x, .. } = p; x },
{ let Point { ref mut y, .. } = p; y },
);
The advantage of using this syntax would be that it wouldn't only work for f(p.x_mut(), p.y_mut()) but also for f(Point::x_mut(p), Point::y_mut(p)).
And even more generally, shouldn't we strive to make it work for free functions, too? E.g.:
fn x_mut2(Point { ref mut x, .. }: &mut Point) -> &mut i32 { x }
fn y_mut2(Point { ref mut y, .. }: &mut Point) -> &mut i32 { y }
f(x_mut2(p), y_mut2(p)); // currently: error, cannot borrow `*p` as mutable more than once at a time
If the goal is to ONLY make it work for methods: Why?
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=9f02f57616eb13e8389440471b6fee53
Regardless of which syntax gets used, it should allow borrowing some fields mutably while simultaneously borrowing other fields immutably. @eddyb How would your suggested syntax allow this?
Wouldn't this be the most natural syntax?
Afaik, there is no semantic difference between
impl Point {
fn x_mut(Point { ref mut x, .. }: &mut Self) -> &mut i32 { x }
fn y_mut(Point { ref mut y, .. }: &mut Self) -> &mut i32 { y }
}
and
impl Point {
fn x_mut(Point { ref mut x, .. } = &mut Self) -> &mut i32 { x }
fn y_mut(Point { ref mut y, .. } = &mut Self) -> &mut i32 { y }
}
Ask someone who knows the parser if either one complicates anything.
I'll note however that neither admits method call syntax because we've set a strong precedent that names besides self disallow method notation, ala
impl<T> Arc<T> {
fn try_unwrap(this: Arc<T>) -> Result<T, Arc<T>>
}
We cannot make &mut Self admits method call syntax because fn foo(x:&mut Self, y:&mut Self) or similar.
If you want method call syntax then you'll need
impl Point {
fn x_mut(Point { ref mut x, .. }: self: &mut Self) -> &mut i32 { x }
fn y_mut(Point { ref mut y, .. }: self: &mut Self) -> &mut i32 { y }
}
or
impl Point {
fn x_mut(Point { ref mut x, .. } = self: &mut Self) -> &mut i32 { x }
fn y_mut(Point { ref mut y, .. } = self: &mut Self) -> &mut i32 { y }
}
At which point = makes considerably more sense.
I think this alters the fn type, so we cannot make the first item of the method body special, like
impl Point {
fn x_mut(self: &mut Self) -> &mut i32 {
let Point { ref mut x, .. } = self;
x
}
fn y_mut(self: &mut Self) -> &mut i32 {
let Point { ref mut y, .. } = self;
y
}
}
I do not understand @eddyb suggestion per se but it sounds like a reason not to rush on this.
What about this?
impl Point {
fn x_mut(self @ Point { ref mut x, .. }: &mut Self) -> &mut i32 { x }
fn y_mut(self @ Point { ref mut y, .. }: &mut Self) -> &mut i32 { y }
}
Or even something like this:
impl Point {
fn x_mut(&mut self @ Point { ref mut x, .. }) -> &mut i32 { x }
fn y_mut(&mut self @ Point { ref mut y, .. }) -> &mut i32 { y }
}
so it's possible to omit the type until you need to specify it explicitly in cases with Pin/Box/etc
Btw, was it already decided if we only want to allow partial borrowing for the methods/self or in general for all arguments (e.g. args of free functions or non-self args of methods)? I think it makes sense to make it work for all args (for consistency).. What do y'all think?
You might think it doesn't make as much sense for free functions / non-self args, because you can just partial-borrow before calling the function, BUT to that I say:
You can also do that now with methods/self. In fact, that's my current workaround, but it's very inconvenient having to call methods in this way:
Foo::method(&mut foo.bar, &mut foo.baz, &foo.asd, other_arg), and it's similarly inconvenient for free functions, especially if you need to call multiple such functions inside a scope, that partially borrow disjoint subsets of the fields of a struct instance!
In several projects I have a lot of small helper functions that simultaneously partially borrow such subsets, some are methods but others are free functions.
I really hope we can make it work for all args in general. (Hopefully with the same syntax as with methods/self.)
For non-self arguments I think one of the big issues is how does the call site look like?
let a = foo(&mut x);
let b = bar(&mut x);
Won't work as a full mutable borrow is made first.
let y = &mut x;
let a = foo(y);
let b = bar(y);
Works but is inconvenient and can't work as a desugaring for method calls
So we need a new syntax for explicit partial borrows.
After brain storming a few ideas (&mut x._ &mut x... &mut x. .. &mut x.(..) &..mut x) I came up with &partial mut x which also works as a type name so we can put it in the signature of function that take partial borrows.
fn method(&partial mut self @ Point { x, ..});
fn funtion(Point { x, .. }: &partial mut Point);
It's possible @ makes sense, and @ is so rare the parser can do whatever, but..
All existing uses have the pattern continue on the left while the binding appears on the right, but you propose the pattern still continues on the left, while the right now just names the value being destructured.
At first blush, it makes more sense to do
fn x_mut(Point { ref mut x, .. } @ &mut self) -> &mut i32 { x }
except we end up with patterns continuing on both sides, like
match foo() {
Point { ref mut x, ... } @ Point { y: 5 } => ...
which sounds worse than repurposing the right like you propose.
I suppose nested calls create another problem here:
impl Point {
fn x_log(&self @ Point { ref x, .. }) -> &i32 { .. }
fn x_mut(&mut self @ Point { ref mut x, .. }) -> &mut i32 {
self.x_log(); // Is this valid?
x
}
...
}
I suspect the comments by @eddyb addressed nested calls better
Thing to the left of @ is always an identifer, not a pattern, isn't it? I though that would serve as most natural way to indicate the argument as self, as well as a way to omit the obvious type, either would nested calls be legal or not. In case nested calls will be legal, same goes for ordinary arguments as well: &other @ Other { .. } or &_ @ Other { .. } instead of Other { .. } @ other: &Other. That syntax for ordinary arguments would be less natural as of today, though
Non-self arguments
If type T implements a method bar that can be used with a partial borrow of T { &foo }, I'd imagine we'll soon start needing ways to pass a T in to another function just so that function can then invoke bar without having to borrow the whole T.
fn do_stuff_with_partial_t(t: ???) {
t.bar() // Partial borrow occurs here
}
Usage of @ in parameters
The following is already usable in nightly with feature(bindings_after_at)
fn parameter_pattern(s @ S { s1, .. }: &S) {
println!("{}, {}", s, s1);
}
I feel the use of @ or anything resembling a pattern on the binding side of the parameter definition would be confusing. Especially when it comes to non-self arguments, having foo(Foo { ref x, .. }: &Foo) suddenly become a partial borrow would be a compatibility hazard as currently crates can use that syntax while still being able to add other fields in the pattern without breaking backwards compatibility. If that syntax becomes a partial borrow, then the callers might hold a mutable borrow of Foo::y while calling that method, so adding ref y in the pattern breaks semver compatibility.
Call site with &mut x
I feel this could be made work by changing the semantics of the &mut operator by having it borrow as much as possible and then dropping the unneeded borrows afterwards:
let a = partial_a_of_x(&mut x) // Initially results in full borrow of all fields
// but only keeps x::a borrowed
let b = partial_b_of_x(&mut x) // Initially results in borrow of everything
// but x::a, which was already borrowed
use_all_of_x(&mut x) // Results in borrow of everything but x::a, x::b,
// which turns into an error:
//
// Error: Cannot mutably borrow x as whole.
// Previous borrows exist.
I still favor eddyb's approach. The "partial borrow" is much more closer to "type information" than "binding information". Also I feel the issue is somewhat orthogonal from arbitrary self types:
- Partial borrows needs a way to subtype (technically supertype?) the existing borrow-type
&Foo. - Arbitrary self types concerns a way to use a non-
Selftype as a receiver.
If we consider the partial borrow as a type level construct, then arbitrary self types should already work for it. Having partial borrow as a real type-level construct would also play much better with other use cases:
// .. ignoring the fact that this would be pain to parse:
impl Foo for MyType { &a, &mut b }
{
fn foo(&self) {
// Self = MyType { &a, &b }
}
fn bar(&mut self) {
// Self = MyType { &a, &mut b }
}
}
I'm not saying "implementing a trait for a partially borrowed type" should be a requirement for any partial borrow support, but keeping that option open would be good here. There's no reason the use of my Display implementation would need to conflict with a mutable borrow of the internal_cache field.
I guess the parsing issues could be solved with the use of MyType &{ a, mut b } - here &{ .. } would be the partial borrow specification and distinguishable from a simple { .. } block that might follow the type (such as in the impl block above).