rfcs
rfcs copied to clipboard
rust-lang
Efficient code reuse
Motivation
Data structures which closely fit a single inheritance model can be very efficiently implemented in C++. Where high performance (both space and time) is crucial there is distinct disadvantage in using Rust for programs which widely use such data structures. A pressing example is the DOM in Servo. For a small example in C++, see https://gist.github.com/jdm/9900569. We require some solution which satisfies the following requirements:
- cheap field access from internal methods;
- cheap dynamic dispatch of methods;
- cheap downcasting;
- thin pointers;
- sharing of fields and methods between definitions;
- safe, i.e., doesn't require a bunch of transmutes or other unsafe code to be usable;
- syntactically lightweight or implicit upcasting;
- calling functions through smartpointers, e.g.
fn foo(JSRef<T>, ...); - static dispatch of methods.
Status
There has been discussion of potential solutions on discuss (http://discuss.rust-lang.org/t/summary-of-efficient-inheritance-rfcs/494) and in several meetings (minutes and minutes).
We clarified the requirements listed above (see the minutes for details) and established that an ergonomic solution is required. That is, we explicitly don't want to discourage programmers from using this feature by having an unfriendly syntax. We also summarised and evaluated the various proposals (again, see the minutes for details). We feel that no proposal 'as is' is totally satisfactory and that there is a bunch of work to do to get a good solution. We established a timeline (see below) for design and implementation. We would like to reserve a few keywords to reduce the backwards compatibility hazard (#342).
Plan
In December the Rust and Servo teams will all be in one place and we intend to make decisions on how to provide an efficient code reuse solution and plan the implementation in detail. We'll take into account the discussions on the various RFC and discuss comment threads and of course all the community members who attend the Rust weekly meetings will be invited. We will take and publish minutes. This will lead to a new RFC. We expect implementation work to start post-1.0. If we identify backwards compatibility hazards, then we'll aim to address these before the 1.0 RC.
RFC PRs
There have been numerous RFC PRs for different solutions to this problem. All of these have had useful and interesting parts and earlier RFCs have been heavily cannibalised by later ones. We believe that RFC PRs #245 and #250 are the most relevant and the eventual solution will come from these PRs and/or any ideas that emerge in the future. For a summary of some of the proposals and some discussion, see this discuss thread.
- https://github.com/rust-lang/rfcs/pull/254
In December the Rust and Servo teams will all be in one place and we intend to make decisions on how to provide an efficient code reuse solution and plan the implementation in detail.
What ended up happening? :)
That paragraph is unfortunately outdated: we'd actually already postponed all discussion to after 1.0 since we believe any changes/features we will add are backwards compatible and there's a lot of more urgent (I.e. backwards incompatible) work that took priority.
One thing I love about rust is that its non-free abstractions are usually explicit. I love knowing that a struct is "just a struct". It would be nice to say that features requiring runtime and fancy "under the hood" data representation support (for some value of fancy) are built from pluggable, orthogonal components.
So how about something like "anonymous composition is inheritance"? The idea would be that
struct Node {
// stuff
}
struct FooNode {
Node;
// more stuff
}
implies a specific struct layout with the requisite syntactic sugar. For simplicity let's suppose that we only allow one anonymous component per struct.
EDIT: It was pointed out to me by @steveklabnik and others that rust has had trait objects since forever. I'll keep the following bit in place and just say that std::raw::TraitObject is the kind of thing I would love to be not only "pluggable" but "leave-out-able", in the sense that if I'm writing an OS and I can't implement that yet, I can tell the compiler that it can't make me any TraitObjects and that it wouldn't be ok for me to use them right now.
For dispatch, it would be nice if we could plug-in and compose the dispatch resolution mechanism. I don't want the "doesn't play well with others" feel of C++ vtables. What if I want to back this part of the language with the Objective-C runtime? Or Glib? The current "personality" of rust feels like this should be possible, in the same way that "give me your allocator function and then I'll let you use boxed things" works.
I guess my main point is that rust is the first language in a long time where I really feel like the modern features of the language don't come with being chained to a runtime layout and functionality set that's given from up on high by the Gods of Rust.
I would love it if the rust team could implement functionality like this while still retaining that ethos.
Virtual dispatch will of course be explicit under this proposal or any other.
It's worth mentioning that @nikomatsakis has been working on this proposal: http://smallcultfollowing.com/babysteps/blog/2015/08/20/virtual-structs-part-3-bringing-enums-and-structs-together/
What happened to @nikomatsakis's proposal?
I'm trying to create a general UI framework for Rust, basically based on push-pull FRP. There's no way to do it, however, because there's no way to model a type hierarchy: enums don't actually "add" types together, and the nodes of enums aren't really types, only tags.
Now that rust 2018 edition happened, shouldn't the priority be bumped up?
This is my main limiting factor and probably the last big remaining reason for outsiders to not learn rust...
Inheritance results in unreadable buggy code, at least as inheritance is implemented in most languages. I think the original servo concerns were largely addressed, as all recent requests by servo team members were far more targeted, but I'm not entirely sure about the servo teams preferred techniques.
I think the most relevant concrete proposal is delegation https://github.com/rust-lang/rfcs/pull/2393 which dramatically simplifies delegating a trait to a field. And partial delegation gives you quite a close approximation to inheritance, while likely remaining explicit enough to reduce bugs. We could see delegation put on 2020s roadmap, but it remains rather complex, and doing ti wrong might break other priorities.
I know that focus of Rust is on systems programming but business applications usually have lot of common state and without support for the issues mentioned here, are simply too awkward to implement.
As a former C++ programmer, I really got excited about all the neat things and elegant solutions provided by Rust, only to realize that I can’t use Rust because of lack of inheritance like abstraction. Adding this support will also ease rewrite of some of the Java based system software
My impression is that for most business applications, Rust's trait system already provides the encapsulation, abstraction and runtime polymorphism needed to satisfy all of the same use cases that traditional OOP inheritance systems do. And even in modern C++, class hierarchies are very rarely the best architectural choice.
It is true that you cannot mechanically transpile class-heavy C++/Java code directly to Rust, but "translating OOP patterns" to Rust is definitely a thing, and there's at least some documentation trying to address the difficulty: https://doc.rust-lang.org/book/ch17-03-oo-design-patterns.html Personally, I'm not aware of any useful OOP pattern whose practical code quality or abstraction benefits cannot be fully achieved in Rust via some other pattern (although transitioning an existing codebase from one to the other may be a difficult refactoring regardless). If you have something specific in mind, I'd recommend asking on https://users.rust-lang.org/ whether anyone's aware of a good Rust equivalent.
IIUC, the motivation for this issue is much more specific, and much more systems program-y: there are some layout and runtime efficiency guarantees that C++'s inheritance system provides, but Rust currently doesn't. AFAIK, they only matter in practice for use cases like web browser UI widget trees where a deep class hierarchy is both legitimately the right architecture and needs to be hyper-optimized (such that Java would've been a non-starter anyway). But those guarantees could be fully provided by Rust without adding traditional inheritance, most likely with things like "fields in traits" and a "prefix layout #[repr(...)] attribute" and so on.
I'm personally waiting for fields in traits to port some of my UI code over. However, it seems like the RFC repo hasn't been updated in about 2 years. Has it been abandoned?
I doubt fields in traits are abandoned per se, but it requires real design work, and they've many unfulfilled obligations around specialization, etc, so don't hold your breath.
We want fields in traits to resolve borrowing conflicts, not as some sugar for getters and setters, so some complexity arises from when and how do you promise disjointness among the fields. If you are not worried about borrowing disjoint fields, then all the disjointness promises would prevent you from doing what that getters and setters do.
There are more profitable issues, like:
I'd expect OO code would exploit dyn Trait heavily so that E0225 became an annoyance. We could either fix E0225 or else come up with good practices for combining supertraits.
You could imagine some proc macro #[derive(AnyTrait)] for trait reflection that somehow knows about other trait impls so T = dyn Trait works for many traits in
pub trait AnyTrait: 'static {
fn downcast_ref<T>(&self) -> Option<&T>;
fn downcast_mut<T>(&mut self) -> Option<&mut T>;
}
Could/should some Any variant include a lifetime?
pub trait Any1<'b>: 'b {
fn type_id<'a: 'b>(&'a self) -> TypeId;
}
impl<'b> dyn Any1<'b> {
pub fn is<T: Any1<'b>>(&self) -> bool { ... }
pub fn downcast_ref<T: Any1<'b>>(&self) -> Option<&T> {
}
I'm personally waiting for fields in traits to port some of my UI code over.
I second this.
While it's possible to do something like the following(where both Shape and Square implement Area):
trait Area{
fn calculate_area(&self) -> f64;
}
struct Shape {
area: f64,
scale: i32,
}
struct Square {
width: f64,
shape: Shape
}
We could then call square.shape.area to grab the area, but to calculate the area we would only need to call square.calculate_area. It doesn't make sense, to me, to have a mechanism for inheriting methods but not fields. I'm seeing a lot of code duplication in my projects because of this, and that code duplication isn't necessary in Rust competitors(mainly, c++)
@es50678 I think that you are asking for #2393. Also have a look at the Ambassador crate https://github.com/hobofan/ambassador
I have a bad feeling that the only reason that this issue hasn't been closed is that it's coming from the Servo team, and that Rust will never actually add any of these things for ideological reasons. Sure, there's work that still needs to be done before this can be achieved, but no one seems to be working on it because it's “gross” OOP.
Well, I think it's a pretty solid guess, given that the issue is 6 years old. If anyone (from the developer) had any interest in this being done, that would already had been indicated.
I was in a salty mood when I wrote that earlier. I apologize for being annoying.
I think it's far more likely that issues on this repo just aren't curated that much. There's never really been a clear policy for when it makes sense to open or close an issue here, unlike on rust-lang/rust where inactive or obsolete issues seem to get closed regularly. Probably the only reason most of us even see these issues is because we're watching the repo for pull requests.
It would be interesting to hear from the Servo team whether they still care heavily about this. My personal impression from being over-attentive to the last several years of Rust design discussions is that "efficient single inheritance" represents only one set of layout optimization guarantees that real programs are interested in, and there are many other layout optimizations that matter, all of which can be achieved "manually" today (sometimes with unsafe, sometimes with safe wrapper types that have a bunch of bitshifting methods), so it's becoming difficult to argue these layouts in particular deserve more first-class support. But that's still just a vague impression I have.
For sure, it's possible to do some of these things already through lots and lots of macros. If it were only Servo that needed this that would probably be the best solution. However, I see similar requests a lot from those working on GUIs.
I think it's far more likely that issues on this repo just aren't curated that much.
Yeah, I didn't mean to imply any ill intent. I was just in somewhat of a mood and I happened to remember this issue for some reason or another. It would probably be more productive for me to work on the mothballed fields in traits RFC or something like that.
@Serentty
- What exactly is the difference to well-named hygienic macros aside of evaluation order (penalizing non-usage) and no explicit syntax?
- Inheritance has a ton of edge-cases and the worst part about it is implicit state hiding from the user.
- Fundamentally this is about enforcing on a language-level not to have a million abstractions with the well-known problems by not making people rely on stateful (and very hard to refactor) abstraction.
And yes, I do get your point. Dynamic layout structure-changes are hard to model without inheritance, which is a fundamental tradeoff in data-driven approaches.
I would argue that Rust conventions are well-established at this point. I don't think the Rust community would rush to use inheritance at every opportunity if they had it. The attitudes I see towards it are proof enough of that to me. Also, from what I can see pretty much any realistic inheritance proposal for Rust would be through traits, not through direct subtyping, as C++-style subtyping leads to all sorts of memory safety issues when objects get truncated. I think that the more roundabout nature of trait-based inheritance would largely offset the temptation to use inheritance where it isn't appropriate.
I'd think GUIs should obtain correctness and security from the lower layers, but then expose "sloppy but convenient" abstractions like inheritance to higher layers.
Servo's script::dom implements subtyping inheritance as required by JavaScript's DOM. Your preferred abstractions might differ if you're doing some native GUI or a game engine. It's true some GUIs like Qt encourage adoption by being JavaScript-like, but if you're doing that then maybe you should build directly upon Servo.
Inheritance distracts from real improvements, like delegation and fields-in-traits, but also..
We need trait specialization, not for being object orientated, but because it closes a performance gap with C++.
We need good toolbox crates for writing proc macros, so that proc macro code become semi-readable, which should improve all Rust DSTs, including more object oriented ones.
We might address E0225 so that dyn TraitA+TraitB becomes more usable, perhaps only if trait aliases exist like pub trait TraitAB = TraitA+TraitB, but perhaps via some "unnaned supertrait declaration" like pub trait _ = TraitA+TraitB;.
There is an extremely manual choice between monomorphisation vs trait objects, and Rust favors monomorphisation currently, so we should make trait objects more useful and ergonomic, develop patterns for being flexible about monomorphisation vs trait objects, and develop profiling tooling and practices that show when trait objects win.
We cannot afaik stabilize the current TraitObject type, partially because smart pointers could be larger than one usize. We could however provide some vtable access mechanism that handles smart pointers correctly like:
pub type VTablePointer = *mut ();
/// see https://github.com/rust-lang/rfcs/blob/master/text/1598-generic_associated_types.md
trait PointerFamily {
type Pointer<T>: Deref<Target = T>;
}
trait PointerMutFamily : PointerFamily
where for<T> <Self as PointerFamily>::Pointer<T>: DerefMut { }
/// Access smart pointer's vtable for types like `&`, `&mut`, `Box`, `Rc`, `rc::Weak`, `Arc`, and `sync::Weak`, but not `Cell`, `RefCell`, `Mutex`, etc.
unsafe trait PointerVTable : PointerFamily {
/// Raw pointer to the vtable of a `Self::Pointer<T>` when `T = dyn Trait`.
/// It normally returns the first aligned `*mut ()` after the pointer itself.
/// If `T` is not `dyn Trait` then dereferencing this pointer is undefined behavior.
fn vtable_offset<T>(ptr: Self::Pointer<T>) -> *mut VTablePointer;
}
I presume "cheap *casting" should really mean casting matrix among dyn Trait types, unlike Servo's DOM. If we can access the vtable pointer then doing this resembles:
/// Dynamic cast family
/// Identifies related related traits among whose `dyn Trait` types casts make sense.
pub trait DynCastFamily { }
/// Implemented by proc macros for `dyn Trait` types corresponding to traits within a dynamic cast family to which trait object casts occur.
unsafe pub trait DynCastTo<CF: DynCastFamily> : ?Sized + 'static {
const dyncast_index: usize;
}
/// Implemented by proc macros for real types within a dynamic cast family on which trait object casts occur. Also a super trait for every trait within the dynamic cast family.
unsafe pub trait DynCastFrom<CF: DynCastFamily> : 'static {
const fn dyncast_vtables(&self) -> &'static [VTablePointer];
}
impl<CF: DynCastFamily> dyn DynCastFrom<CF>> + 'static {
fn dyncast<T,P>(mut self: P::Pointer<Self>) -> Option<P::Pointer<T>>
where T: DynCastTo<CF>, P: PointerFamily+PointerVTable,
{
let i = <T as CastTo<CF>>::dyncast_index;
let new_vtable = self.dyncast_vtables()[i];
if new_vtable.is_null() { return None; }
let old_vtable = PointerVTable::vtable_offset(self);
Some(unsafe { *old_vtable.write(new_vtable); mem::transmute(self) })
}
}
In short, we declare "casting families" of traits so that proc macros and build tooling construct a "casting matrix" that gives every type T within the casting family a slice of vtable pointers, and gives every dyn Trait for a trait within the family an index into all these slices. The vtable pointer is null if T: !Trait.
We'd expect high performance from this solution because it requires only two pointer dereferences from potentially well traversed tables and one if check. Arbitrary self types appear essential, but we do not require that PointerFamily ATC tricks, since separate dyncast_* methods for every self type work too, but they help with user defined smart pointers. We could achieve this with proc macros alone, and avoid build tooling, if we populate this casting matrix at runtime using lazy_static.
I agree so strongly with everything you said there. Fields in traits are one of the features I find myself missing most in Rust. I don't care if one type can automatically inherit from another as long as I can abstract over types in terms of which fields they have. I think fields in traits are absolutely a good substitute for inheritance, while not necessarily encouraging people to use an inheritance-based design pattern.
Multiple trait objects are also something I've found myself running into and missing before.
Another interesting aspect is trait "object safety" and methods on dyn Traits, because they make working with "derived traits" (e.g. traits which have another trait as supertrait) a bit annoying, because traits can't be easily "upcasted". It is possible to work around this (I've done it in the as-any crate, even without many macros), but it starts to become annoying as soon as more traits are involved.
I do very much agree. Having the behavior explicit via macro_syntax/arguments would work.
We need trait specialization, not for being object orientated, but because it closes a performance gap with C++.
Could you shortly elaborate?
We need good toolbox crates for writing proc macros, so that proc macro code become semi-readable, which should improve all Rust DSTs, including more object oriented ones.
We might address E0225 so that
dyn TraitA+TraitBbecomes more usable, perhaps only if trait aliases exist likepub trait TraitAB = TraitA+TraitB, but perhaps via some "unnaned supertrait declaration" likepub trait _ = TraitA+TraitB;.
Would it make sense to enforce centralized type lookup on these (ie by cargo) or in a special file types.rs/traits.rs and have special macro syntax for aliasing aliased traits, if they are not in the local package/crate?
We need trait specialization, not for being object orientated, but because it closes a performance gap with C++.
Could you shortly elaborate?
The original RFC https://github.com/rust-lang/rfcs/blob/master/text/1210-impl-specialization.md is still a decent resource for this. Some details have changed, but it introduces all the important concepts, and provides clear and compelling examples.
Would it make sense to enforce centralized type lookup on these (ie by cargo) or in a special file types.rs/traits.rs and have special macro syntax for aliasing aliased traits, if they are not in the local package/crate?
Multi-trait objects have been covered extensively in many past threads, so we should probably avoid duplicating discussion. See https://github.com/rust-lang/rfcs/issues/2035 for example.