rfcs
rfcs copied to clipboard
`Forget` marker trait
Add a Forget marker trait indicating whether it is safe to skip the destructor before the value of a type exits the scope and basic utilities to work with !Forget types. Introduce a seamless migration route for the standard library and ecosystem.
Unresolved questions
- [ ] Unsafe guarantee that
!Forgetgives to theunsafecode is already fulfilled for all'statictypes. Because of that, it doesn't really make a lot of sense to have!Forget + 'static, it is still sound tomem::forgetthat type. Should we forceimpl<T: 'static> Forget for Tthen? Won't that impl create some unexpected problems?
This definitely seems to be a reasonably motivated RFC, although it's going to definitely be something I'll have to read through a lot more closely to fully comment on it. A few first impressions:
- I don't think that
Forgetas a name is very good for a marker. Forgetting something as a verb is fine, but as… an adjective? Not really. I think that at leastForgettablewould be a better name, although something likeForgetSafewould probably be more in line for this trait, in line withUnwindSafe. - While I appreciate the effort to be as accessible as possible by explaining everything, including concepts like RAII guards which not everyone might be familiar with, your desire to explain everything in advance of using them means that the RFC itself is difficult to follow. For example, the motivation section doesn't really get into the actual motivation until it provides multiple code blocks and examples. This makes the entire thing difficult to follow, since I can't easily look at the top of the section and understand what it's about. Chronological order is good for explaining the prior art, but not for the motivation, IMHO.
- On that note, I don't think that the current guide-level explanation is very good for a guide-level explanation. It feels very structured like an FAQ, which I don't think is good for the guide level: you're explaining to someone what the feature is outright, and not just answering their questions about it.
Again, I do want to go through this a bit more closely before fully commenting on it, but I think that you definitely need to go back and make the primary motivation for this crystal clear: because of forget, invariants in types cannot be violated by borrowing wrappers. This was not a problem before async code, because you could effectively ensure that function calls happen to completion (minus poisoning semantics) and nothing is ever left in an invalid state. However, with async code, you now have the issue that you can "forget" to finish part of a full function call via forgetting its Future, which leaves things in an invalid state.
Generally, the only way to deal with this is to make these calls unsafe and tell people to pinky-promise they run the future to completion, but it would be nice to be able to have a safe way to do this instead.
@clarfonthey We do use verbs as trait name when the only purpose of the trait is to support that verb, think of Send or Copy or Borrow or std::iter::Extend.
This is definitely a much desired feature that would enable new safe design patterns in Rust.
Having a gradual migration path with default_generic_bounds sounds reassuring.
I appreciate all the links to the prior art.
It should be noted that a type being Forget does not imply in any way type linearity: The destructor might or not be dropped, thus any code that relies on that assumption for safety is still unsound. This should be documented clearly on the trait to avoid people footgunning themselves.
This can happen in many ways, which can generally be grouped into the following categories:
- Not continuing execution of thread or task. This can happen easily with a deadlock, or more rarely with an infinite
loop {} - Leaking the resource directly:
Box::leak, or putting it into a static, likeOnceLockorLazyLock. - aborting the process
This does not make things like async_cuda sound as one can easily poll and leak the future.
It should be noted that a type being
Forgetdoes not imply in any way type linearity: The destructor might or not be dropped, thus any code that relies on that assumption for safety is still unsound. This should be documented clearly on the trait to avoid people footgunning themselves.This can happen in many ways, which can generally be grouped into the following categories:
- Not continuing execution of thread or task. This can happen easily with a deadlock, or more rarely with an infinite
loop {}- Leaking the resource directly:
Box::leak, or putting it into a static, likeOnceLockorLazyLock.- aborting the process
This does not make things like
async_cudasound as one can easily poll and leak the future.
This is incorrect as explained in the RFC. Please provide an example of unsoundness that exploits that loop {} can cause unsoundness. And RFC directly forbids Box::leak and talks about statics, which indicates that you probably did not read the RFC itself.
And RFC directly forbids Box::leak and talks about statics
I retract my comment with the correction in the typo of the RFC :stuck_out_tongue:
Please provide an example of unsoundness that exploits that
loop {}can cause unsoundness.
The point has already been addressed here: https://github.com/rust-lang/rfcs/pull/3782#discussion_r2003437666
And RFC directly forbids Box::leak and talks about statics
I retract my comment with the correction in the typo of the RFC 😛
Please provide an example of unsoundness that exploits that
loop {}can cause unsoundness.The point has already been addressed here: #3782 (comment)
Great. Then you can find an explanation from @kornelski or me under that comment you linked.
Imo the problem of forget is that there is no safe way to enforce to run cleanup of the object before the memory becomes invalid. The examples in the rendered RFC are all showing this problem.
Wouldn't be enough if the proposed Forget trait prevent memory being invalid? Then I think these assumptions are enough to solve the problems using Forget trait.
As for any T If
- T is 'static and Forget then It is safe to
Box::leakandforget. - T is 'static and !Forget then It is safe to
Box::leak, but not safe toforget. - T is non 'static but Forget then It is safe to
Box::leakandforget. - T is non 'static and !Forget then It is not safe to
Box::leakorforget.
To support case 2 properly, Having additional derived Leak(or other better name) trait is enough.
pub trait Leak {}
default impl<T: ?Sized> Leak for T where T: 'static {}
impl<T: ?Sized> Leak for T where T: Forget {}
...
impl<T: ?Sized> Box<T> {
pub fn leak<'a>(self) -> &'a mut T where T: Leak;
}
@storycraft You can't have channels then, and PhantomData in general would not be protected. It would be quite funny to look for "some byte I can borrow" instead of just having PhantomData. And even with some memory, connecting it to the lifetime is not trivial - rarely unsafe code stores the reference, it is usually converted into pointer + PhangomData. Thus, only guarding memory is both harder and less ergomic.
You can't have channels then
How so? The Leak trait I wrote is derived from the Forget trait in the RFC. It is not required actually, but allows the user to use as bounds for some exceptionally safe case.
You can't have channels then
How so? The
Leaktrait I wrote is derived from theForgettrait in the RFC. It is not required actually, but allows the user to use as bounds for some exceptionally safe case.
Oh, I see, I thought you proposed to reduce the safety guarantee to just memory. Looking at it again, it is tackling that unresolved question about !Forget + 'static, right? Well, I certainly do not oppose any ideas, if this has enough use cases and would not block Forget, I think it may be.
I have found this text to be unsubstantiated, and rather simply a "what if" exploration with examples and counterexamples. There are no new answers to questions I've faced from Rust major contributors, leaving progress on this feature still stuck.
I have found this text to be unsubstantiated, and rather simply a "what if" exploration with examples and counterexamples. There are no new answers to questions I've faced from Rust major contributors, leaving progress on this feature still stuck.
Please note that the single case of “what if” you commented on earlier occurs in the guide-level explanation, which is intended to provide an intuitive understanding of the feature.
From the rust-lang/rfcs repository:
- Explaining the feature largely in terms of examples.
- Explaining how Rust programmers should think about the feature, and how it should impact the way they use Rust. It should explain the impact as concretely as possible.
The section you commented on explained concretely: there should be a borrow-checked lifetime between tx and rx handle, no matter the implementation details about where sent values are living - on the rx's stack, on the stack of the tx's and rx's parent, inside the shared allocation etc.
I have found this text to be unsubstantiated, and rather simply a "what if" exploration with examples and counterexamples. There are no new answers to questions I've faced from Rust major contributors, leaving progress on this feature still stuck.
Please note that the single case of “what if” you commented on earlier occurs in the guide-level explanation, which is intended to provide an intuitive understanding of the feature.
From the rust-lang/rfcs repository:
- Explaining the feature largely in terms of examples.
- Explaining how Rust programmers should think about the feature, and how it should impact the way they use Rust. It should explain the impact as concretely as possible.
Doesn't change what I've said, nothing new.
The section you commented on explained concretely: there should be a borrow-checked lifetime between tx and rx handle, no matter the implementation details about where sent values are living - on the rx's stack, on the stack of the tx's and rx's parent, inside the shared allocation etc.
Discussed in review.
Considering I couldn't find anything in the reference-level explanation closely related to the explanation on why rendezvous channels could not operate on unforgettable types, let me exactly describe why rendezvous channels create a major hole in the current design.
Let's start by exploring the new JoinGuard design. JoinGuard has to be an unforgettable type, but one of the obvious restrictions on unforgettable types is to restrict transfer of object's ownership to itself, rendering the destruction of an object to be avoidable while reaching the end of a lifetime. This means there should be no way to transfer ownership of the JoinGuard object to itself.
You could restrict JoinGuard to also be !Send type. Logic is simple: transfer of JoinGuard between threads is prohibited, thus transferring it from the source thread to the one this JoinGuard refers to is impossible. Graph of threads borrowing from other threads may only be a DAG (a tree in this case), meaning there is an order in which threads could terminate while not invalidating any reference.
Just like async drop, unforgettable types were intended to enable safe borrowing async tasks as part of an idea of the structured concurrency. Composition of borrowing async tasks means being able to hold one such task handle inside of another (borrowing) async task, and analogously designed non-thread-safe ScopedAsyncTask: !Send makes it impossible to store the inner task inside of the outer task without making the outer task !Send too, thus rendering structured concurrency unobtainable in today's rust.
One might speculate and try to fix some holes, for example by making
JoinHandle: !Send, but this can only count as a workaround. If we look at the problem in depth, we can see thatForgetis generally incompatible withRc, as well as other APIs that can be expressed with its signature, because it creates a hidden self-reference.
A traditional approach to the message-passing cannot be applied to
!Forgettypes - slightly different APIs should be developed, preserving a lifetime connection betweentxandrxhandles.
It is clear to a library developer that any synchronization channel with a buffer (for example mpsc) could easily create a self-reference by sending a receiver object to its own channel buffer, as such T: Forget bound is obvious. However your reasoning for a sound rendezvous channel having T: Forget bound too is incomplete, which is simply "because it makes thread-safe JoinGuard unsound as in combination can create a self-reference." If you cannot derive both of these APIs as sound from the language's semantics independently, it becomes fair to say this feature will make the language itself (more) unsound. And that is just a single example with rendezvous channels, how could you really know there're no other holes?
What you are actually trying prohibit is the transfer of (any) JoinGuard between threads, while one of them is handled by a JoinGuard too. You have stated that solution with JoinGuard: !Send is a "workaround", which I've shown is a good enough for JoinGuard alone, while the proposal with channels and lifetimes to me seems as vague as possible and wouldn't be clear to any library developer unless there's a clear elaboration on it, otherwise in currently described terms being a patchwork.
Is what I'm talking about clear now? If this issue with rendezvous channels wasn't there I would have already published this RFC. For now unforgettable types proposal have only been discussed on the rust zulip. Here's a thread about this exact problem:
#wg-async > Structured parallel tasks require !Send in Send coroutines
EDIT: If you think your RFC have already addressing this concern, please leave a citation.
as such
T: Forgetbound is obvious
However your reasoning for a sound rendezvous channel having
T: Forgetbound too is incomplete
Do we have a misunderstanding? You absolutely can and should send T: !Forget types using channels! RFC is not claiming that rendezvous channels must have T: Forget, it is saying that to be able to transfer T: !Forget, tx and rx should be connected via some lifetime (borrow the allocation, not both ownlike Arc), even if phantom, and are !Forget themselves - it has no ergonomic downsides. They anyway share some data via some allocation or the stack where new() was called, how would tx know where rx's stack is? And, RFC is stating things about rendezvous channels to help them to adapt to the new change, not the other way around (it is elaborated later).
If your question is "your reasoning is incomplete because you ban tx and rx without a lifetime connection not based of the language semantics but based on the broken use case" - no. Reference Level Explanation gives a guarantee that values borrowed by !Forget types will remain borrowed untill drop is executed. Current channels do not fulfill it, they will "weaken" the state and remove the borrow if combined with anything like Box<dyn Trait>, while drop is not executed. There is nothing special about scoped and JoinHandle, see guide level explanation. By giving tx and rx lifetime (borrowing from a single owner instead of being like Arc with a shared allocation ownership) you can have sound code. And generally, tx and rx not having a lifetime is a consequence of the fact that you can't pass them to tasks/threads because those require 'static closures. It is a vicious cycle of Arc and 'static we are part of. If not 'static requirements, channels didn't need to have Arc inside for shared state, and we wouldn't even have to have that discussion today about channels without lifetimes.
The text later was written before I spot this misunderstanding, but I think it still should be relevant. I do not enjoy long scrolls, so I would put in the details.
Long response to the previous comment
The reason this RFC is published by me is because I fundamentally disagree with you on that point, and what is new. Moving JoinGuard to another thread is a problem only if you are trying to move it inside the same thread - and borrow checker is already fine-tuned around detecting any kinds of self references, it is already enforcing DAGs you mention. But tx and rx without a lifetime, which is only possible with some unsafe code involved, like Arc, can sidestep those checks.
The most important thing is the cost-benefit ratio. Benefits are immense, motivation is huge. From the side of the costs, you don't loose anything by requiring tx and rx to be connected via lifetime and be !Forget themselves, except the ability to create that unsoundness. With async scope you can pass them into other tasks freely. Rendezvous use case is covered by the design, as well as all others. There are literally no downsides in that approach except the migration, do you want me to do a formal soundness proof like RustBelt?
And please note, we are not talking about structured concurrency alone in this RFC, there are plenty of other use cases.
What I am trying to say in this RFC is that everything should be connected via lifetime - ergonomics is still at the highest level, but borrow checker is able to control you code better. Rc-like APIs are using unsafe code to introduce dynamic lifetimes, they are directly sidestepping the borrow checker.
simply "because it makes thread-safe JoinGuard unsound as in combination can create a self-reference."
The fact that you can exploit that unsoundness by sending JoinGuard into itself is not the motivation, it is just that it is literally unsound and you can reveal that unsoundness by placing a lifetime, which was removed and made the borrow checker blind.
To give more intuition: disconnected tx and rx for !Forget type is literally unsound, !Forget is not 'static, but send() has weakening signature (after that function T is gone for the borrow checker), and borrow checker stops any protection. This is why we disallow channels, not because we want JoinGuard be Send. But with a lifetime between tx and rx borrow checker is not making such assumption, because it sees where the value may have been moved, with a lifetime.
To reiterate, the requirement to have a lifetime between tx and rx is the statement of this RFC. When writing unsafe code, it is your responsibility to assess, would it be compatible with the language rules or not (if you have use case to read from the foreign *const i32 while &mut i32 to the same memory exists, your use case is not compatible with Rust). RFC does not leave any way to get unsoundness with std by using already existing borrow checker, while other libraries should audite themselves according to the language rules. This RFC made additional effort (in order to be more attractive) to help the ecosystem and already solved many uses cases for them, those listed in the motivation, including those rendezvous channels. Even if there exists another way to solve this by making JoinGuard: !Send, this is just another way to make things sound (I am not claiming that you will not find more holes with that directed patch like JoinGuard: !Send, but let's assume it), you can analyze which approach has better cost-benefit ratio to understand why RFC took another approach.
If your question is "Why require lifetime between tx and rx instead of forcing JoinGuard: !Send, it is because the former has no downsides and "feels natural" and uses the borrow checker while the latter has some downsides and "feels like a hack".
You certainly do have a grasp on this proposal. I would truly appreciate if you would be open-minded and instead of purely destructive actions (for now you just brought up some cases and speculations where you personally believe RFC should be discarded, without considering const-benefit ratio) put some work into improving the RFC. For example, looking for additional costs, investigating how you can use safe code to create unsoundness (which happens from time to time even in the stable Rust due to human factor), or thinking about real use cases to decide on 'static + !Forget situation using cost-benefit ratio, improving the migration plan, which is really the only pain point with this RFC.
I'm sorry if I've offended you in any way or if you think I've stolen your credits - I'm not claiming anything, I just believe that you're not correct and I'm willing to spend the time and effort to improve the language. If you feel like I answered your concerns, please mark your comments as resolved, thank you.
Wouldn't be enough if the proposed
Forgettrait prevent memory being invalid? Then I think these assumptions are enough to solve the problems usingForgettrait.As for any T If
- T is 'static and Forget then It is safe to
Box::leakandforget.- T is 'static and !Forget then It is safe to
Box::leak, but not safe toforget.- T is non 'static but Forget then It is safe to
Box::leakandforget.- T is non 'static and !Forget then It is not safe to
Box::leakorforget.
After considering this for a bit, I don’t think category 2 makes sense. If you have an owned value (on the stack or in a Box), you can always invalidate the memory simply by moving the value out to some other location. If the value has to stay in a particular location, the proper mechanism is Pin.
After considering this for a bit, I don’t think category 2 makes sense. If you have an owned value (on the stack or in a
Box), you can always invalidate the memory simply by moving the value out to some other location. If the value has to stay in a particular location, the proper mechanism isPin.
If the value has to be pinned in particular place, than using Pin is correct. Maybe my first sentence was a bit misleading because I thought that moving values are not invalidating memory(place for the value still exists, although it's moved). So category 2 is more like movable and unforgettable 'static type(not sure if it would be useful or not). If it leaks, the backing memory would be still valid to use. But forgetting value on the stack deallocate memory without dropping. Pin has almost same drop guarantee too. But only for 'static type(for non 'static type references can be dangling). Forget trait prevents these cases.
AFAIR original reason why std::mem::forget was deemed safe is because you can effectively omit drop by stuffing guard object into refcounted pointer loop. Object is alive but unreachable.
How does !Forget solve that issue?
AFAIR original reason why
std::mem::forgetwas deemed safe is because you can effectively omit drop by stuffing guard object into refcounted pointer loop. Object is alive but unreachable.How does
!Forgetsolve that issue?
I'd assume things like Rc<T> are changed to require T: Forget (edit: rather than requiring T: Forget for any Rc<T>, instead T: Forget is required whenever constructing a Rc<T>, unless you use a new unsafe function)
AFAIK there was another solution, scope-bound lifetimes. You effectively forbid object from leaving scope it was created in.
AFAIK there was another solution, scope-bound lifetimes. You effectively forbid object from leaving scope it was created in.
Design of the RFC should support any extensions, as you can just create your own abstractions and use unsafe constructors internally, publish as a crate etc.
Does this RFC address this issue raised by @withoutboats?
Does this RFC address this issue raised by @withoutboats?
I don't think it has that specific issue, because it has a different backwards compatibility issue that prevents you from getting into that situation:
If crate A has a trait like:
trait A {
fn foo<T>(t: T);
}
and crate B has an implementation of A, then if A changes the bound on T to be ?Forget, then that is a breaking change, because the impl in B would have the constraint of Forget, which is more specific.
Unless there was some mechanism to allow an impl to have more specific bounda than the trait it impls.
Would this impact any std traits?
Edit: I think this issue has some similarity to const methods in traits, and perhaps it could be solved in a similar way, once that is finalized.
Relevant, from @lcnr: https://lcnr.de/blog/2025/07/28/implicit-auto-trait-bounds.html
I've been doing some work trying to make a stack allocated structured concurrency runtime that would be sound if a Forget marker trait existed. I believe (very possible I'm wrong) that you can take the approach you outlined in the performance section a step further and introduce a sound ValueGuard pattern that enables similar functionality to having a single Rc and Weak ptr without heap allocation.
Here's an implementation I wrote for ValueGuard
The use in an async runtime would be introducing an unclonable GuardRegistration<'a, Wake> inside core::task::Context<'a>. This would enable sound (and I think backwards compatible) storage of ptrs to tasks that live for 'task in an event loop living for 'events where 'events: 'task without heap allocation or static task pools.
@anglesideangle it sounds great, never thought about using Context for this. In combination with TAIT it may get extremely useful I believe. You may be interested, I made a little crate called extend-mut and there's async version too, which basically utilizes the same approach of guards.
I noticed that not all types are equally unleakable. For example, a ValueGuard can be safely leaked to the heap because it only requires drop to clean up pointers to itself. This is different from the RAII guards demonstrated in the RFC, for which leaking to the heap allows them to escape the scope drop must be ran in. Do you think it's worth making a distinction between stack leaking via mem::forget and heap leaking via Box::leak, Arc::new, etc?
it seems @storycraft brought up similar ideas previously
- T is 'static and Forget then It is safe to
Box::leakandforget.- T is 'static and !Forget then It is safe to
Box::leak, but not safe toforget.- T is non 'static but Forget then It is safe to
Box::leakandforget.- T is non 'static and !Forget then It is not safe to
Box::leakorforget.To support case 2 properly, Having additional derived
Leak(or other better name) trait is enough.pub trait Leak {} default impl<T: ?Sized> Leak for T where T: 'static {} impl<T: ?Sized> Leak for T where T: Forget {} ... impl<T: ?Sized> Box<T> { pub fn leak<'a>(self) -> &'a mut T where T: Leak; }
I agree the proposed system with:
- it's unsafe to
mem::forget!Forget - it's unsafe to
Box::leak!Leak - all
!Leakare!Forget
would be more comprehensive