dotnet
generic constraint: where T : ref struct
Today it's not possible to use a ref struct as a type argument to a generic method or method of a generic type. We'd love to see it become possible to:
class A<T> where T : ref struct
{
// T t; <-- this would generate a compiler error. not allowed to leak a T to the heap.
static void SomethingLegit(T t)
{
//totally fine to do ref-struct compatible things with t here.
}
}
this feature would be valuable to us especially in combination with: https://github.com/dotnet/csharplang/issues/1147
Design Meetings
https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-09-26.md#ref-struct-improvements
Would be nice also if ref structs could implement interfaces, and these interfaces be generic constraints. Naturally the ref struct can't be converted to an object known to implement an interface, but the members could still be accessible on the type itself. If the runtime emitted independent functions for ref struct as it does for normal struct this ought to have no within-method cost. I don't know if the call-site is specialised or requires indirection, but this could be an overhead. I should expect that such generic specialisation would at least be applicable to performance-hungry domains that ref struct is targeting, even if an extra indirection is required (if I've correctly understood how it already works in the CLR).
// generic method is JITted independently for each combination of struct/ref struct type params
void Terminate<T>(T t) where T : ref struct, IAdvanceable
{
IAdvanceable a = t; // error: illegal conversion
while (t.Advance()) // statically dispatched
{ }
}
There may be some syntax horror if interface disambiguation is required, because ((IFace1)refTypeArg).CommonMethodName() at least looks like a conversion, but I don't know enough about the compiler to know if this is a significant technical concern.
Again, this would be a somewhat limited capability, but I don't see why it can't run as far as ref struct, and would allow, e.g. ReadonlySpan<T> to implement IReadonlyList<T>, which - while requiring a new method owing to the ref struct constraint requirement - would at least preserve implementation independence, and allow such existing and familiar APIs to be adopted from the outset.
@lucasmeijer can you please explain more about how you envision this to tie in with #1147?
would allow, e.g.
ReadonlySpan<T>to implementIReadonlyList<T>
IReadOnlyList<T> inherits from IEnumerable<T>, which has IEnumerator<T> GetEnumerator(). Without further changes, that enumerator could legally be stored off somewhere for MoveNext() to be called when the original span has gone out-of-scope.
I guess you could have the compiler play detective on everything that originated from the T parameter's methods, but that sounds to me like a ton of work (both for the developers to implement and for the compiler to validate) for an unclear amount of benefit.
Obviously, a new interface could be created, but that sorta defeats this:
allow such existing and familiar APIs to be adopted from the outset.
I'm bringing this up because if we want to support the use case of having one method that accepts either IReadOnlyList<T> or ReadOnlySpan<T> and then (after JIT) performs no worse than a method that accepted just one or the other, then let's make start an open-ended proposal for that, and then maybe the development and design effort would be better spent on a more specific language feature instead of serving as auxiliary justification for more weird generic type constraint soup to enable more ugly code.
Like, a method where TList : maybeRefStructButMaybeNot, IReadOnyList_RefStructCompatible<T> leaves a bad taste. But we're looking at designing / developing the language itself here, so we're not restricted to just thinking inside the box. We're writing the rules, so this is fair game (please go easy on this idea, I don't claim to be a good language designer, it's just a top-of-my-head idea to illustrate a point):
static void ActOnAllEvens<T>(listlike_parameter<T> lst, Action<T> action)
{
for (int i = 0; i < lst.Count; i += 2)
{
action(lst[i]);
}
}
static void Main()
{
Span<int> someList = stackalloc int[10];
/* populate someList */
ActOnAllEvens(someList, Console.WriteLine);
// lots of businessy code at my work returns ReadOnlyCollection<T>
ReadOnlyCollection<int> someCollection = SomeBusinessyCode();
ActOnAllEvens(someCollection, Console.WriteLine);
}
That doesn't seem impossible to implement (minimally, you could make overloads for a well-defined set and then restrict how such a method could be overloaded, though that feels ugly), and IMO the code would look a lot cleaner than a magic set of generic type constraints.
I don't expect anything like that to actually happen, but only because I don't see much value in the use case of being able to have one method that accepts either IReadOnlyList<T> or ReadOnlySpan<T>.
@airbreather I suspect you meant to ping me!
You are absolutely right, GetEnumerator() would effectively preclude a ref struct inheriting from IEnumerable or IReadOnlyList etc. which wouldn't work. I'd completely missed that. However, I don't think this is a general concern: if an interface doesn't make sense to be implemented by a ref struct then that can be left to the ref struct's author's discretion, and the compiler will already prevent, for example, a 'correct looking' implementation of GetEnumerator(). I wouldn't support any 'detective' work on the compiler.
I reckon that a maybeRefStructButMaybeNot - horrendous as it looks - would be an interesting feature (has all the restrictions of a ref struct, but allows another things (perhaps just structs, I haven't thought the possibilities through). I should really further my understanding of ref structs: perhaps the ref struct constraint itself would allow ref variants of normal structs, or something daft like that. This became wishy-washy rather quickly...
I'm bringing this up because if we want to support the use case of having one method that accepts either IReadOnlyList<T> or ReadOnlySpan<T>
Yeah, that would certainly be nice. I think we can already wrap an IReadOnlyList<T> in a ReadOnlySpan<T>, so it shouldn't be a dire concern for new APIs.
@airbreather I suspect you meant to ping me!
@VisualMelon I should have, yes, though my first line was in reference to this line:
this feature would be valuable to us especially in combination with: #1147
As for my reply...
However, I don't think this is a general concern: if an interface doesn't make sense to be implemented by a
ref structthen that can be left to theref struct's author's discretion, and the compiler will already prevent, for example, a 'correct looking' implementation ofGetEnumerator().
My comments here were in response to:
I don't see why it can't run as far as
ref struct, and would allow, e.g.ReadonlySpan<T>to implementIReadonlyList<T>, which - while requiring a new method owing to theref structconstraint requirement - would at least preserve implementation independence, and allow such existing and familiar APIs to be adopted from the outset.
I don't have data to support the following claim, but my intuition is that many "familiar APIs" have such "showstoppers". Each "showstopper" weakens the value of this part of the proposal, and I think that if it can't buy us the ability for ReadOnlySpan<T> to stand in for pretty much any interface that ultimately inherits from IEnumerable<T>, then I'm really not seeing much value (which is why I prodded about #1147).
I reckon that a
maybeRefStructButMaybeNot- horrendous as it looks - would be an interesting feature (has all the restrictions of aref struct, but allows another things [...]
I was thinking that that would be the semantics of the generic type constraint proposed in this issue. My intuition is that nobody actually wants to literally constrain T to be a ref struct type; rather, the desire is to be able to use ref struct types as generic type parameters.
e.g., I see no fundamental reason why your Terminate<T> method should reject a reference-type argument that implements IAdvanceable.
I think we can already wrap an
IReadOnlyList<T>in aReadOnlySpan<T>, so it shouldn't be a dire concern for new APIs.
Can you please elaborate more on what you mean by this? As-written, that's true for some implementations of IReadOnlyList<T>, but there exist perfectly valid IReadOnlyList<T> implementations that cannot be wrapped in ReadOnlySpan<T>, such as implementations that compute their output on-the-fly.
I don't this this would be a constraint on the type; as it wouldn't allow you to use a non-ref struct or class in the method; which I assume isn't the desire. Rather prevent boxing to the heap; so its more of a promise of what the method/class will do with the type; rather than constraining what the type can be.
Perhaps to this end a modifier on the type would be more useful (like out and in for variance).
So ref interface? Meaning the interface cannot be boxed; but you can still apply it to classes and non-ref structs?
public ref interface IReadOnlyIterable<T>
{
int Length { get; }
readonly ref T this[int index] { get; }
}
public ref interface IIterable<T> : IReadOnlyIterable<T>
{
new ref T this[int index] { get; }
}
public ref struct RefStruct<TIterable, T> where TIterable : IIterable<T>
{
TIterable iterable; // ok, is ref struct
void ForEach(Action<T> action)
{
var roiterable = (IReadOnlyIterable<T>)iterable; // ok, in ref inheritance chain
var obj = (object)iterable; // compile error, can't cast to non reflike
for (var i = 0; i < roiterable.Length; i++)
{
action(roiterable[i]); // ok
}
}
}
public struct NonRefStruct<TIterable, T> where TIterable : IReadOnlyIterable<T>
{
ref TIterable iterable; // error is not-ref struct
void ForEach(TIterable iterable, Action<T> action)
{
for (var i = 0; i < iterable.Length; i++)
{
action(iterable[i]); // ok
}
}
}
@airbreather
I don't have data to support the following claim, but my intuition is that many "familiar APIs" have such "showstoppers".
Yes, on reflection I would be inclined to agree with your intuition. Even if the use of existing APIs is limited, I'd still be keen to see this capability, so that 'performance hungry' code isn't constrained in expressivity. Assuming it is technically feasible, my main concern is that the syntax could be misleading.
I see no fundamental reason why your Terminate<T> method should reject a reference-type argument that implements IAdvanceable.
If that is the case... then all the better! However, there are good reasons why the class and struct constraints exist, so I'm not sure that this is necessarily viable (again, I've not thought this through properly, but it feels too good to be true).
Can you please elaborate more on what you mean by this?
No, what I wrote doesn't make any sense... terribly embarrassing...
@benaadams
So ref interface? Meaning the interface cannot be boxed; but you can still apply it to classes and non-ref structs?
This is interesting. Can you provide a more substantial example (i.e. with concept implementation and consumption)? If I'm understanding correctly, it could allay any concerns about code which would be correct when handling a reference-type being illegal/wrong when handling what might be a struct or ref struct if clearly annotated where the type is consumed.
I could really use this feature as well in combination with https://github.com/dotnet/csharplang/issues/1147 for eliminating code redundancy in e.g. Span.Sort (https://github.com/dotnet/corefx/issues/15329) that I am working on. Using this I could define a generic type argument for the case when just sorting a single span or when there is an extra Span<TValue> of items in the same code with zero overhead. And this is hundreds of lines of code. I can give an example of what I would do if people are interested.
And I am sure lots of other possibilities will open up if we get it. C++/STL like code for example.
Do we really need new constraint for that though? By ref structs cannot implement interfaces anyway if i remember correctly (and object -derived methods are expected to throw anyway) so boxing seems to be non issue. I think relaxing rules for ref structs to allow to use them as type argument in methods and delegates ONLY, should suffice?
The benefit would be that existing net core API would instantly benefit from this change after making internal only changes to languages while constraint route would require changes in languages internally AND on surface then discovering ALL apis that would benefit from said constraint and apply this constraint accordingly
@BreyerW
I don't think so. An arbitrary generic method could attempt to box that ref struct by casting to an object, or assigning to a field on a reference type somewhere.
Bumping this issue again also with allowing ref struct to implement interfaces.
I don't this this would be a constraint on the type; as it wouldn't allow you to use a non-ref struct or class in the method; which I assume isn't the desire. Rather prevent boxing to the heap; so its more of a promise of what the method/class will do with the type; rather than constraining what the type can be.
Could we instead broaden the generic constraint? and use for example where ref, which would say that it will not allow storing the passed variable to the heap. It would be a constraint on its usage, not on the type itself. So this would work with any type, ref struct, struct or even class.
Not storing it on the heap means that boxing should not be allowed (so IL box is forbidden on such variable), meaning that interface boxing would never be possible (although that's unfortunate that duck typing for IEnumerable will not work out of the box...), meaning that we could allow ref struct to implement interface with such constraint.
The fact that we can't build abstraction around ref struct is restricting a lot their (generic) usage. We can't develop algorithm around them, that's really annoying.
For allowing interface calls on ref struct-constrained generic arguments we need to consider the interplay with the default interface methods feature. Consider:
interface IAdder
{
void Add(int x);
void PlusPlus() => Add(1);
}
ref struct Adder : IAdder
{
public int Value;
void IAdder.Add(int x)
{
Value += x;
}
}
class Program
{
static void Perform<T>(ref T val) where T : ref struct, IAdder
{
val.Add(1);
// 💣 Implicit boxing
val.PlusPlus();
}
static int Main()
{
Adder a = default;
Perform(ref a);
return a.Value;
}
}
I think we would currently throw while JITting the Perform method with an InvalidProgramException because JIT would attempt to generate boxing code for the ref-like type.
Obvious way out would be to error out if the ref struct doesn't implement all interface methods, but that would mean after updating a NuGet package reference that defines the IAdder interface (and adds the PlusPlus method), we would get a build error, completely defeating the purpose of the default interface methods feature (the purpose being allowing library authors to add methods to their interfaces without breaking consumer code).
We would probably need to annotate the interface in a way that disallows default interface methods for the interface.
Shouldn't we disallow explicit interface implementation for ref struct anyway? (as they are only available through interface casting which is not compatible with ref struct)
Shouldn't we disallow explicit interface implementation for ref struct anyway? (as they are only available through interface casting which is not compatible with ref struct)
They're callable without boxing through the constrained. (callvirt) IL instruction prefix, which is what the C# compiler emits in the Perform method above. For the purposes of interface dispatch within the Perform method above it doesn't matter whether the interface method is implemented implicitly or explicitly. What matters is where the resolution ends up going - whether to an instance method on a value type (where the this pointer is a byref), or whether to an instance method on an interface (where the this points to an object).
I see. So it means that default interface method on structs will always end-up into an implicit boxing , until there is a plan for the JIT to reify the default interface method at some point...
Ok, not sure I want default interface method anymore... 😅
until there is a plan for the JIT to reify the default interface method at some point
Removing the boxing later would be observable, and a breaking change.
Removing the boxing later would be observable, and a breaking change.
Yep unfortunately.
I would probably prefer default interface to be renamed to trait and keep normal interface as interface. We would then disallow a ref struct if it implements a trait instead of an interface.
I'd very much like this constraint added. It should not enforce that what you pass in is a ref struct, but only that your usage of the type is compatible with ref structs.
My goal is to have heterogeneous lookup for our keyed collections, like so:
class Dictionary<TKey, TValue>
{
bool TryGetValue<TOtherKey>(TOtherKey key, IEqualityComparer<TKey, TOtherKey> comparer, out TValue value) where TOtherKey : ref struct;
}
var stringMap = new Dictionary<string,int>();
var key = new ReadOnlySpan<char>(...);
var spanStringComparer = ...;
if(!stringMap.TryGetValue(key, spanStringComparer, out int foo))
{
stringMap.Add(new string(key), 0);
}
Given the focus we have now on in-situ parsing, where we try to reuse buffers rather than copying them, this will enable reducing allocations.
It might make more sense to make this a modifier of the generic parameter itself, not a constraint:
public ref struct Span<ref T>
{
}
// or
public ref struct Span<refable T>
{
}
This alleviates the maybeRefStructButMaybeNot problem that @airbreather mentioned: generic parameter modifiers (covariance and contravariance) have always been constraints that a types/delegates imposes on it itself. This is seemingly the correct place to put this.
As an aside: there is still an argument for where: maybeRefStructButMaybeNot because it could be seen as a variant of unmanaged (which is basically definitelyRefStruct).
While we're at it.... could we perhaps allow pointer types to be passed when T is constrained to be ref struct?
https://github.com/dotnet/runtime/issues/13627 suggests that the only issue with allowing pointer types in generic is issue with boxing (as the runtime currently does not allow boxing pointers). Since ref structs disallow boxing of instances I think it should be okay to allow doing so from the language perspective.
While it does not solve all the problems (pointers are allowed on heap/regular fields, whereas ref structs can't) this certainly would solve some of the problems.
It does look like it'll still require changes in the runtime / IL spec since ECMA-335 disallows pointer types in type parameter, so it might be just better to take dotnet/runtime#13627's approach though.
While we're at it.... could we perhaps allow pointer types to be passed when T is constrained to be ref struct?
Aside: you can do that with the unmanaged constraint (struct that contains no references)
public unsafe void M<T>(T* value)
where T : unmanaged
{
}
Not quite. I meant something like Example<int*> where the method definition is void Example<T>(T value).
Not quite. I meant something like Example<int*> where the method definition is void Example<T>(T value).
You can't, because pointers are not objects (unlike other primitives and structs), so they can't be boxed: object Example<T>(T value) { return (object)value; } is just not possible with a pointer.
Not quite. I meant something like
Example<int*>where the method definition isvoid Example<T>(T value).
What could you do with it in that situation? int* as a vague T would have no methods and no operations.
With Example<T>(T* value) where T : unmanaged you can do pointer arithmetic, casting and derefencing.
You can't, because pointers are not objects (unlike other primitives and structs), so they can't be boxed:
object Example<T>(T value) { return (object)value; }is just not possible with a pointer.
Which is why I asked if we could make it possible with where T : ref struct as the constraint would disallow boxing.
What could you do with it in that situation?
For example, basic generic collection types e.g. types under System.Collections.Generic can't be used because T can't be pointers, even though there would be no problem for doing so since they're effectively just special numbers (i.e. IntPtr).
Note that in this specific example allowing pointers when T is constrained to be ref struct is not going to help (System.Collections.Generic allows any types), but IMO allowing pointer types to be freely used in unconstrained generic can help simplify certain codes around the use of pointers.
...Maybe it's a bit backwards to discuss about this here. Probably a separate issue for this would be a better idea.
We're going to be doing some feature triage on Monday. I'm somewhat interested in championing this proposal, as well as a sibling proposal (that hasn't yet been opened as far as I can tell, I'll open one later) that would allow ref structs to implement interfaces, just never be boxed or converted to that interface. This would theoretically allow you to write generic code that operates on a ref struct where you have constrained that struct to an interface type, so you can do more than just treat it as an opaque holder of data. IE, something like:
public string M<T>(T t) where T : ref struct, Formattable
{
return Formatter.Format(Resources.Message, t.Args);
}
@333fred
as well as a sibling proposal (that hasn't yet been opened as far as I can tell, I'll open one later) that would allow ref structs to implement interfaces, just never be boxed or converted to that interface.
Is #2975 what you're looking for?
Yep, something like that. I had been thinking about the DIM scenario there, and a ref interface is an interesting way to handle it. I'm not sure how acceptable it would be, given that you'd have to introduce an entirely new set of interfaces, but it's an idea to consider.
I'm not sure that's possible:
Consider the following example that must be legal:
public interface I { int P { get; } }
public int M<T>(T t) where T : ref struct, I => t.P;
And this which must be illegal:
public interface I { int P => this.GetHashCode(); }
public int M<T>(T t) where T : ref struct, I => t.P;
That would make adding a DIM a breaking change, which would defeat the purpose of DIMs.