csharplang
csharplang copied to clipboard
dotnet
Champion: fixed-sized buffers
Introduce a pattern that would allow types to participate in fixed statements.
- [x] Proposal added (https://github.com/dotnet/csharplang/blob/master/proposals/fixed-sized-buffers.md)
- [ ] Discussed in LDM
- [ ] Decision in LDM
- [ ] Finalized (done, rejected, inactive)
- [ ] Spec'ed
LDM history:
- briefly discussed in LDM on 2/7/2018 with some interest, identifying champion.
- https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-04-03.md#fixed-size-buffers
- https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-04-10.md
- https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-05-01.md#fixed-size-buffers
- https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-05-03.md#inline-arrays
- https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-05-17.md#inline-arrays
Tagging @VSadov @jaredpar
public fixed DXGI_RGB GammaCurve[1025];
// ...
DXGI_RGB my_rgb;
How is this different from...
[StructLayout(LayoutKind.Explicit, Size = 1025)]
public struct DXGI_RGB { }
// ...
DXGI_RGB my_rgb;
The latter is a struct which is 1025 bytes in size.
The former is an inline array of 1025 elements, where each element is a DXGI_RGB struct instance.
Shouldn't this
public fixed DXGI_RGB GammaCurve[1025];
be
public fixed DXGI_RGB[1025] GammaCurve;
The array thingy [] in C# always goes after the type name, not the variable name like in C/C++ so it looks kinda wrong to have it the other way around here. Unless I'm misunderstanding some part of this syntax 🤷♀️
The array thingy [] in C# always goes after the type name, not the variable name like in C/C++ so it looks kinda wrong to have it the other way around here. Unless I'm misunderstanding some part of this syntax
No, fixed-size buffers use this syntax to distinguish them from standard fields. GammaCurve is a fixed-size buffer field, not a field having an array type. DXGI_RGB[1025] is not a valid type name.
However, since a special type is always generated for such a field, why not introduce a new special type syntax fixed T[size] that refers to this type?
public static fixed int[5] GetValues() => return default(fixed int[5]);
This is effectively producing a by-val array. Then the syntax fixed DXGI_RGB[1025] GammaCurve; would be reasonable and valid.
Yes. Fixed-size buffers are an existing language syntax (which this proposal is extending) and are different from normal arrays (which are heap allocated and tracked by the GC). The syntax difference ensures the two features (arrays and fixed sized buffers) can be differentiated and updated independently without worrying about possible ambiguities.
However, since a special type is always generated for such a field, why not introduce a new special type syntax fixed T[size] that refers to this type?
@IllidanS4, see ref returns and Span<T>. However, that requires a stackalloc to allow use within functions and has various limitations to ensure that you aren't accessing memory from a stack frame that no longer exists.
No, there is a difference between my idea and Span<T>. Span<T> is a reference to an arbitrary location in the memory, but fixed T[size] is a by-value array. Moving a value of this type would move the whole array. In essence, fixed T[2] is like (T, T), fixed T[3] is like (T, T, T) and so on.
That would be a completely separate/distinct feature and would need its own proposal.
How insane would it be to "just" allow consumers to use the existing fixed stuff as Span<T>?
Picturing...
struct MyStruct
{
public fixed ulong Values[4];
}
class Program
{
MyStruct s = default(MyStruct);
Span<ulong> values = s.Values;
Console.WriteLine(values.Length); // 4
}
Seems like "all it would take" (heh) is an attribute that tells the compiler the size of the field, and then the offset to the start of the buffer from the start of the struct could be either derived from metadata if sequential / explicit layout or computed live if auto layout (maybe at JIT time? I don't know how this one would work, sorry)... combine a ref to the the struct instance itself, the offset to start of fixed-size buffer, and the length of that fixed-size buffer (from the attribute), and that's enough to make what I think is a perfectly safe Span<T>.
Is this insane?
Hmm, to answer my own question, without runtime support, that (edit: "that" = just exposing an easy safe way for callers to get Span<T> over the existing fixed-size buffer stuff) doesn't feel like enough to support the "any element type" part of the proposal, since if the buffer contains reference types, the GC would need to see each element of the fixed-size buffer as a separate reference to track. So it seems like there would still be a need to do something like the transformation in the current proposal, at least for reference types.
Right?
No, because there is nothing unsafe in the layout of the actual compiler-generated struct. Each element is represented by a separate field with the correct type, and since the whole struct is a field of the containing type, the runtime has perfect knowledge of all the references which may be stored inside.
No, because there is nothing unsafe in the layout of the actual compiler-generated struct. Each element is represented by a separate field with the correct type, and since the whole struct is a field of the containing type, the runtime has perfect knowledge of all the references which may be stored inside.
Sorry, my comment was a bit ambiguous. I was positing a problem with my own Span<T> idea w.r.t. reference types and identifying this as a reason that we would still need something like the current proposal's compiler-generated struct with separate fields.
A lot of C++ libraries have support for specifying the size of fixed arrays using template arguments. The fixed arrays are still allocated on the stack, but the size is specified at compile time. For instance, the armadillo library has the mat::fixed<n_rows, n_cols> class. (Reference) Having this capability in C# would be very useful for linear algebra applications.
A lot of C++ libraries have support for specifying the size of fixed arrays using template arguments.
@johnwason See [Proposal] Const blittable parameter as a generic type parameter - constexpr used for blittable type parametrization
I've played around with working around these limitations in order to operate directly on blittable data structures. It becomes more work to maintain once you start nesting structures or arrays of structures, but it's functional.
Simple example:
public class BlittableStructReader
{
private Memory<byte> _data;
private ref Layout Data => ref Unsafe.As<byte, Layout>(ref _data.Span[0]);
public Span<byte> FixedArray1 => _data.Span.Slice(0, 0x100);
public Span<byte> FixedArray2 => _data.Span.Slice(0x100, 0x100);
public uint Field1
{
get => Data.Field1;
set => Data.Field1 = value;
}
public uint Field2
{
get => Data.Field2;
set => Data.Field2 = value;
}
[StructLayout(LayoutKind.Explicit, Size = 0x800)]
private struct Layout
{
[FieldOffset(0x200)] public uint Field1;
[FieldOffset(0x204)] public byte Field2;
}
}
@gafter, what part of this was done? This proposal is for allowing fixed MyStruct name[size];, which is still illegal.
This was not done in 7.3, otherwise I would have updated the championed issue title to say this shipped.
My mistake. This is listed as shipped in 7.3 in https://github.com/dotnet/roslyn/blob/master/docs/Language%20Feature%20Status.md under "Custom Fixed".
@gafter As a guess, that doc might be pointing to the wrong issue then. I think "Custom Fixed" might be referring to being able to support fixed statements for custom types by implementing an appropriate GetPinnableReference() method.
The LDM has pushed this out. It requires more design work and is not near the top of our priorities.
@gafter, could you clarify if this requires more work on the proposal and if that is something that I can/should update?
Or is this more design on the language side and is something LDM will eventually get to?
The only design approach that we have given the current CLR specification requires a lot of complex code to be generated to implement this. Getting CLR support would permit a simpler implementation, but we do not want to risk our other language-support requests in this timeframe.
Thanks @gafter, so something like https://github.com/dotnet/coreclr/issues/23408 would help unblock this, if my understanding is correct?
The only design approach that we have given the current CLR specification requires a lot of complex code to be generated to implement this.
Think I may have communicated this incorrectly. The code is complicated, possibly we could simplify, but really the issue is lifetime rules for ref struct. This feature requires us to change the rules a bit. Rather than doing it for this one specific feature we should include all of the other features which could benefit from updating ref struct lifetime rules. Basically I think we need to get to the bottom of that design first before we could tackle this specific instance of the problem.
This is a pretty important feature for us at Unity -- as we're trying to move more and more code into C#, being able to do high performance packed data types is important. Note that for our use case, what we need is fixed-size buffers of (unmanaged) structs. We don't need them to be usable in a safe context.
In other words, we would like the restriction on the types of fixed buffers that can be created to be relaxed from the limited set of types to be those types + any type that meets the unmanaged constraint. Would that restriction make this more acceptable and easier to implement in the shorter term?
struct MyElement {
public float3 pt1;
public float3 pt2;
}
unsafe struct MyStruct {
public fixed MyElement elements[16];
}
Been meaning to comment on this issue for a while, and was suddenly reminded of its existence after seeing talk of fixed buffers elsewhere.
This would also greatly help writing bindings.
This is one of my biggest use cases too. Which brings me to the question I wanted to ask: Would it be possible to also apply this to pointers?
For example: fixed StructFromLibrary* arrayOfPointers[8];
I suppose one could just do fixed IntPtr arrayOfPointers[8]; but that requires a bit more ceremony to actually use.
@vvuk I second this example. Have wanted this for years. This and the new function pointers would be epic additions to C#.
@gafter Many people need this feature, I think it should have a high priority.
