gdext icon indicating copy to clipboard operation
gdext copied to clipboard

`OutArray`, a possibly typed array.

Open Bromeon opened this issue 7 months ago • 1 comments

Quite a fundamental change to the way how we handle arrays, which should also fix #727. Still an early draft version and up for discussion.

This PR adds a third array type to the 2 existing ones:

  1. Array<T>, typed arrays
  2. VariantArray == Array<Variant>, untyped arrays
  3. OutArray, a possibly typed array

Out variance

"Out" here means out variance. In short, this means that such a type supports all operations where data flows out of the array, because those are covariant:

let a: Array<i32> = array![1, 2, 3]; 
let mut b: OutArray = a.into_out_array(); // loses static type info

let v: Variant = b.get(1); // works

However, it does not allow any data flow in to the array:

b.set(1, 5.to_variant()); // forbidden
b.set(1, "str".to_variant()); // forbidden

As you see in the 2nd example, this would otherwise ruin type safety. This is btw a problem that GDScript is fighting with.

It is important to understand that "out" does not mean "read". It's perfectly fine to perform write operations, as long as they don't involve data flow in to the array.

let v: Variant = b.pop(); // no problem
b.shuffle(); // no problem either

Engine APIs

One area where more research is needed are the Godot class/method APIs. So far, we have used either VariantArray or Array<T> in the signatures, and I think it has worked quite well. One commit changes VariantArray to OutArray, but I'm not yet sure if we should really do this.

Some thoughts:

  • OutArray is more flexible when you have to provide it, because you can now pass Array<T> (typed arrays) and convert them. Today, one has to use an untyped array.
    • For parameters to engine functions.
    • For return types in virtual functions.
  • However, OutArray is more limiting when receiving it, as one first needs to (fallibly) convert to VariantArray.
    • For return types of engine functions.
    • For parameters in virtual functions.
  • One option would be to choose OutArray when returning, and VariantArray when receiving.
  • It's not completely clear to me which guarantees Godot gives us today. If the signature is returning Array, could it technically point to a (runtime-)typed array or are we guaranteed that it's untyped?

Drawbacks

While it does solve some type-safety problems, an extra OutArray also brings quite a bit of mental overhead. It can also require conversions in more places. We should consider that GDScript doesn't have this, and few people are complaining about the covariance problem.

This feature could pull in more APIs just to be useful. Something like impl AsArrayArg or impl Into<OutArray>, similar to AsObjectArg, may be necessary. Then there's array!/varray! macros, etc.

As soon as Dictionaries become typed (which will happen), we'll have the same situation there, with both key/value variance. An OutDictionary would be the pendant.

A lighter way to address this would be to allow converting from Array<T> to VariantArray. If we do this, we can still panic on "in data flow" operations for VariantArray. This would have less strict guarantees, but might be more ergonomic for gamedev code.

Bromeon avatar Jul 21 '24 21:07 Bromeon