godotengine
Add support for nullable static types in GDScript
Describe the project you are working on: This applies to many projects. This is an offshoot from https://github.com/godotengine/godot-proposals/issues/737, example use cases and other discussions are welcome.
Describe the problem or limitation you are having in your project:
Let's say you have a method that accepts a 2D position, which would look something like this:
func whatever(vec):
A problem with this is that there's no type safety, so the function could unexpectedly break if the passed-in value is not a Vector2. One option is to use static typing:
func whatever(vec: Vector2):
This works, and now it's not possible for users to, for example, pass in a Color or any other type that's invalid for this method. However, now you can't pass in null to mean N/A or similar.
Describe how this feature / enhancement will help you overcome this problem or limitation:
If GDScript's static typing system allowed specifying nullable types, we would be able to restrict the type to either a valid value or null. The presence of a valid value can then be detected simply by checking if it is not null, as non-null nullable typed values must be valid values.
Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:
My suggestion is to simply allow this by adding a question mark after the type name, which is the same syntax used in C#, Kotlin, and TypeScript. User code could look something like this:
func whatever(vec: Vector2?):
Describe implementation detail for your proposal (in code), if possible:
Aside from the above, I don't have any specific ideas on how it would be implemented.
However, I will add that we could expand this idea for engine methods. Many parts of Godot accept a specific type or null to mean invalid or N/A, or return a specific type or null when there is nothing else to return. For example, Plane's intersect methods return a Vector3 if an intersection was found, or null if no intersection was found. Nullable static typing could essentially self-document such methods by showing that the return type is Vector3? instead of Vector3.
If this enhancement will not be used often, can it be worked around with a few lines of script?: The only option is to not use static typing if you need the variable to be nullable.
Is there a reason why this should be core and not an add-on in the asset library?: Yes, because it would be part of GDScript.
You could pass in Vector2.ZERO and check for it, as you would need a check either way. This is also safer, as null "could be the biggest mistake in the history of computing". I'm also kinda scared that the whole type system will get even more complex, which could scare away new users from gdscript. (I know the complex syntax of java scared me away)
Note that with nullable types we could require people to handle nulls explicitly, similar to how kotlin does it. So:
func whatever(v: Vector2?):
print(v.x) # Warning or error: v could be null
print(v?.x) # (if there is a null-checked dot operator) Prints null if v is null
if v != null: print(v.x) # Does not print if v is null
print(v.x if v != null else 42) # Prints 42 if v is null
Here's some current use cases of mine (if I understand the proposal correctly):
# Use custom RandomNumberGenerator, or the global one:
var ri = RNG.randi() if RNG != null else randi()
# Adjust impact sound volume by relative velocity
func _adjust_volume_to_velocity(velocity_override = null):
var strength = max_strength
if velocity_override:
strength = velocity_override.length()
# configure db ...
You could pass in Vector2.ZERO and check for it
The logic could fail exactly at Vector2.ZERO position if you depend on it here (for tile-based levels this could happen more often I believe):
var attach_pos = get_attach_point_position()
if attach_pos == null: # not attached to anything
return linear_velocity
The logic could fail exactly at
Vector2.ZEROposition if you depend on it here (for tile-based levels this could happen for often I believe):
I often use a getter (which returns a "safe" boolean) to check for stuff, for example:
if not is_attached(): # not attached to anything
return linear_velocity
You could pass in Vector2.ZERO
@Jummit See the discussion in https://github.com/godotengine/godot/issues/32614 for why this doesn't work.
You could pass in Vector2.ZERO
@Jummit See the discussion in godotengine/godot#32614 for why this doesn't work.
It works, just not everywhere. Also, OP makes a good point:
A better result might be to return Vector3(float.NaN, float.NaN, float.NaN).
I'm not proposing to use this in core functions, but in personal projects this is a good way to avoid null.
It works, just not everywhere.
So it's not a good solution.
[from your link against the null value] For example, if a function returns a positive int number, then upon failure to calculate the desired return value, the function could return -1 instead of null to signify the error.
I'm not agreed with that, basically you return a valid value when you precisely want to warn that something failed. This approach lack of consistency (the value sometime will be -1, some time an empty string, etc.) and limited: what do you do when the function can return the whole range of an int?
As far I know, the cleanest solution for the return type case, it's to raise exceptions that must be handled by the region of the code that call the function. But error handling isn't available in GDScript and don't solve the case described by @aaronfranke
[from you link against the null value] which means that the compiler can't warn us about mistakes caused by null at compile time. Which in turn means that null exceptions will only be visible at runtime.
At least there is an error visible somewhere, if you return a valid value and forgot to handle it, there will be the worst case scenario for a bug: no hint.
I find this proposal essential in a typed environnement, especially for the type returned by a fonction, it also useful for parameters like @aaronfranke suggested, to keep all the logic of a function, inside it. The alternative being to do pre-checks before to call the function, and so have a part of the function's logic outside it.
Moreover, it becomes more important in conjunction of #173, where you will be able to force the typing, without this proposal, the solution for returning a "error/null" value from a fonction will be to disable the type system with the any keyword...
If we want to get rid of the null and have a safe/clean approach, I was thinking about the Optional object in Java (how java solved the null issue)
If we implemented something similar in a less verbose/python like way, that a very good solution I think:
whatever(Optional(vec)) # or whatever(Opt(vec))
whatever(Optional.EMPTY) # or whatever(Opt.EMPTY)
func whatever(vec: Vector2?):
var result := vec.get() if vec.is_present else Vector2.INF
@fab918 The only practical difference between your proposal and mine is that you're wrapping the value in an object and using methods and properties instead of using a nullable type. I'm generally opposed to wrapper objects and this just adds complexity to something that would likely be easier if we mimic the way other languages do it.
For a nullable type, I would remove .get() and replace .is_present with != null:
func whatever(vec: Vector2?):
var result: Vector2 = vec if vec != null else Vector2.INF
@aaronfranke I'm fine with your proposal.
But if the problem with your solution for some , it's the usage of null as @Jummit has pointed out, because "it could be the biggest mistake in the history of computing". My proposal is an alternative that protects users from runtime error that a forgot null value can produce (more cleaner but more verbose).
Anyways, no matter the implementation, I think this feature is essential to fully embrace the typed GDScript, especially with #173
The Kotlin style ?. Syntax is very nice to work with, but it does lead you down a rabbit hole of dealing with nulls that new people might find wired. ?.let {} in Kotlin, gaurd in Swift.
If you are going to do strict null typing as part of the type system that is.
That being said, I too think this proposal is essential for GDscript to be and to be used in larger projects.
Yeah, this is quite a serious issue, forcing me very often to skip types.
From the proposed solutions (assuming no more improvements to the type system are done, like type narrowing), the != null is weaker than .get(). While both are not great, at least .get() forces user to unpack the value and crash on the exact spot where "null" should have been checked. If user forgets to use != null, that value may propagate and can cause crash in unrelated parts of code.
I'm generally opposed to wrapper objects and this just adds complexity to something that would likely be easier if we mimic the way other languages do it.
This way it's done for example in Scala, Java, Haskell, OCaml, Reason, PureScript and looking at wiki in many more languages I never used. So it's definitely something other languages are doing too. Sum types are stronger (type-wise), because they force user to handle (test) for an empty value.
!= null could work if implemented like in TypeScript (type narrowing). For example accessing position of Node2D? (Node2D | null in TS) would not compile, because null doesn't have such property. But if you do a check first, the type get narrowed (inside if block) to Node2D and you can access the property if a != null: a.position = x.
I personally would prefer != null with type-narrowing, but if type-narrowing wouldn't be implemented, then I would go with wrapper object.
I am not sure of the scope of this proposal, but I would like to see supported it everywhere, not just function arguments or return type.
I've been thinking about this and what I have in mind is that all variable will be nullable by default. So even if you type as a Vector2 or int, you can get a null instead. This will decrease safety in general but it will be much simpler for fast prototyping, which is the main focus of GDScript. I will still keep default values as non-null (except for objects), so it's less of a hassle if you don't initialize variables (even though you always should).
Especially because internally the engine is mostly okay of taking null as values for the primitive types in functions and properties.
To counter that, we could introduce non-nullable hints, so it forces the value to never be null, even on objects. Something like this:
var non_nullable: int! = 0
var always_object: Node! = Node.new()
Using ! for non-nullable as opposed to using ? for nullables.
This will give an error if you try to set null, both at compile time and runtime if reaches that point.
@vnen I very much dislike that idea. The amount of projects I've seen currently using static types everywhere shows that the use cases for non-nullable static types are extremely common, with this proposal only relevant for a small number of use cases. If your proposal was implemented, many people would fill their projects with !, or be left with bugs when nulls are passed.
I also think your proposal goes against the design goal that GDScript should be hard to do wrong. If a user specifies : Vector3, I think it's a bad idea to also allow null implicitly. It would be very easy to design a function that can't handle nulls and you forget to include the !.
I think I would actually rather have nothing implemented than !, since it would encumber GDScript with either many ! characters or many bugs, all for the few use cases where you want nullability, and in those cases there is already a decent work-around (no type hints).
There is also value in the simple fact that ? is familiar to programmers from many languages.
Hi @vnen. First, thanks for all your good works, really amazing. It’s hard to deal with devs, and harder to make choices, so I wanted to support you
Here is my 2 cents, hope that can help.
I thought about the right balance between fast prototyping and robust code. I often saw this debate here and on discord, I ended to think the best is to avoid to compromise to ultimately satisfy no one. Instead, stick on 2 ways with opposite expectations:
- For those who seek a fast prototyping language, don’t use types. Make sense, they are probably not those who will be bothered with them.
- For those who seek a robust application and/or the maximum performance, use types, but it should be the most restrictive as possible to actually be robust.
So in this spirit, I will avoid decrease safety and stick on classic nullables approach in conjunction of #173 (seems directly related to this proposal to fully be able to embrace the type system). In addition, the use of nullable is rather standardized contrary to your proposal.
The type narrowing proposed by @mnn is the cherry on the cake but no idea of the implication on the implementation.
Here is my workaround, that I think works in any case:
const NO_CHARACTER := Character.new()
func create_character(name : String) -> Character:
if name.empty():
return NO_CHARACTER
return Character.new(name)
func start_game(player_name : String) -> void:
var player_character := create_character(player_name)
if player_character == NO_CHARACTER:
return
You end up with some new constants, but I think it makes it more readable than using null.
@vnen
Going off what @aaronfranke mentioned, adding ! to Godot 4.0 would break backwards compatibility more than I think is warranted. I think most projects assume that the base types are not null. Putting backwards compatibility aside for now (since 4.0 doesn't prioritize support with 3.2), I think we'd see tons of int!, Vector2!, String! everywhere. Part of the usefulness of these types is that they aren't nullable by default - but that's my opinion.
On another note, what do people think about adding more null conditions in general?
Currently, to null check you need to do something like
var x = y if y != null else 10
but I think this operation is common enough that is should be allowed to be shortened to:
var x = y ?? 10.
@Lucrecious The first line you posted doesn't behave as you may expect, since if y will be false not only for null, but also for 0 and 0.0 and false and Vector2(0, 0) etc. If you want a null check, it would have to be y if y != null else 10
While I do like wrapper types, coming from powerful type systems like Rust that use that kind of pattern, an Optional really isn't that useful in GDScript. The reason is because there's no way to specify (and therefore check) the type of the inner value, much like Arrays. It's not much better than removing the type from the function signature altogether. Sure, you have another pattern to check whether the value is null, but then you've still only guaranteed that it's a non-null Variant.
To add to my previous comment: If we were to add a syntax that allows generic types like Optional<int> or Optional[int], I still don't think Optional is the best/cleanest choice.
I'm not convinced that null is a bad thing here. Yes, it can be hazardous in places where null is accepted as a value for any type, however that's not the case here. The static typing we have already rejects null values (the types are already "non-null", to put it another way). Given the existence of null, the much cleaner solution would be to extend the type syntax and add a "nullable" indicator, like int? or ?int, which allows null values to be passed in addition to values of the given type.
I have an example where being able to define null or Vector2 as a return type would be necessary (unless it's changed internally how TileSets work):
tile_set.cpp:594
Vector2 TileSet::autotile_get_subtile_for_bitmask(int p_id, uint16_t p_bitmask, const Node *p_tilemap_node, const Vector2 &p_tile_location) {
ERR_FAIL_COND_V(!tile_map.has(p_id), Vector2());
//First try to forward selection to script
if (p_tilemap_node->get_class_name() == "TileMap") {
if (get_script_instance() != nullptr) {
if (get_script_instance()->has_method("_forward_subtile_selection")) {
Variant ret = get_script_instance()->call("_forward_subtile_selection", p_id, p_bitmask, p_tilemap_node, p_tile_location);
if (ret.get_type() == Variant::VECTOR2) {
return ret;
}
}
}
}
Technically _forward_subtile_selection is allowed to return a variant, and only acts on it if the type is a Vector2, which is great, but if you statically type it as returning a Vector2, there is no way around this.
func _forward_subtile_selection(autotile_id: int, bitmask: int, tilemap: Object, tile_location: Vector2) -> Vector2:
return null
This errors with The returned value type (null) doesn't match the function return type (Vector2). So either you must leave off the return type, or completely reimplement autotiling code. Being able to define:
func _forward_subtile_selection(autotile_id: int, bitmask: int, tilemap: Object, tile_location: Vector2) -> ?Vector2:
return null
just like in many languages would get around the problem. For reference, Vector2.ZERO is a valid value, so that can't be used. I suppose a negative vector may be fine, but it would be better if we had either union types or nullable types.
On Twitter, @vnen wrote:
My suggestion for "default nullable" was more to be consistent because object types always have to be nullable.
See https://twitter.com/vnen/status/1290697381470711810
So, my question is: why do object types always have to be nullable in GDScript?
There are other languages where objects type hints are non-nullable by default, for example, in PHP:
class MyClass { }
function null_not_allowed(MyClass $variable) { }
function null_allowed(?MyClass $variable) { }
// Works.
null_allowed(null);
// Fails.
null_not_allowed(null);
Why can't GDScript work in the same way as PHP, where objects type hints are non-nullable by default?
Working with the optional typing in PHP (where the type hints are non-nullable by default) is honestly really great. One of my least favorite things about working in C# (7.0 or earlier) is that all object type hints are nullable, so you have to constantly be accounting for one of your method parameters being null and/or deal with the inevitable crashes. BTW, this is generally considered a mistake in the design of C# (article by the lead designer of C#) which is apparently fixed in C# 8.0 (although, I haven't personally used 8.0 yet).
Could GDScript perhaps avoid having this design mistake? :-)
(As an aside, when optional type hinting was added to PHP, there were only object types and they were only non-nullable. Primitive type hints and nullable type hints were only added later.)
Doing this will probably significantly affect the VM ability to optimize the code and the complexity of the parser because, internally, things are either typed or not and this is a grey area. Additionally, I don't like the feature myself, I think it makes code more difficult to understand (you can't take things for granted when reading it, you need to add an extra state for variables) and does not really add a lot of value for the large cost it implies implementation wise.
@reduz Thanks for responding!
does not really add a lot of value for the large cost it implies implementation wise.
Type hints provide two possible values:
- Allowing the VM to optimize performance based on the types (not done in Godot 3.x, but planned for Godot 4.x)
- Allowing the developer to depend on type safety so that you can write code with the assumption that a particular method will always return a certain type, or that your method will be called with parameters of a certain type, to prevent unexpected crashes.
I don't know the internals of the compiler and the VM the way that you and @vnen do, but it sounds like you're arguing that having nullable types isn't worth implementing because it wouldn't be able to easily provide the performance value.
However, I'd personally be happy with nullable types sacrificing performance (ie. running the same as untyped code) while still providing the safety of type hints. If nullable type hints acted internally just like untyped code, but simply changed the type checks that happen with checking the parameters and return values on a function call, that'd be absolutely fine. :-)
In fact, if we imagine a world where in Godot 4.x type hinted code runs with faster performance, but doesn't have nullable types, then the only option we have when we need to return or accept null is to drop type hints completely, which means we lose both performance and safety. If there was a way to keep safety (and a nice developer experience), even at the expense of performance, that's still an improvement from where we're starting from (in this imaginary world).
So, my question is: why do object types always have to be nullable in GDScript?
There are other languages where objects type hints are non-nullable by default [...]
You have to remember that GDScript is made in the context of Godot. It isn't always possible to replicate what you see in other languages.
In the context of Godot, there are many APIs that will return a null instead of an object, like when some error happens. If you try to get_node() and it doesn't exist, you get null as a result. Changing this requires changing a lot of the behavior in the internal API, even when GDScript is not involved. While this isn't impossible, it is a great amount of work, so you to be very convincing on the advantages of it.
This is generally not the case for built-in types. If something says it returns a Vector2, it always return a Vector2 (I know there might be some exceptions but they are likely some mistake and should be solved, and it's a much smaller amount of work).
Regarding nullable types, we do have to consider all possible implications in the VM, type-checking, optimizer, etc. to see if it brings enough advantage to be justified.
If we do add nullable types, we could say that object types are non-nullable and mark all of the API as being nullable when it comes to objects. That would soon get in your way and a lot of extra checks would be needed in your code, even in the cases where you can be confident. It would be a hassle to write code like this (unless you declare all of your variables to be nullable, but then it misses the point of the feature).
That's why I mentioned the addition of a non-nullable mark: then you could add those checks when relevant and be safer in the rest of the code when that's important. People say that then it would litter the code with those marks, but I have a feeling it would be the opposite: if you add a nullable mark then you would use that a lot, because you would be kind of forced given the internal APIs.
Again, we need to consider the implications of adding such notation. Might not be as beneficial as people think.
@vnen thank you for explaining the stakes, you should always do it, now I understand why you propose this "unexpected" solution.
so you to be very convincing on the advantages of it.
Typing's users want a fully typed environment, currently we have 2 bigs problems that make impossible to have a fully typed project:
- No nullable solution, in particular for functions (arguments and return)
- Any type with an option to force typing in the project scripts.
In the context of Godot, there are many APIs that will return a null instead of an object, like when some error happens. If you try to get_node() and it doesn't exist, you get null as a result. ...That would soon get in your way and a lot of extra checks would be needed in your code
that's why other languages use error handling when an error happens (get_node not found) instead of null. You can implement extra logic if you want to handle that case in a script or let the error for the output console. I think is the cleanest approach.
But I/we understand that this kind of solution will involve too much work (for now?) or maybe performance/design issue. That lead us for 2 choices:
- use your proposal:
func foo(variable : Vector2!): - let the typing system allow
nulland wait for an implementation in the future (but at least unblock typing's users):
var test : Vector2 = null
foo(test)
func foo(a : Vector2) : -> Vector2:
return null
For my point of view, you proposal is fine now we know that's not just a decision based on syntax preference. I prefer you spent time for performances like AOT (is there a proposal for that?) instead of turn the ! to ?.
Typing's users want a fully typed environment, currently we have 2 bigs problems that make impossible to have a fully typed project:
- No nullable solution, in particular for functions (arguments and return)
- Any type with an option to force typing in the project script.
I agree that these are the big problems with GDScript if users want a fully typed environment.
I'm still confused as to way this is such a difficult endeavor?
Isn't it possible to simply treat a nullable Vector2 as its own type? Maybe under a wrapper class, like for weakref and automatically generate corresponding nullable types to all the value types like Vector2, Transform2D, etc. i.e. a type can be NullableVector2 and then we can use Vector? as syntactic sugar. NullableVector2 ~can simply be a ref with functions like is_null, in addition to reflecting~ can reflect all Vector2 methods and return true/false on equality checks with null.
# if we have
var vec: Vector2
var n_vec: Vector2?
# then
type(vec) != type(n_vec)
not n_vec is Vector2
n_vec is Vector2?
# as operator always returns a nullable type
var other_vec: Vector2? = n_vec as Vector2
var other_vec2: Vector2 = n_vec as Vector2 # works like before, if n_vec were a variant, error when it's null
Would something like this be a possible solution?
I'm mostly just lurking here and following along, but the point about nullible types requiring a bunch of extra code from the user can be mitigated quite a bit with things like the null coalescing operator foo.?bar() # only calls bar if foo isn't null
That would be a really nice operator to have even if we don't go the full nullable types route.
Kotlin has a very strict and full featured type system incorporating null but they've made it very nice to work with even though you are forced to deal with nullability everywhere.