pandora

pandora copied to clipboard

Description

This PR modifies the Array type to encode the type of its values when it is serialized. I spent quite a lot of time trying to solve the issue I reported in #159, but could not do so without significantly altering the loading process.

So this is the alternative I am proposing.

When parsing serialized Array data, if it includes the new keys:

1. If the supplied settings have a type value, that type will be used to parse the values.
2. If no settings are supplied, the type value present in the stored data will be used, unless it is UndefinedType
3. If the stored type is Undefined and no settings are supplied, the values will be passed alone as-is
4. If the new keys are not present, the old (existing) logic is used.

When writing an Array type, a new format is used that includes the type of the values. If the value types are mixed, it will be "undefined". This should update old data to the new format when it's next saved.

Finally, I fixed an error that resulted from trying to look up a non-empty type name that wasn't a valid type.

Addressed issues

• closes https://github.com/bitbrain/pandora/issues/159

One more thing: could we set up an array like this in the demo project so we can see how it is stored in data.pandora?

Quick status update. Had to take a brief hiatus from my game project (and this issue by extension) due to other priorities.

Here's where things lay:

1. I believe the current state of the PR addresses the root issue, however
2. Due to the nature of the issue, existing serialized array properties for types that require special deserialization won't be properly deserialized, so the fixes implemented in this PR can't catch them. New properties created with this fix in place will be fine.
3. I am working on a way to try and catch these pre-fix properties and fix them the next time they're loaded with this fix in place. This will likely need to occur at a stage in the loading process when the PandoraProperty objects themselves are fully loaded as it will be necessary to read their array type setting and get the correct contents from each entity's override of the property.
4. Need to finish adding additional tests, particularly for point 3 if I can get that working.

@nkrisc did you manage to progress on this?

Since we're not passing back typed arrays (since the typing would require an inherited protocol which would still require casting the array anyway) I feel like it would be simpler to just write out the type into a sub-dictionary against each key. This has the nice effect of being upgrade path since it will just treat previous non-dict values as existing types so the current state is maintained but all new values are written as type dicts.

I'd be happy to take over this PR and flesh out my below pseudocode and add some tests. Will work on it tonight.

read_value

     if variant is Dictionary:
		var array = []
		var dict = variant as Dictionary

		for i in range(dict.size()):
			var value_dict = dict[str(i)]

			if value_dict is Dictionary:
				var value_type = value_dict["type"]
				var value = value_dict["value"]

				var type = PandoraPropertyType.lookup(value_type)
				if type is not null:
					value = type.read_value(value)
					array.append(value)
					continue
			
			array.append(value_dict)

write_value

var dict = {}
	if not array.is_empty():
			var types = PandoraPropertyType.get_all_types()
			var values_type : PandoraPropertyType
			for i in range(array.size()):
					var value = array[i]
					var value_type = UndefinedType.new()

					for type in types:
							if type.is_valid(value):
								value_type = type
								value = type.write_value(value)

					var value_dict = {
							"type": values_type.get_type_name()
							"value": value
					}

					dict[str(i)] = value_dict