Possibility for custom serializer implementation

[scatterlogical]

Would it be possible to provide a mechanism to implement our own serializers? I want to be able to do a binary serialization of entities, via MemoryPack, which I'm already using for other elements in my game, but I'm unsure of the complexities of pulling and pushing stuff from the EntityStore.

[friflo]

There is a similar request at https://github.com/friflo/Friflo.Engine.ECS/discussions/49

At the bottom of this discussion after a short explenation how to write a custom serializer.

A custom generic Writer can use Entity.Components to get entity component data. These components can be written as text or binary.

A custom generic Reader read binary / text data. Than create an entity with store.CreateEntity(). The deserialized data can be added using [Entity] (https://github.com/friflo/Friflo.Engine-docs/blob/main/api/Entity.md) or EntityUtils methods.

[scatterlogical]

Oh ok, so there's no nuances to reconstructing an entity and components beyond just creating them and setting the members? Sorry I'm used to dealing with serialization of entities with unreal engine, and that is a nightmare of pitfalls 😆

[jhimes144]

@friflo

You mention in that discussion that you do not believe there to be much demand for this feature. I would lean to disagree. Serializing the entity tree is a must for any multiplayer game that wishes to replicate the entity tree upon world load.

For anyone still trying to do this. I created this code using message pack. It passes my tests and usage, but not very robust. It's not what I would consider "real-time performant" for millions of entities, but its purpose is to save game state and serialize and send the world to newly connected players.

One thing to note is that .Value on the component collection for entities is marked as deprecated, but I have no idea how you would be able to get a list of components without calling GetComponent<T> with all the types known, which would be insanely messy.

readonly IBinarySerializer _serializer;
public EntityStore Store { get; }

public byte[] SerializeBinary()
{
    var entityPacks = new List<EntityPackage>();

    foreach (var entity in Store.Entities)
    {
                if (!entity.Parent.IsNull)
                {
                    continue;
                }

        entityPacks.Add(getPackage(entity));
    }

    return _serializer.Serialize(entityPacks);
}

public void DeserializePopulateFromBinary(ReadOnlyMemory<byte> data)
{
    var entityPacks = _serializer.Deserialize<List<EntityPackage>>(data);

    foreach (var pack in entityPacks)
    {
        addPackage(pack);
    }
}

Entity addPackage(EntityPackage entityPack)
{
    var entity = Store.CreateEntity();
    var schema = EntityStore.GetEntitySchema();

    foreach (var childEntityPack in entityPack.ChildEntities)
    {
        var childEntity = addPackage(childEntityPack);
        entity.AddChild(childEntity);
    }

    foreach (var componentObj in (object[])entityPack.Components)
    {
        var component = (IComponent)componentObj;

        if (schema.SchemaTypeByKey.TryGetValue(component.GetType().Name, out var schemaType)
           && schemaType is ComponentType componentType)
        {
            EntityUtils.AddEntityComponentValue(entity, componentType, component);
        }
    }

    return entity;
}

EntityPackage getPackage(Entity entity)
{
    var pack = new EntityPackage
    {
        ChildEntities = entity.ChildEntities.Select(getPackage).ToArray(),
        Components = entity.Components.Where(c => c.Value is not TreeNode).Select(c => c.Value).ToArray() // c.Value is marked deprecated
    };

    return pack;
}

    /// <summary>
    /// This class is used only to package entities before they are serialized and sent over
    /// network.
    /// </summary>
    [MessagePackObject]
    public class EntityPackage
    {
        [Key(0)]
        public EntityPackage[] ChildEntities;

        [Key(1)]
        [MessagePackFormatter(typeof(TypelessFormatter))]
        public object Components;
    }

public interface IBinarySerializer
{
    object Deserialize(Type type, ReadOnlyMemory<byte> data);
    T Deserialize<T>(AutoResizeReadBuffer buffer);
    T Deserialize<T>(ReadOnlyMemory<byte> data);
    void Serialize(IBufferWriter<byte> writer, object obj);
    byte[] Serialize(Type type, object data);
    byte[] Serialize<T>(T data);
}

[DependencySingletonPer]
class BinarySerializer : IBinarySerializer
{
    MessagePackSerializerOptions _options = MessagePackSerializerOptions.Standard
        .WithResolver(CompositeResolver.Create([TypelessFormatter.Instance], [StandardResolver.Instance]));

    public T Deserialize<T>(ReadOnlyMemory<byte> data)
    {
        return MessagePackSerializer.Deserialize<T>(data, _options);
    }

    public T Deserialize<T>(AutoResizeReadBuffer buffer)
    {
        var result = MessagePackSerializer.Deserialize<T>(buffer.PositionedDataMemory, _options, out var bytesRead);
        buffer.Advance(bytesRead);
        return result;
    }

    public object Deserialize(Type type, ReadOnlyMemory<byte> data)
    {
        return MessagePackSerializer.Deserialize(type, data, _options);
    }

    public void Serialize(IBufferWriter<byte> writer, object obj)
    {
        MessagePackSerializer.Serialize(writer, obj, _options);
    }

    public byte[] Serialize<T>(T data)
    {
        return MessagePackSerializer.Serialize(data, _options);
    }

    public byte[] Serialize(Type type, object data)
    {
        return MessagePackSerializer.Serialize(type, data, _options);
    }
}