Entitas
Entitas copied to clipboard
sschmid
The future of Entitas - Multithreading
The future of Entitas - Multithreading
This is sth I wanted to work on for a long time and I think this can be a game changer!
The elevator pitch:
Make Entitas ready for multithreading. Define best practises and workflows to enable everyone using Entitas to take full advantage of the given hardware and multiple processors.
Multithreading can be very hard and with Unity's limitation to work only on the main thread we lose a lot of potential! The ultimate goal is to simplify the workflow and turn multithreading into a no-brainer.
First results seem very promising
Feel free to retweet if you want to see this happening
https://twitter.com/entitas_csharp/status/840581685846634497
what is the main problem with multithreading in entitas? concurrent access? I mean, what needs to be changed/made to accomplish proper MT?
All common pitfalls when using MT also apply when using Entitas. My main goal is to explore and define workflows and conventions to help developers avoid having issues. Those conventions could lead to additional apis to encourage people to structure their code in a certain way. After that, the ultimate goal which I'd love to achieve somehow is that you don't even have to care about details of MT; Entitas could figure out those details for you and run systems in parallel by itself. This would not be part of Phase 1, but I like to set the goals high ;)
There is an older ECS framework for .NET on github called Forge (https://github.com/jacobdufault/forge) which implemented MT by enforcing immutable state. The whole architecture basically uses double buffering for the gamestate. During one update cycle, the systems read only from the old state and only write to the next. No interdependencies between state changes in a single update cycle are allowed. I didn't use it much though and sadly it didn't pay any attention to cache coherency and data locality.
If I take Entitas-Shmup as a case for MT, most of the systems there could run code in parallel on the Entity level, for example updating positions, velocity, damage and all that. Basically, any processing that does not depend on another entity's components can run in this mode. Using something like this could be as easy as changing IReactiveSystem to IReactiveMTSystem.
Running systems in parallel is not as simple, but because entities are a composition of many components, systems that operate on separate components could run in parallel. It would be nice if we had systems that operated on subsets of the components and combined the results at the end somehow. Maybe under the existing code, something like EntitySystems that are composed of ComponentSystems could be used to achieve this.
An interesting approach might be something similar to what this guy is doing with a C++ ECS framework:
While he uses C++ templates to setup the metadata and configure systems, I like the idea of having predefined "modes" for parallelization. There would also be the option to customize for specialized use cases, as needed. His technique also ensures that systems later in the chain must wait for all of their dependencies to complete before executing, which seems to remove the need for double buffering.
https://youtu.be/YHweZ8dhOJA?t=4483 It may go side by side with the new Unity C# job system (safe multithreading)
@KumoKairo - That's what I was thinking, assuming the Jobs system comes out at a reasonable pace.
Just throwing in here that there might be a conflict with WebGL builds and multithreading.
I actually don't see how Job system would help Entitas. I mean, it can be used while Entitas does not have multithreading, or maybe it can be used by Entitas to take advantage of whatever thread safety mechanism it has, if it's possible, but I assume it will have some overhead. It seems to me that some solution like double buffering is the easiest one to work with and it would "just work".
In Entitas I imagine you could have base classes for multithreaded systems, and they would be managed under the hood. Entitas would run multithreaded systems (reading from buffer1, writing to buffer2), then in the main thread swap the buffers, run through systems that can't be multithreaded for some game logic reasons, swap buffers, start multithreaded systems again, and run all view systems in the main thread meanwhile (without them being able to write to the buffer).
@periodyctom Parallelisation without the double buffering? Is that doable for Entitas? Would determinism be sacrificed? Would reduced memory footprint be the only gain compared to double buffer? It thinks double buffer doesn't really require almost any extra maintenance internally.
@VladislavKostin Honestly, you're probably right about double buffering being the simplest, I'm merely pointing out that I like the policy based approach presented in the links I posted above, for the people who need more control for specialized scenarios. The idea of customizing the threading work chain (with some rules to enforce determinism) is very appealing.
What I would like to see is the ability to setup the threading policy for systems, with double-buffering probably being the "default, simple, most tested" option, but having the ability to say "run the system chain synchronously, but for Feature/System X, split the entity workload across all cores, then resume on the main thread", perhaps with some stipulations or deferred operations like in the example I posted.
Hello @sschmid, we are experimenting with Entitas right now. I was suprised to discover that using Entitias would make code significantly harder to multithread. That is our main concern with Entitas now. We are currently considering modifying Entitias to add some form of double buffering or sadly discarding it. I was really happy to read about your plans here. I am curious how would you like to achieve it. I guess you have some thoughts already about that. Could your shed some light on it and share the rough ideas of that practices and workflows you have came up with? Also, is there any time horizon at which you expect it to be ready?
surely we can use code similar to this and subclass ReactiveSystem -> ParallelReactiveSystem? https://github.com/thelinuxlich/artemis_CSharp/blob/master/Artemis_XNA_INDEPENDENT/System/ParallelEntityProcessingSystem.cs
Or am I missing something?
I wanted to get everyone's thoughts on this before implementing it myself and trying it out, don't want to waste my efforts if it's not going to work (perhaps the component/context events triggering in parallel cause unintended side effects. not sure how that works.)
I don't develop my games in unity directly though, we can't use the Unity C# Job system outside of unity and entitas isn't a unity library. It has unity extensions but the core is not coupled to it.
I'm wondering about the future of Entitas now that Unity releasing it's own ECS.
Unity's new ECS + Job System + Compiler Tech is an insanely optimized system. They claim that their code is even faster than vectorized C++ code, because of their custom compiler optimizations. And this will be available only in Unity. They have manual memory management with all it's drawbacks and performance advantages with zero garbage allocation. That's gonna be the "right" way to create games, and all advanced Unity developers will learn it. Unity has almost 100% multicore utilization.
Entitas probably won't be able to reach the same level of performance. Entitas might have better API though and better workflow now with compiler. But then Unity ECS probably gonna have better support, docs and tutorials. Most of the developers don't look for super high performance, they just want a good architecture and API that are easy to work with. Also Entitas is portable and can be stand alone. The reactive systems might be an optimization (is it?) that Unity doesn't have (at least yet).
I'm not even sure what I would choose, of course, depends on a project as well.
True but not everyone will be using unity so for instances when you don't want to use unity but want an ECS framework you'd be glad Entitas exists whether you're making a game or not. :)
@T2RKUS I would go and implement this thing anyway. After these experiments we will at least know current friction points of multithreading with Entitas and will know where to move next. As for the Unity's ECS - their solution is likely to be very specific, while general multithreading is usually more general and applicable to more usage cases, so it will find its user even after Unity will release their super-fast-and-specific ECS
@T2RKUS Generally your idea of "ParallelReactiveSystem" is valid and it would be a good and easy way to multithread stuff to some degree. If it fits your needs then go ahead. When writing my post I thought about higher level parallelization tools, something like upcoming Unity Job system with job dependencies and other stuff.
@Lynx32 it's been a while since June 1st, can you share your experience with Entitas so far? Did you decide to discard it or still using it?
@KumoKairo We have used entitas and we are glad we did it. I really enjoy working with it. Unfortunately, as the game is on a prototyping stage, we still did not bother to adapt entitas for our needs in MT. We have reviewed important parts of the source code and prepared "strategy" for implementing it. We also keep annotating dependencies between the systems and try to keep code ready for the change, but that is all for now in this matter.
It is hard to tell when I will get to make that change. It will not be sooner than January 2018. I also really look forward to Unity Job System and Unity ECS. Depending on the time of its introduction and state of development of our game, we would also consider switching to Unity solution.
I played around with threading again. I added a JobSystem.cs as a proof of concept.
This system distributes the execute over multiple threads and joins them again. This can speed up performance if you either have lots of entities or if the calculations are very intense.
It's really simple to set up:
public sealed class RotateSystem : JobSystem<GameEntity> {
public RotateSystem(GameContext context, int threads) :
base(context.GetGroup(GameMatcher.AllOf(GameMatcher.Rotation, GameMatcher.RotationVector)), threads) {
}
protected override void Execute(GameEntity entity) {
entity.rotation.value = entity.rotation.value * Quaternion.Euler(entity.rotationVector.value);
}
}
Mutating the component values directly (non reactive) and therefore circumventing groups, collectors and reactive systems can already result in massive performance gains.
I think you should approach performance critical systems in the following order:
- Use system as normal (reactive)
- Use system without generated methods (non reactive)
- Use JobSystem
HI, how is this gping along? Is this on par with Unitys ECS/Jobs? Cant really decide which to use for my next project
I guess Burst is out of question tho.
@rikkuporta
If you have high requirements and you already know that you have to multithread big parts of your simulation, try Unity ECS. Entitas currently has a JobSystems class which split the task on multiple cores, but has limitations similar to Unity ECS where you have to delay entity creation and deletion.