Systems as entities

[andriyDev]

Background

Today, bevy_ecs can roughly be split into two parts:

1. Data Storage and Access: This includes World, entities, components, resources, archetypes and querying.
2. Systems and Schedules: This is all the infrastructure for creating systems and executing them.

The data storage part is very flexible and powerful. It has facilities for storing arbitrary data on objects, composing data on objects, reacting to changes in data, and so on.

On the other hand, schedules are a hand crafted storage solution for systems which includes all manner of metadata: system name, before/after relations, system sets it belongs to, etc.

As a result, we have to build a lot of APIs. Schedules today can't have systems be removed (#279, wow that's an old issue). We'd have to come up with an ID scheme for systems, provide a function to remove systems, and also make sure the schedule graph reacts appropriately to the change.

Proposal: Systems-as-entities

(queue anything-as-entities meme)

Instead of building more stuff on top of the system storage, we can instead rewrite it to use the data storage and access part! Now we don't need an ID scheme for systems: it's just an Entity. Now we don't need a function to remove systems: that's just despawning an entity. As for having the schedule graphs react to changes, we have several tools to handle this. Change detection, hooks, and observers could be used.

As part of this, we will likely want schedules and system sets to also be entities. Systems can then have a system set or schedule component that matches the particular type.

Issues that help this effort

Immutable Components (#16372)

Today, we likely need to use change detection to keep things in sync. With immutable components, we can just use hooks to sync any schedule data structures.

🌈 Relations (#3742)

Today, we either need to store before/after relationships as one component for all a system's constraints, or we need to spawn an entity to represent that constraint. Either way, these are both cumbersome to manage. With relations, we can use them to represent all the before/after constraints as relations. Finding all edges is very easy then.

Similarly, schedules and system sets could use relations to define which systems are in the schedule or system set.

Risks

• This would be replacing a ton of our schedule code. New code can be buggy!
• Users may accidentally mutate the system entities without realizing it by either deleting their entities or mutating their components.
  • This could be fixed with a separate "schedule world", which will be a world that just holds the systems, schedules, and system sets.
• Mutability may be hard to juggle. I'm not certain of this problem - this may be an implementation detail. The schedule will likely still hold a schedule graph representation of some kind which we can probably just take out of the corresponding component and put it back once we're done.

Prior Art

• flecs implements systems as entities! Some of the cool features that just pop out for free was disabling systems the same way entities can be disabled!

[hymm]

Mutability is probably the biggest problem here to solve. i.e. How do you get mutable access to all the systems in the schedule in a safe way? You have to have exclusive access of the BoxedSystem and the cached system params to be able to run a system safely. While you can query for all the systems in a schedule, you have to drop the query before you can run the schedule, since that requires passing &mut World to the schedule. So we either need to hokey-pokey the systems or use some sort of interior mutability:

• @maniwani suggested storing the systems in an Arc Mutex. This will introduce the possibility of panics/deadlocks if you try to run the same system on two different threads. The cost of an uncontested mutex is fairly low and arcing the systems vs boxing will also introduce a bit of overhead. This extra overhead likely won't be measurable, but should be checked. But there will also be more overhead as we'll probably we storing the system param caches as separate components which will also need to be arc mutexed.
• The hokey-pokey would probably be something like storing the system in a Option and remove the data as needed. I suspect that this will be too expensive as we potentially have low 1000's of systems to run every frame. We do this with schedules and it already has a noticeable impact on frame times.
• Some in between method where schedules still own some of the system's data. There probably a sliding scale here of what data the schedule owns vs the world (BoxedSystem, system meta data, system cached params, etc). This would allow some of the data to be accessible, but not sure if there's a line to draw here that actually makes sense.

This could be fixed with a separate "schedule world", which will be a world that just holds the systems, schedules, and system sets.This could be fixed with a separate "schedule world", which will be a world that just holds the systems, schedules, and system sets.

I'm not sure this makes sense as we need queries as entities for relations. This needs observers on adding/removing components in the main world to update the query caches of the matching archetypes. So these need to live in the main world. I suppose it could work if the observers in the main world have exclusive access to the schedule world, but this would block the observers from running in parallel (if we ever enable that).

I'd say other risks are that moving systems into the world might make some future scheduling algorithms harder to implement or block perf improvements. Scheduling is a very perf sensitive part of the ecs and not being able to use customized data structures for storing systems could lock us out of making certain improvements. But that's all rather nebulous, so it's not something that I would consider blocking.

[alice-i-cecile]

I've written a write-up about the principles I think we should follow when doing *-as-entities refactors like this: https://hackmd.io/@bevy/SypE1qZP1l

[andriyDev]

@hymm What do you think about a simple v0? For example:

1. There is a schedule entity with its state as one big component (basically storing Schedule that we have today, maybe behind a Box to speed up hokey-pokey).
2. Systems are entities with a System component that holds the system state until it is added to the schedule. So basically this would hold Option<Box<dyn System>> that is taken and added to Schedule.

The reason I want to get a v0 here is to unblock removing systems, and the main blocker as far as I am concerned is having a way to identity systems. Making an API for adding and removing systems is likely going to be more controversial than just using entities and spawning/despawning them.