Add Examples and Guides

[pablomayobre]

Add examples on how to use the different parts of Concord. Some of these already exist in the README but could see some improvement with in depth explanations, for example, when to use them, or how they relate to other features.

Introduction

• [x] Getting Started (Rewrite from README)
• [ ] Tips & Tricks (Utils, Assemblages, Method Chaining)
• [ ] Making a game with Concord and LÖVE
• [ ] Tooling? Once we have tooling

Basics

These following articles assume previously reading Getting Started They go into the same topics but going a bit deeper into each topic and covering all the available methods. They would then link to more advanced topics when needed.

• [ ] Components
  • [ ] Flag Components
  • [ ] Adding methods
  • [ ] Do's and Don't's
• [ ] Entities
  • [ ] Give vs Ensure
  • [ ] Remove Components
  • [ ] Assemblages
  • [ ] Removing Entities (go into detail on safe ways to do it)
  • [ ] Entity's World
• [ ] Systems
  • [ ] Filters (including negation)
  • [ ] Pools (how to create many)
  • [ ] Enable/Disable
  • [ ] Callbacks
  • [ ] Initializing
  • [ ] Storing data
  • [ ] System's World
• [ ] Worlds
  • [ ] Events (basics)
• [x] Why ECS? Why Concord?

Intermediate

These articles are for more specific uses and functionality not used by many but useful for those that would like to benefit of the feature complete benefits of Concord.

• [ ] Cloning a Component
• [ ] Entity Keys and Resources
• [ ] Serialization
• [ ] Loading Namespaces
• [ ] Sorting pools
• [ ] Component:removed
• [ ] onAdded/onRemoved
• [ ] onEmit

Advanced

Finally these articles go into some of the internals and are tailored to advanced users or people writing libraries that go alongside Concord.

• [ ] Tweens
• [ ] UIs
• [ ] Cloning an Entity
• [ ] Reusing Entities (mark as empty, find and reuse them)
• [ ] Before and After Events
• [x] Pool Flushing and Timing
• [x] Custom pools
• [ ] Finding the Entity and World in populate functions

[pablomayobre]

Pool Synchronization

This article will help you understand when the Pools' Filters get synchronized with the latest changes to Entities in the World, the relationship with Events, how it affects your game and how you can avoid the default behaviors.

Synchronous Operations

Concord has a synchronous part and an asynchronous part. For the most part you'll be using the synchronous part, so whenever you make a change, most of the things update immediately, for example, giving a component, removing a component, getting a reference to an entity, sorting a pool and many others are synchronous operations, they execute immediately when you perform them and you can see the effects in real time.

entity:give("position", 10, 20)

entity.position -- { x = 10, y = 20 }

entity.position.x = 30

entity.position -- { x = 30, y = 20 }

world:emit("update") -- All Systems with an update listener are called immediately 
                     -- in the order you added them to the World

As you can see most operations are immediate and you can see the results right after

Asynchronous Operations

There is one place however where we can't have immediate updates without introducing other sets of issues. That is Pools and Filters.

As discussed in the in-depth Systems article, all Pools have an associated Filter, this Filter specified all the requirements Entities should have to be in the Pool. The one checking that Pools have all the Entities that meet their requirements is the World.

Syncing the Pools

In order to make sure that the Pools have the Entities they need, the World goes through all the necessary Entities and checks them against the filters of every Pool, one by one. This is a very expensive operation which we call Flush.

You can actually trigger a Flush manually through World:flush() and you may sometimes want to do so. We will discuss the use cases later on.

When do we sync?

Entities change a lot, we often change multiple Components one after the other, add Entities in bulk, apply Assemblages that give multiple Components at once, delete a bunch of Entities that died simultaneously and so on.

The World gets notified of ALL these changes, then keeps track of all the Entities that have been changed, removed, or added to the World, and waits until it's ready to perform a Flush.

Flushes can be manually triggered as we previously discussed, but Concord also has a built-in mechanism for flushes to happen automatically.

When can the sync cause issues?

A sync performs additions and removals on Pools, so the biggest issue it can create is if you were iterating over one Pool, and suddenly the Entities inside were to shift around. This is not a problem with Concord, its List implementation or flushes, this also happens in regular Lua:

for k, v in ipairs(t) do
  table.insert(t, 1, k) -- This has undefined behavior and can cause an infinite loop
  print(k, v)
end

So it's not a good idea to add or remove inside to the current Pool we are iterating on. So we should not flush while we iterate. These iterations generally happen inside event listeners inside your Systems, since that's where Pools live, so we shouldn't flush while a System's event listener is executing.

World:emit()

Concord tries to keep the list of Entities consistent through an entire emit event and won't automatically flush until the Event is over. This is so that if System A and System B handle the same event their Pools are consistent with each other, this assumption may be broken if World:flush() is called manually, but that's an explicit choice by the user.

The best time for the automatic flush then is before the Event is forwarded to the Systems, it's a spot where no updates are being made, no iteration is going on, the World has complete control of the main thread.

However World:emit() can be called from inside World:emit() so Concord makes sure that by default a nested World:emit() doesn't cause an automatic flush.

If you wanted to emit an Event without causing a flush at all you can use World:emitNoFlush() as a replacement to World:emit(). This can be used to gain more control on when flushes happen.

love.update = function (dt)
  World:emit("update", dt) -- Flushes before forwarding the event to the Systems
end

love.draw = function ()
  World:emitNoFlush("draw") -- Doesn't flush
end

function RenderSystem:update ()
  self:getWorld():emit("prepareForRender") -- Doesn't flush

  self:getWorld():flush() -- This is a manual flush
end

World:query()

This function is always executed against the most up to date version of the Entity list, it is not affected by flushes and it includes the latest updates at the time of executing the function.

local myEntity = World:newEntity():give("position", 10, 20)

local list = World:query({"position"}) -- list contains myEntity even though there was no flush

If you want to perform additions/removals of Entities, keep in mind you shouldn't use the onMatch function since that happens a loop. You can still use the list alternative.

-- Don't do this
World:query({"position"}, function (e)
  World:newEntity()
  e:remove()
end)

-- Do this instead
local list = World:query({"position"})
for _, e in ipairs(list) do
  -- This is safe because list is not mutated
  World:newEntity()
  e:remove()
end

Forbidden Flush

There are a few places where flushing is forbidden the list is short though:

• World:onEntityAdded / World:onEntityRemoved
• Pool:onAdded / Pool:onRemoved

These callbacks happen during a flush, you can't flush during a flush since the buffers are in use.

If your code performs a flush but may be called from inside of one of these callbacks you can use World:canFlush() to check if flushing is allowed, otherwise an error will be thrown.

if World:canFlush() then
  World:flush()
end

[pablomayobre]

Why ECS?

There are various reasons why people choose ECS for their projects, there is a very large FAQ for ECS made by the author of FLECS, but we can go through some of the reasons here as a starting point:

It's a trend

Yeah it's a trend, most game engines are moving towards ECS as their pattern of choice, big names in the industry are using the pattern and more people are following this trend (Unity, Unreal Engine, Overwatch (video), etc).

This however not a real reason to use Concord but it does help that more and more documentation is being written towards ECS.

But this is also a downside, ECS is NOW being used by more game engines, but OOP has been the king for way longer and is more prevalent and used in this market.

Concord isn't here as part of the trend, and doesn't intend to follow what other ECS frameworks do. It instead presents it's own interpretation of ECS and makes a heavy focus on the ease of use and development.

It doesn't rely on inheritance

So to follow this discussion, a reason people choose ECS is so that they can get away from the ugly parts of OOP.

The ugly part being inheritance and all the problems it comes with. A possible solution would be mixins or decorators, but most people agree that the problems are still there.

So that's when they start to consider composition over inheritance. This is a very common pattern in OOP where you put objects inside objects instead of trying to define their behavior in term of inheritance. And ECS is very close to this pattern, remember how we put Components inside of our Entities? That's composition.

But in "Composition over inheritance" we would still define our behaviors in our objects, and we would still need to forward our events to each component. So that's when people start to consider ECS, behaviors existing outside of the object/entity helps organize the code more neatly.

Concord does make use of OOP and the metaprogramming functionality in Lua, to help implements ECS, and does still allow you to work with Components and Systems as if they were objects. So if you are escaping from the "bad parts" of OOP, your OOP knowledge will not be wasted.

It's faster

Well this is the argument most game engines make to sell you ECS as the best architecture.

Remember our behaviors and data are stored separately? This helps keep our data small. Remember how we don't make distinction between entities and instead rely on filtering? This allows us to pack our entities close together in memory so that we don't have to jump around so much when looping through them in our systems.

Unfortunately this doesn't apply to Concord because in Lua land we can't benefit from all of this, our representation of Components, Entities and Pools don't provide an advantage towards objects. Yeah Concord is fast, but this is not its selling point. You can probably write faster code without a library, and there are plenty alternatives that focus on speed.

Lua although fast, is not used for optimization purposes, and the same applies to Concord. Concord tries to be reliable, easy to use and feature complete, it is not an optimization. (We actually made an experiment making a super efficient ECS library, but it was awful to code with and one line could throw all the optimizations away, in our opinion although it's possible and makes a huge difference, it's not worth it when working in Lua)

Reminds me of Functional Programming

If you have tried "The Elm Architecture" or "Redux" you may find the unidirectional data flow in ECS familiar.

If we think about our Entities and Components as our Model or State, and Systems as our Update or Reducers. The only way to mutate our Model/State is through an Event through our Update/Reducer

This unidirectional data flow and the concept of using reducer functions as a way to update state is a proven concept and widely used in Functional Programming.

This mental model has worked really well for the Front End world, and is a very good way to reason about how our system updates over time while keeping our state as our only source of truth and purely as data. If you are already familiar with these architecture you'll be able to pick up Concord and ECS in general, with relative ease.

In addition to that, if we combine ECS with immutable data structures for Components and Entities, then we wouldn't be too far away from "The Elm Architecture" and all its benefits like Time Traveling debugging.

Unfortunately immutable data structures aren't cheap and the Lua garbage collector is not tuned for them. Concord doesn't use immutable data structures, but if you treat your Components as just data, it does allow you to recover state through serialization and deserialization.

It's more organized

If you find this mental model intuitive and start to reason only through this paradigm you'll find yourself writing code in a more structured way. There are very specific ways to do things in ECS and Concord will push you towards these patterns.

This is however at the expense of a higher learning curve with a big payoff in maintainability and reusability of your code. Concord keeps this risk low by allowing you to write code outside of Concord without extra hurdles.

In the long run we hope you'll find yourself at home using Concord, without needing these escape hatches, in order to accomplish this we try to ease this learning curve through guides, documentation and tooling that will hopefully make working with Concord a great experience.