RealTime
RealTime copied to clipboard
dymanoid
Citizens virtualization for large cities
In the vanilla game, the citizens get spawned randomly, regardless of time of the day. Furthermore, there are only a few movement reasons that cause a certain spawn: like e.g. illness (find hospital) or need for goods (go shopping). Entertainment and going to work are reasons that are throttled by the game depending on current number of spawned citizens and vehicles.
Since Real Time changes the going to work behavior, this movement reason gets a very high priority. The only throttled reason left is entertainment. However, the throttling only occurs while a citizen is at home. This is because whenever a citizen is not at home, there are time restrictions to be applied (e.g. everyone must leave parks until night time, the workers must leave workplaces according to their work shifts etc). The only unrestricted place to be is "at home".
For large cities (>65k), this introduces a new kind of problems. The spawned citizens throttling relies on the current number of spawned citizens. So, e.g. at 3 p.m., when almost all first shift workers are still at work (already despawned, because inside of buildings), the game allows to spawn lots of unemployed citizens (going e.g. to entertainment). At 6 p.m., when the first work shift ends, all the workers need to be spawned to get back home (or go shopping etc, according to their schedules). Since there is no throttling for citizens being at work, the number of spawned citizens quickly jumps to the maximum 65k. For very large cities, this even causes problems with citizens trying to leave their workplaces, because they cannot be spawned. So they have to wait at work for a "free slot".
This problem needs to be addressed somehow, but currently I have no ideas in which way.
i guess this is where teleporting comes in in the original game behavior. the only options i see is a) increase the limit - have not checked if its technically possible or just too much work (best solution ^^ though performance will drop even further) b) distribute the leaving of working places to a bigger time range - no solution since it only delays the problem (sucks but works for certain city sizes 65k < x < ?) c) let them wait until a "citizen vessel" is available -> as is the case already (kinda sucks since it is basically a forced distribution with an open end, who knows, so cims will maybe never leave work ^^) d) let them teleport (as vanilla i guess, kinda sucks, but it is what it is) e) use another datastructure as an extension where the invisible cims are put into and where their position etc. is also calculated although they are not shown on the map. develop a mechanism to move cims from and to the extension depending on likelyhood of visibility to the player. -> (okeyisch, have not checked but unlimited trees mod does something like that i guess, only that trees are much easier than cims since their position is fixed and does not need to be calculated on update)
Perhaps reduce the allowance of unemployed workers to be spawned, so it's actually less busy during certain times of the day and especially at night.... In the middle of the night (3-4 AM), my 120k city still has around 55k cims spawned..
@DaEgi01, I see no difference between d) and e). Since the citizens are invisible, they don't walk, don't use public transport, don't drive. So there's no point in simulating their movement at all - just plain CPU load without any benefits.
@Sipke82, this doesn't help unfortunately. If there are 100k first shift workers that want to get home at 6 p.m., even completely disabed spawning of unemployed ones doesn't solve the problem.
The high number of citizen instances you observe at night can be explained: the dummy traffic, the shopping at night citizens, and the extended first shift workers are active at that time.
Possible solution - smart teleporting:
- calculate the teleporting probability according to current city population
- the starting point is always "at home"
- for every activity (work, shopping, entertainment), citizens either get spawned and move in their normal way, or teleport directly to the target
- a teleporting flag is applied to the citizens that were teleported
- when moving further according to their schedules, citizens act according to the teleporting flag (spawned ones can spawn further, teleporting ones can only be teleported)
- this continues until the citizen is back home; then, the teleporting flag is reset and will be recalculated according to the current situation
Advantages:
- apparently, the problem gets solved
- no ditched cars because of partly moving as spawned and partly getting teleported
- no skew towards workers (distributing the teleporting equally among employed/unemployed)
Disadvantages:
- more info needs to be saved in the save game (but maybe I can squeeze it in the current data structure)
- less realism
- probably less visible citizens e.g. in parks or hanging around by the buildings
nah .. e) is much more complex in my mind than that. they could become visible if you change the camera position for example. covering distance still takes time, they still produce traffic and congestion. they are just phased out visually while still contributing to the simulation. its like in MMOs where players get "phased out" to other servers if it is too crowded. its not easy to implement properly though, i would not want to do it, but look for another solution that is for sure ^^
Okay, now I get it. This is impossible with the current game's implementation, since you'd need to run all the instances simulation using the whole citizens buffer. Movement only is not enough because of public transport, collisions, etc etc. Even if that'd be technically possible, you'd need a monster overclocked machine to be able to run a, say, 400k city - the simulation is single-threaded (okay, with exception of path finding).
i wonder ... isn't smart teleporting what the game already does in a similar way? ok ... lets call game's default behaviour dump teleporting ;)
No, the vanilla game only teleports citizens in exceptional cases (e.g. when they are "confused", evacuating while instance limits reached, or in some invalid state like "at work" with no work building assigned - this occurs sometimes due to bugs in the game). Otherwise, the citizens just stay in the buildings until there's a next possibility to move.
Another idea: track the number of citizens that are not at home. (This number is not equal to the number of spawned citizens.) When the number of not-at-home citizens reaches the 65k threshold, no further citizens can leave their homes (regardless of their schedules).
Advantages:
- No teleporting
- Technically simple
Disadvantages:
- Large cities will function (mostly) as if they had only 65k citizens (visits count, income from shopping etc)
- Additional data still needs to be saved (but just one number)
This last solution, if it is simple to do, can be tested. I had observed this from time to time: unemployed climbed when there were jobs to spare, and I did not understand. Anyway, it is very timely and scarce.
I guess it also produces jams that are too long in time from 6pm
@benzoll37, both unemployment and jobs assignment have nothing to do with this issue. It's only about spawned citizens.
i like option a:) from egi the most.... but i guess it will be impossible to increase the cim/vehicle limit.
So basically, the game does not get more challenging past 65k cities, from a simulation game perspective.. in fact i think it will become less challenging, because the traffic (cims+vehicles) is will be more spread out over a larger area, and thus less congestion....
that is exactly the case @Sipke82 i call it the empty streets syndrome ;D its funny how ppl show cities >300k and are proud that they have no traffic issues ;(
I just took a look at the current implementation of CitizenManager. It is possible to tweek the limit of 65536 maximal citizen instances, but quiet hacky.( It used a type named Array16 to store all possible instances, which could only take 16bit indices) Is there any other problem of setting the limit to a higher value, despite the CPU and GPU cost? Or, is it a solution for this issue if we could make it possible for the players to set this limit value through setting menu?
ps. Besides the limit of spawned citizen instances, there's also a limit for citizens living in the city, which is 524k. Don't think this would make trouble though
@Sipke82 Technically, it's the same as Citizen instance limit, and can be modified. Just... loads of methods need to be patched.
It'll be easy if we just modify the dll file, but it's bad for realeasing it as mod in the workshop
PS. Performance can be improved if we move the simulationStep function into multiple threads. Current implementation already uses a dedicated thread for this job, basic thread safty is already been made, so it might be not that difficult to move it onto multiple threads
@liiir1985, I don't think it's even possible to increase the instances limits. Currently, there are many methods that do use hard-coded upper bounds while processing the arrays. How do you want to solve this? Please don't suggest the Harmony's "transpiler magic". Messing around with the IL code during run-time is the worst thing one could imagine. I know, some modders do use thins technique. But they maybe don't realize the danger of that approach.
Citizens limit is 1m by the way.
One more word about the multi-threading: the game doesn't support this, sorry. The current implementation has no built-in parallelism support, with a single exception for the path-finding thread.
@dymanoid It's actually a constant value which got compiled into IL code. yes, the only way to do it is to messing around the il code, but in a more clever manner. We could use code analysis library like Mono.Cecil(I had a lot of experience with such stuff, also at work. see my github repository- ILRuntime, which is a IL interpreter execution engine) to generate the patch automatically, instead of making it mannually, this would make it more reliable and consistent during future game updates.
What makes the whole thing problematic is the definition of m_instances as an Array16, and all related indices used ushort, this may make the whole thing impossible through runtime patching
The game doesn't support multi-threading, yes. But the calculation of the AI and all the simulation stuff(Logic is seprated from rendering) is already on another thread, rather than main thread. And synchronization lock also exists(See SimulationManager, which will start a new Thread for all the simulation stuff), So basically we can modify the SimulationStep function of SimulationManager, to make the for loop multithreaded, or at least, to make the citzen/viehcle simulation multi threaded. The current implementation is very friendly to the new ECS system introduced in Unity 2018, really hope the developer could make use of it, so simulating millions of Citizens would become reasonable
OK, I took a deeper look at the current implementation, 16bit instanceID is everywhere, it's impossible to patch all of these methods including parameter definition/ field definition. We can basically forget about increasing the instance limits :(
@dymanoid How about share riding? Will it solve the problem in some degrees? If the Citizen cannot be spawned, then call another citizen which has a viehcle to pick him up, then they move together. Maybe it's somehow more realistic than just teleporting the citizen directly
This would need a comprehensive search mechanism which will drastically drop the simulation performance. The citizens will still need to be spawned to get into car and later to move from the car to the building. If we don't spawn them and let them ride "virtually", then why bothering at all? It's still like teleporting them directly to the target.
Let them ride virtually will still produce traffic to the area where this citizen lives, so maybe it will make the streets look more busy?
I know it's calculated by car. What I mean is, maybe it won't be any car driving by otherwise, and now some one needs to pick him up, so it looks like a "spawned" car is driving by.
I don't think it's a good solution in general. It won't solve the issue actually. For very large cities, every passenger car would need to pick up 3 passengers each time. Just imagine the traffic nightmare and the simulation performance drop in that case.
The feature itself is actually not bad. It adds some realism. But I see it rather in the "mass transit" domain - how does a particular citizen want to travel: by car, on foot, by public transport, or share a ride. So this is out of scope for Real Time.
@dymanoid I like your smart teleporting idea. It's unfortunate that C:S has these limitations but I think that's probably the best way to go about it- at least for the time being. Maybe someday CO will improve on this given the improvements of the hardware of the average user since the game's launch- or if not, then perhaps in their successor to C:S they've mentioned before that'd eventually solve many of the technical limitations.
What if you take a "time zone" approach, especially for larger cities....
Imagine the city tile layout: A B C A B C A B C
Everyone can still go to work at ~6am for first shift but citizens in row A go to work at 5am, citizens in row B go to work at 6am and citizens in row C go to work at 7am. Then, they get off 8 hours later.
Example: At my job we all work "first shift" in the typical sense, but half of us come in at 6 and leave at 3 and half of us come in at 8 and leave at 5.
Any possible way to assign a "shift" randomly to a citizen? I know many jobs which require odd working hours...for example my dad goes to work at 2am for a 3am start and gets off at 1-2pm and gets home at 2-3pm. Even if you just randomly assign a work start time, specifically for office/factory jobs. Some factories are 24h/7d operations. assign a start time to a citizen, say 5am. They leave at 4:30am for their job and get off of work 8-9 hours later, +/- overtime chances.
For the time zone idea, would that be based on where cim lives, or the destination? Doesn't existing variance already do that sort of thing anyway (config: on-time ratio, and max overtime hours)?
On similar theme to time zone, maybe as city grows the distribution of shifts could become more equalised?
tl;dr: As city grows, increasingly disregard shift ratios and some other config settings in order to more evenly spread cims spawn times.
The config set by player would define how smaller cities work, but beyond a certain point RT could start equalising the ratios between shifts (weighted to so that evening shift grows faster than night).
They'd never become fully equalised, as you still want less people in evening, and even less at night, to create distinct periods of the day; but it would give player general sense that the city is getting more busy as they'd notice more people on streets in evening/night. It gives a way to make the city feel like it's still growing once you're regularly over 65k limit during day time.
Does dummy traffic count towards the 65k limit? If not maybe that could be increased to make the roads look busier?
EDIT: Maybe the on-time ratio and overtime hours config settings could also be less adhered to as city grows, in order to increasingly spread the load as the 65k limit is reached?
- Small city: Does what you tell it to
- Big city: You're still the Mayor, but the lunatics have taken over the asylum
Maybe as population grows, increase +/- time randomisation factor of schedules, applied to all cims.