scikit-mobility
EPR cannot generate 5K agents or more with 126GB of RAM
Hi, I tried to generate 5K+ agents with EPR and I see that the generation works fine, but you have a bottleneck here: https://github.com/scikit-mobility/scikit-mobility/blob/master/skmob/models/epr.py#L354
At this line, the script starts increasing the RAM a lot. With 10K agents, for the city of Boston and for 1 month of data it uses more than 128GB of RAM.
Can you help me?
PS: I think that one first step to make it more efficient would be to use datetime precision to the second/minute, without considering milliseconds.
The memory used by the simulation may depend on the size of the spatial tessellation. Consider that, as explained in #59, the simulation relies on an origin-destination matrix the size of which is quadratic in the number of locations in the spatial tessellation. Though by default the origin-destination matrix is implemented as a scipy.sparse.lil_matrix to save space, the more agents you simulate the more you populate the matrix and hence the more memory you need if the spatial tessellation is big. How many locations do you have in the spatial tessellation?
