[Feature Request] Chunk management improvement

[gentlegiantJGC]

Feature Request

The Problem

The way we store chunks is not the optimal solution. We have a dictionary of chunks in RAM which needs to be manually cleared to minimise memory usage. Chunks that do not exist and chunks that error during loading are both handled as None in the storage leading to the chunk being fully loaded each time and throwing the error each time.

Feature Description

Switch to a weakref dictionary so that chunks are automatically unloaded if unreferenced. Add a dictionary in the world to store references to chunks that have changed so that they are not automatically unloaded. Perhaps store a small buffer of chunks so that frequently accessed chunks do not get unloaded immediately. collections.deque Switch chunk storage to distinguish between null chunks, error chunks and valid chunks.

This would move the management of the change state out of the chunk and into the level so Amulet-Team/Amulet-Core#257 would need to be implemented to make the old changed method work however I think it would be best if it was depreciated and replaced with level.put_chunk.

Here is a prototype including this https://gist.github.com/gentlegiantJGC/d0f0b6d34db696b5e5d41d166a0033e0