project-curator icon indicating copy to clipboard operation
project-curator copied to clipboard

Improve overall performances

Open ogxd opened this issue 2 years ago • 1 comments

Context

Database persists in memory as an array of reference types (AssetInfos), with 2 array for each AssetInfo to keep track of references of dependencies. This is rather simple and straightforward however it has a few drawbacks:

  • It relies on Unity serialization/deserialization, which is kind of slow.
  • Database is saved as a json file, which is not the most "optimal" format for keeping track of thousands or even hundreds of thousands of assets.
  • It produces quite a lot of memory fragmentation due to the number of reference types it represents (1 for AssetInfo, 1 for string guid, 1 for dependency array and 1 for references array. All of this multiplied by the number of assets to keep track of)

Solution

Make AssetInfo an immutable struct and use a GUID type internally. Reference references and dependencies with indexes and ranges. This way AssetInfo is of constant size (blittable) and we can serialize everything in a few blobs. In memory, that would also mean keeping one big chunk instead of thousands of references.

ogxd avatar Jul 10 '23 07:07 ogxd