NBT icon indicating copy to clipboard operation
NBT copied to clipboard

Implement deserializer cache for readonly tag types and TAG_Compound keys

Open dktapps opened this issue 3 years ago • 3 comments

During deserialization of complex NBT, we're likely to encounter NBT with many repeated values which could be cached.

For example, when deserializing a block palette file, the following types of tags are predominantly used:

  • TAG_Byte (used for true or false)
  • TAG_Int (commonly used for a small range)
  • TAG_String (commonly used for small enums)

Particularly in this case (since block state permutations are extremely repetetive) it could offer a significant memory usage advantage (and possibly performance) to keep a temporary lookup table of tag types and their values, so that the same Tag objects can be reused in different places.

For example, deserializing the current block palette (1.19.0) results in:

Type Count (without deduplicating) Count (with deduplicating)
TAG_String 18191 150
TAG_Int 3585 64
TAG_Byte 7168 2

This results in a considerable memory usage saving (16 MB -> 11 MB).

In addition, deduplication of TAG_Compound keys may also save a considerable amount of memory for complex NBT trees, due to frequently repeated keys.

dktapps avatar Jul 08 '22 20:07 dktapps

In 1.19.50, this now makes a difference of 15 MB vs 22 MB in memory footprint, due to a significant increase in the number of blockstates.

dktapps avatar Dec 27 '22 16:12 dktapps

It eventually occurred to me that the solution to this problem was to just ... not have NBT objects floating around at runtime.

dktapps avatar May 03 '23 22:05 dktapps

A Tag object cache might be useful for improving performance of chunk reads in PM5, where blockstate NBT data has known possible values. These object allocations currently cost a lot of performance.

dktapps avatar Mar 06 '24 11:03 dktapps