TileDB icon indicating copy to clipboard operation
TileDB copied to clipboard

Out-of-core fragment metadata reading

Open stavrospapadopoulos opened this issue 6 years ago • 0 comments

Currently we load all metadata of a fragment in main memory. For large arrays, the MBRs and other information about tiles may become a large memory issue, especially when multiple independent processes run on the same machine (which do not share a common fragment metadata cache).

We should transition to a solution where certain fragment metadata (like MBRs) are kept in persistent storage and read in an out-of-core fashion on demand, in order to minimize the memory footprint of the fragment metadata.

stavrospapadopoulos avatar Feb 04 '19 22:02 stavrospapadopoulos