forte icon indicating copy to clipboard operation
forte copied to clipboard

Allow storing actual data content and large meta-data as additional references

Open hunterhector opened this issue 4 years ago • 0 comments

Is your feature request related to a problem? Please describe. Currently, the whole data pack is stored in one place, which includes the metadata, the actual data, and optionally the creation records. Serializing and deserializing such data is a very time-consuming process. In cases like the initialization stage, we only need to check the meta-data for basic validations, thus loading the actual data content would be a waste of time.

Describe the solution you'd like A feature that stores the real data content, or additional creation records as separate files/pointers on a disk would be great, then doing loading, the system can "lazy load" just the meta data first, and then only take the actual data when requested.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

hunterhector avatar Apr 06 '21 21:04 hunterhector