arctic
arctic copied to clipboard
VersionStore datatype specific metadata
It would be useful if some datatype specific metadata were added to the metadata in version store. example - if storing a dataframe, could the metadata not contain more information about the index? My specific want would be to have min and max dates from the datetime index in the metadata. It seems useful and easily doable (on the first write store the min and max, update them as applicable in subsequent appends, etc). Any thoughts @dimosped @jamesblackburn ? Is there a feature that more easily exposes this information I'm not aware of, or are there compelling reasons to not do this in this way?
I also think auto adding min and max for time column would save my current self manage metadata efforts.
Has there been any work on this? I would be quite interested as well.
nope, PRs are welcomed
Ok I have never done that before. Let me see what's involved...