arctic
arctic copied to clipboard
Arctic library (mongodb collections) - Best way to store 1000's of symbols for 15 years of history
trafficstars
Arctic Version
# Arctic version here
Arctic Store
# VersionStore, TickStore, or ChunkStore
```ChunkStore
#### Platform and version
Put here
#### Description of problem and/or code sample that reproduces the issue
Hi
We are using Arctic library data to process trades/quotes historical data. For these data we have 1000's of symbols and we are expecting to load last 15 years of history. So far we have loaded 1 year of history and everything is working as expected.
We are using Arctic Chunkstore with Date Chunker to store these data.
We would like to know is it ok to store all data in one collection (arctic library)? or should we split collections by based on date range of data ? (e.g 2017 to 2020 data in collection1? 2013 to 2016 data in collection2).
Thanks for your time
Dhru
I will let @bmoscon reply on the Chunkstore stuff, but it should be fine to store multi-year history in one collection if it's sharded properly (if you do have shards) and the indexes in Mongo look okay which most of the stores create for you.
thank for reply shashank and I will wait for further confirmation from @bmoscon