piker
piker copied to clipboard
TSDB ingest of historical data sets: a dedicated `piker` tool to run parallelized history gathering
Once we get to the point where we're pulling both historical and real-time data (after #247 lands) there will be an obvious need for a local caching system to avoid needless network usage/latency especially when displaying real-time charts for massive sets of ever increasing price data.
TODO:
- [ ] a
piker ingestcommand to run a parallelized history gatherer program based on a default symbol set and the user's config - [ ] some kind of crawler system to obtain all the symbols?
- eg.
ultra-financeseems to have an example of this in itsstockCrawlerscript.
- eg.
ultra-finance seems to have an example of this in its stockCrawler script.
For financial data time series I just found alpaca's MarketStore.
It looks like just the thing to back distributed charting and real time algorithmics.
The provide a python client though it'll likely need to be trio-ized for streaming.