quantopian
Can zipline ingest data incrementally?
Dear Zipline Maintainers,
every time I ingest data, zipline seems download all data and process ingest, does it has approach to execute incremental update?
Hi @hotea you're correct in that the data is downloaded all at once; we don't do incremental updates. I think it's possible that this could be a good addition, though there might be some caveats (e.g. what if there's an update to data at a previous date)?
I don't think we currently have time to get to something like this, but if someone wants to give it a shot and open a PR, that'd be awesome.
Surely then the answer is to subscribe to a data service which simply downloads updates to CSV files and then ingest those? CSI Data for instance.
There seems to be no information anywhere on how to download daily updates for bundles without downloading an entire database each time you want the latest prices.
How are people achieving this?
