quant
quant copied to clipboard
DataProviders should have a common lifecycle
I want authenticated API sources flow through the Warden + Devise + Omniauth provider. They should have a common lifecycle.
On initial authentication, all historical data should be downloaded from the provider (if possible). Then, a synced_at attribute should be set with the current time stamp.
Data should be stored with any uniquely identifiable information from the API. This data should be indexed.
Subsequent requests should:
- only request data after the synced_at date
- use the stored ids as a uniqueness constraint
- be performed without the need of scheduling (assuming they aren't web-hook style)
Progress made in these PRs:
https://github.com/jdjkelly/quantify/pull/20 https://github.com/jdjkelly/quantify/pull/22