Revisions

[jqnatividad]

trafficstars

It'd be nice if snapshots of the data are also kept around. So we can turn back the clock and fetch the dataset for a given date. Most of these datasets are not really that big.

Also, S3 storage is getting cheaper by the day, and with Glacier, we can even move datasets off to long-term storage if a certain quota is set.

This will essentially make opendatacache an opendatatimemachine as well :smile:

[talos]

Socrata does keep a few old versions around right now, although the interface to pull those old ones is not transparent at the moment.

I have some hope that dat will eventually be a scalable alternative to keeping dozens of big CSVs lying around. But as of several months ago I did not find the performance suitable...

[jqnatividad]

Spoke with @karissa last week. With dat beta officially out, it might be worthwhile to revisit as we're investigating how use dat with CKAN as well.