food-inspector
food-inspector copied to clipboard
Make API importer only update new records
I am currently working on some improvements to the current API importer.
However, as it is right now, we import the whole database from Durham County - despite their API providing us with a field Update_Date and syntax to filter it by Update_Date__gt=mm/dd/yyy that we can use to significantly limit our data / cpu use. We should take advantage of it.
Implementation: we will need to run a query on our db before import starts to establish the "latest" Update_Date value for each table (establishments, inspections, violations) and then incorporate it into the query string.