datproject-discussions
datproject-discussions copied to clipboard
dat-ecosystem-archive
OHI-Science
From @karissa on March 26, 2015 21:21
https://github.com/OHI-Science uses http://iucnredlist.org/
As public data is updated year to year, people must update their scripts, see what changed, and sometimes they even write very long scripts to do things like remove duplicates.
They have a dataset of "truth" that they update every year. There's a column that says what year the row was modified. But there's no data on how it was modified (added?). Sometimes, the data is changed without updating the 'modified' column.
They built this script that pulls out rows that have a modified year of 2013, 2014 and then compare them https://github.com/OHI-Science/ohiprep/blob/master/Global/NCEAS-SpeciesDiversity_v2014/ingest_iucn.R
With dat + a visual diffing tool, they wouldn't have to go through all of this trouble to find what changed. They could just import the data for the new year.
@jafflerbach @jules32
Copied from original issue: maxogden/dat#290
