ITF_Power_BI icon indicating copy to clipboard operation
ITF_Power_BI copied to clipboard

CIA Data source is down, need to remove altogether

Open beansrowning opened this issue 3 years ago • 1 comments

@dorquiola @als329

get_country.R#L83 expects a table from CIA world factbook which as of 1/27 doesn't exist: https://www.cia.gov/the-world-factbook/field/population/country-comparison.

image

This breaks the Internal dashboard refresh at 0_output_data.R#L31 where it's called.

Remediation

  • in 1663ff4 I pushed a patch to remove the data pull and instead read in the existing CSV data we've saved
    • (edit: See below, this is likely not sufficient because the function is also referenced internally to other functions)
  • Moving forward, get_country should be removed altogether in favor of onetable from SaviR
    • 2020 pop estimates are out of date, we use 2021 for all routine reports now
    • Web scraping from several data sources each run is unnecessary and fragile

Steps

  • [ ] Investigate impact of using onetable on downstream code
  • [ ] Implement SaviR-base approach
  • [ ] Write a test to ensure stability

beansrowning avatar Jan 27 '22 14:01 beansrowning

Also breaks all scripts that use get_ncov()->get_jhu_data() beginning here: https://github.com/CDCgov/ITF_Power_BI/blob/648bc9c10fd6d17849b96afc3ae4ae2fe6ce6c99/Rfunctions/get_jhu_data.R#L50

I don't have a full inventory, but at least vaccine curves and variant maps for now (in addition to dashboard)

beansrowning avatar Jan 27 '22 16:01 beansrowning