list icon indicating copy to clipboard operation
list copied to clipboard

Change ingestion behaviour for Canada dataset

Open AnyaLindstromBattle opened this issue 4 years ago • 1 comments

Noticed that the location of the Canada data has recently changed, i.e.:

Individual-level data (cases.csv and mortality.csv) have been moved to a new directory in the root directory entitled “individual_level”. These files have been split by calendar year and named as follows: cases_2020.csv, cases_2021.csv, mortality_2020.csv, mortality_2021.csv. (on 2021-01-27)

This will require some changes to ingestion, we will have to set up two sources and ingest the cases_2020.csv once but set the cases_2021.csv to ingest daily.

No major rush as will need to wait for chunking to be fixed anyway.

@Mougk tagging your FYI

AnyaLindstromBattle avatar Feb 04 '21 13:02 AnyaLindstromBattle