Mayank Daswani

"{"logname": "VenezuelaHumDataSource", "timestamp": "2020-11-18T11:53:07.1124", "classname": "VenezuelaHumDataSource", "loglevel": "error", "source_name": "VenezuelaHumDataSource", "config": {"name": "pipelines.epidemiology.ve_humdata.VenezuelaHumDataSource", "fetch": [{"url": "https://docs.google.com/spreadsheets/d/e/2PACX-1vQI4s0no2TS1dYxbv82nhKD7iz8fbDGwdsOI4kzJ0cg3gjOR51KIw_rNOff97Xic_fRQD41xmsDGUfM/pub?gid=1029482781&single=true&output=csv", "opts": {"ext": "csv"}}], "test": {"location_key_match": "VE_.*"}, "automation": {"job_group": "all", "deferred": true}}, "traceback": "Traceback (most...

"{"logname": "AlaskaDataSource", "timestamp": "2020-11-18T11:00:53.2333", "classname": "AlaskaDataSource", "loglevel": "error", "source_name": "AlaskaDataSource", "config": {"name": "pipelines.epidemiology.us_ak_authority.AlaskaDataSource", "fetch": [{"url": "https://www.arcgis.com/sharing/rest/content/items/867f802ce1624b46b40d2bd281490078/data", "opts": {"ext": "xlsx"}}], "parse": {"key": "US_AK", "sheet_name": "Table 1"}, "test": {"skip": true, "location_key_match": "^US_AK$"},...

"{"logname": "epidemiology", "timestamp": "2020-11-18T11:09:36.5360", "classname": "DataPipeline", "loglevel": "error", "source_name": "CongoDRCHumdataDataSource", "source_config": {"name": "pipelines.epidemiology.cd_humdata.CongoDRCHumdataDataSource", "fetch": [{"url": "https://docs.google.com/spreadsheets/d/e/2PACX-1vQGOGhcfVhfuE_wP5Y78ZoKd6xPlc0BVt6uEPTNJ_6J_dqI29LlwC2KHFqw-9viNsBN60z7wC5ruOtt/pub?gid=0&single=true&output=csv", "opts": {"ext": "csv"}}], "test": {"location_key_match": "CD_.+"}, "automation": {"job_group": "all", "deferred": true}}, "message": "No output...

"{"logname": "epidemiology", "timestamp": "2020-11-18T11:09:41.4108", "classname": "DataPipeline", "loglevel": "error", "source_name": "IraqHumdataDataSource", "source_config": {"name": "pipelines.epidemiology.iq_humdata.IraqHumdataDataSource", "fetch": [{"url": "https://docs.google.com/spreadsheets/d/e/2PACX-1vQh_BwL222rdcpIH2rLPIbvdKLJu3fevAy2L82FHUcl-84w6byWRITQicetYzpqX707EUc3qgAJm7Hr/pub?gid=0&single=true&output=csv", "opts": {"ext": "csv"}}], "test": {"location_key_match": "IQ_.*"}, "automation": {"job_group": "all", "deferred": true}}, "message": "No output...

MZ data source failing

1

``` "MozambiqueHumdataDataSource", "source_name": "MozambiqueHumdataDataSource", "config": {"name": "pipelines.epidemiology.mz_humdata.MozambiqueHumdataDataSource", "fetch": [{"url": "https://data.humdata.org/dataset/96f9bc99-28ee-4046-8a96-9f7c1a1462d7/resource/2857979e-a528-429e-b7ce-e4b1c3317718/download/mozambique-covid-19-cases.xlsx", "opts": {"ext": "xlsx"}, "parse": {"skiprows": 1}}], "test": {"skip": true, "metadata_query": "key.str.match(\"MZ_.*\")"}, "automation": {"job_group": "all", "deferred": true}}, "traceback": "Traceback (most recent...

``` "Traceback (most recent call last): File "/env/lib/python3.7/site-packages/xlrd/book.py", line 474, in sheet_by_name sheetx = self._sheet_names.index(sheet_name) ValueError: 'Table 3a - Hospital Confirmed' is not in list During handling of the above...

Wikipedia epidemiology source fails with no output (Malaysia source)

1

``` File "/home/vmagent/app/pipelines/epidemiology/xx_wikipedia.py", line 104, in parse: assert data is not None and len(data) > 0 at run (/home/vmagent/app/lib/data_source.py:205) at _run_wrapper (/home/vmagent/app/lib/pipeline.py:169) ```

Multiple match failures in Libya datasource parsing

1

``` /Users/mdaswani/Documents/open-covid-19/data/src/lib/error_logger.py:30: UserWarning: [2020-07-30T15:33:09.1220][LibyaHumdataDataSource] No key match found for: match_string البريقة Test Samples 13 total_confirmed 1 total_recovered 0 total_deceased 0 date 07/30/2020 country_code LY _vec البريقة Name: 2, dtype: object...

"Key provided but not found in metadata" is misleading in many cases

2

When running the unit tests, I can see a lot of messages that say (e.g.) `{"timestamp": "2020-08-26T14:22:58.2411", "classname": "CzechRepublicAgeSexDataSource", "record": {"key": "CZ_31_311", "date": "2020-03-13", "age": "20-29", "sex": "male", "new_confirmed": 1,...

Add L3 data for Australia

7

Link to related request in previous repository https://github.com/open-covid-19/data/issues/309