covid-19-data icon indicating copy to clipboard operation
covid-19-data copied to clipboard

VACCINATIONS: how to add new countries or entries to our data

Open edomt opened this issue 3 years ago • 1076 comments

This is a centralized issue to let our users suggest sources to add new countries, or new entries for existing countries, to our data on COVID-19 vaccinations.

  • We only take into account numbers that are announced by official sources (head of government, ministry of health, public health agencies, public officials in charge of the vaccination campaign, etc.).
  • We only count administered doses, not distributed doses.
  • We will not include participants in the vaccine arm of clinical trials, as this data is not available for many of the hundreds of trials currently taking place.
  • Our current sources are visible in vaccinations/locations.csv.

For countries that are already included in our data, most of our imports are automated and will collect the latest number when we update our dataset. If you don't see the latest number appear in our data, please wait at least 24 hours before suggesting it here.

Note that contributions via pull requests are not possible due to the way our data pipeline is set up.

Emoji system

@lucasrodes and @edomt use emojis to track comments in this issue.

  • 👍 means that the comment has been read and looked into, and the data will be added during our next update (our vaccination dataset is updated each morning, London time)
  • 👀 means that the comment has been read and looked into, but that the data (or some of the data) will not be added. This can be for a number of reasons: because we already have this data, because there is something wrong with the source, because the numbers contradict what other sources are showing, etc. If you have questions in this regard, you can read our contribute guideline or open a new issue referencing the comment.

Remarks

  • Please read our "how to contribute" section for more details.
  • This thread is meant to be used only to report new data points, for specific questions/suggestions please consider opening a new issue. If your comment is not a data report, a new thread may be opened to follow the discussion (see example).

edomt avatar Dec 25 '20 09:12 edomt

Minister of Health of Israel: On December 27, approximately 98,900 people were vaccinated in Israel. In total, about 379,000 people were vaccinated in Israel. https://twitter.com/YuliEdelstein/status/1343423578205794305

EladHeller avatar Dec 28 '20 06:12 EladHeller

In case some additional sources can be found from https://www.bloomberg.com/graphics/covid-vaccine-tracker-global-distribution/ - it seems to have very similar data, except Russia is assigned an order of magnitude more vaccinations (440K) compared to your data (55K).

artdgn avatar Dec 30 '20 14:12 artdgn

@artdgn Thanks! We know about that 440k estimate from Bloomberg for Russia, but we can't identify a source that would confirm this. In fact, we realized a few days ago that the total number of doses administered so far in Russia was much lower than previously thought: https://twitter.com/redouad/status/1343544952052133888

edomt avatar Dec 30 '20 17:12 edomt

Status update for Bulgaria, 30-12-2020 3844 already vaccinated. Information were presented by ministry of health at today's government meeting. https://www.gov.bg/bg/prestsentar/novini/pravitelstvoto-otpusna-oshte-125-miliona-leva-za-vaksini-sreshtu-COVID%E2%80%9319

philiprusinov avatar Dec 30 '20 17:12 philiprusinov

Hello, Official statistic for vaccination in Bulgaria is available now. Please check the table on the right, on the government COVID portal. It seem this could be automated. For now there are 4608 vaccinated. Best regards,

https://coronavirus.bg/bg/statistika

philiprusinov avatar Dec 31 '20 07:12 philiprusinov

Thank you @philiprusinov, that's very useful! We'll be able to automate the collection with this.

edomt avatar Dec 31 '20 08:12 edomt

Italy: https://app.powerbi.com/view?r=eyJrIjoiMzg4YmI5NDQtZDM5ZC00ZTIyLTgxN2MtOTBkMWM4MTUyYTg0IiwidCI6ImFmZDBhNzVjLTg2NzEtNGNjZS05MDYxLTJjYTBkOTJlNDIyZiIsImMiOjh9

Trying to find in a better format.

Daavide avatar Dec 31 '20 15:12 Daavide

Hello, Thank you for this massive effort. Regarding sourcing data for Greece, the source you used for vaccinations is a regional newspaper. A more official source would be the periodic announcements from the National Public Health Organization. In particular for Dec 30 they gave the specific numbers in an answer to a journalist's question. The announcement's transcript can be found here: https://eody.gov.gr/enimerosi-20201230/

RozaGkliva avatar Dec 31 '20 20:12 RozaGkliva

Thank you @RozaGkliva!

edomt avatar Jan 01 '21 09:01 edomt

How would you feel about adding stats for Catalonia?

You have England/Scotland/Wales/NorthernIreland in addition to UK so i think that maybe adding Catalonia is OK?

Total number is available as text at https://dadescovid.cat/ (the Vacunats field), as a daily number in https://dadescovid.cat/diari (the vacunats column) and as a csv at https://dadescovid.cat/static/csv/catalunya_diari_total_pob.zip

Maybe adding Catalonia we can convince Spain/the rest of regions to start giving vaccination numbers?

tsdgeos avatar Jan 03 '21 10:01 tsdgeos

How would you feel about adding stats for Catalonia?

@tsdgeos We've considered the opportunity to add partial data for Spain, but it would likely lead to requests to add many more subnational regions, which we simply don't have the resources to handle right now. My hope is that the Spanish government will soon publish aggregated data for the whole country. If that's still not the case by January 10, we'll probably have to use some manually-aggregated numbers.

edomt avatar Jan 03 '21 13:01 edomt

Can you please explain what's your rationale to include Wales but not Catalonia?

I mean don't get me wrong, this is your dataset and you do whatever you want with it, but i sincerely would like to understand why a subnational region gets included and another one doesn't.

tsdgeos avatar Jan 03 '21 17:01 tsdgeos

Can you please explain what's your rationale to include Wales but not Catalonia?

@tsdgeos Of course! That's a completely legitimate question. There are 3 main reasons:

  • Timing: we added UK subnational data very early, when everyone was very eager to see vaccination data and the UK was the only country in the world administering the vaccine.
  • Automation: maintaining this data is basically painless as the collection is fully automated via the official API. In many countries, gathering subnational data involves manually collecting data from dashboards on a daily basis. (Catalonia is of course a counter-example since the collection could be automated, but I'm guessing that it's sadly not the case for some of the other 16 autonomous regions.)
  • Size: while there are only 4 nations in the UK, adding subnational data for Spain would mean collecting data for 17 locations, the United States would add 50 locations, etc. This is simply something we can't see ourselves doing with our current resources.

That's not to say that we'll never do this—but national data itself is taking the bulk of our time right now.

edomt avatar Jan 03 '21 18:01 edomt

Norway: The director of the Norwegian Institute of Public Health announced "over 2200" persons vaccinated in a press conference on Jan 3rd, as reported here: https://www.tv2.no/nyheter/11869599/

michael404 avatar Jan 04 '21 15:01 michael404

Thanks @michael404 !

edomt avatar Jan 04 '21 16:01 edomt

Regarding the information about vaccination in Spain, an update:

  • Initially you used a tweet sent by a Spanish newspaper: https://twitter.com/eldiarioes/status/1346150722178576389
  • Then you were told an actual official document existed in PDF format: https://www.mscbs.gob.es/profesionales/saludPublica/ccayes/alertasActual/nCov/documentos/Informe_GIV_comunicacion.pdf
  • I have seen that in your spain.py automation script, you are scraping the vaccinations figure from this URL: https://www.mscbs.gob.es/profesionales/saludPublica/ccayes/alertasActual/nCov/vacunaCovid19.htm

However, I have just seen that they have added a new file in ODS format with the same information as in the PDF, but in a more easily parseable format. It is available here:

https://www.mscbs.gob.es/profesionales/saludPublica/ccayes/alertasActual/nCov/documentos/Informe_Comunicacion.ods

kevloral avatar Jan 05 '21 02:01 kevloral

Thanks @kevloral—you're right that the ODS file will likely become much more usable in the future, but I'm tempted to wait for its next update to see whether the format changes (for example the name of the sheet Hoja3 sounds very temporary).

edomt avatar Jan 05 '21 10:01 edomt

Update on Norway:

Per-day numbers are now available on page 46 here: https://www.fhi.no/contentassets/8a971e7b0a3c4a06bdbf381ab52e6157/vedlegg/andre-halvar-2020/2021.01.06-ukerapport-uke-53-covid-19.pdf

This report will be updated on Wednsdays and linked to from here: https://www.fhi.no/publ/2020/koronavirus-ukerapporter/

michael404 avatar Jan 06 '21 12:01 michael404

Thanks @michael404! This format is… disconcerting. I hope they come up with a simpler system soon.

edomt avatar Jan 06 '21 12:01 edomt

Looks like Norway data is also here in a better format: https://www.fhi.no/sv/vaksine/koronavaksinasjonsprogrammet/koronavaksinasjonsstatistikk/

edomt avatar Jan 06 '21 12:01 edomt

@edomt How frequently do you update the vaccine data? 1, 12, 24 hourly?

nibble0101 avatar Jan 06 '21 19:01 nibble0101

@nibble0101 We update the data at least once a day (some time in the afternoon, Europe time) but often more times after this.

edomt avatar Jan 06 '21 20:01 edomt

Hello, thank you for your effort! There is now the official vaccination number in the Czech Republic. It is in the yellow box here: https://onemocneni-aktualne.mzcr.cz/covid-19

Unfortunately, it seems to be pretty rushed considering the duplicate HTML IDs. It will probably be changed in the next few days. Let's hope there is an official dataset soon.

pehovorka avatar Jan 07 '21 09:01 pehovorka

Hi @pehovorka! I just happen to have automated the collection for Czechia based on this a minute ago :) And yes I agree that it does seem like a very temporary thing, I expect they'll add it to the actual API soon. Thank you!

edomt avatar Jan 07 '21 09:01 edomt

Great, thank you @edomt! :)

pehovorka avatar Jan 07 '21 09:01 pehovorka

Hi, finally opendata for Italy! Should be better than the PowerBI dashboard...

https://github.com/italia/covid19-opendata-vaccini/blob/master/dati/vaccini-summary-latest.csv

Hope this helps, and thanks for your work!

Daavide avatar Jan 07 '21 09:01 Daavide

No open data, but a little bit of data from The Netherlands: https://www.msn.com/nl-nl/nieuws/Binnenland/acute-zorg-verwacht-inenten-maandag-af-te-ronden/ar-BB1cy5JQ?li=BBoPOOe (The National Acute Care Network (LNAZ) says on the first day of vaccination jan 6th, 6.000 people were vaccinated)

ghost avatar Jan 07 '21 11:01 ghost

Thank you @wh82!

edomt avatar Jan 07 '21 11:01 edomt

No open data yet in France (Coming soon...) but the approximate number of vaccinated for the day has been published by the Minister of Health : https://twitter.com/olivierveran/status/1347275767839944712

Since the beginning, 45,000 French people have been vaccinated (first dose). (source: Prime Minister at a press conference - https://twitter.com/gouvernementFR/status/1347226837311635457 at 29'50)

Aymerik avatar Jan 07 '21 20:01 Aymerik

Merci Aymerik :)

edomt avatar Jan 07 '21 21:01 edomt