zimfarm icon indicating copy to clipboard operation
zimfarm copied to clipboard

Wikimedia Maps HTTP 403 (acting like a bad bot)

Open kevinmcmurtrie opened this issue 1 year ago • 0 comments

  • Location: Worker
  • Schedule Name/ID: wikipedia*
  • Task ID: https://farm.openzim.org/pipeline/94167b91-8b89-41de-a351-6287bab02c49

Problem

The Wikipedia scrapers request map tiles from Wikimedia Foundation but have no permission. This results in a lot of HTTP 403 responses. It's possibly enough to be seen as a bad bot.

https://maps.wikimedia.org/img/osm-intl,5,40.415555555556,50.008611111111,260x260.png?lang=it&domain=it.wikipedia.org&title=Ateshgah_di_Baku&revid=137071110&groups=_4ddc15001962e3bddf2312007a0240d4b5316888 Error: 403, Forbidden: Map tiles are restricted to Wikimedia & affiliated sites only. See https://wikitech.wikimedia.org/wiki/Maps/External_usage if you believe your usage supports the Movement. at Sat, 06 Jul 2024 18:55:57 GMT

Reproducing steps

Caused by scraping Wikipedia articles with maps.

kevinmcmurtrie avatar Jul 06 '24 19:07 kevinmcmurtrie