apache-ultimate-bad-bot-blocker icon indicating copy to clipboard operation
apache-ultimate-bad-bot-blocker copied to clipboard

Published 20 hours ago • verified publisher icon mitchellkrogza

FB AI training - Meta-ExternalAgent

Open ShaiMagal opened this issue 1 year ago • 1 comments

FB is using public websites for AI training: XXX 57.141.0.13 - - [28/Nov/2024:12:03:57 +0100] "GET /?fbclid=XXXXXXXXXXXXXXX HTTP/2.0" 200 285 "-" "meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)"

user agent "meta-externalagent" - so we should block this.

This is confirmed by FB: https://developers.facebook.com/docs/sharing/webmasters/web-crawlers

image

ShaiMagal avatar Nov 28 '24 11:11 ShaiMagal