phpList
Block BingPreview (auto-visits links in emails)
Outlook.com (therein also Hotmail and Live Mail) customers are being auto-subscribed to my newsletter. They open all links by default for security purposes. phpList should block their User-Agent on the confirmation page to prevent this from happening.
As far as I can tell, this is only a problem with email services from Microsoft.
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b
Interesting. Would be good to get a few steps to reproduce this. I have several Outlook accounts, so I can try to replicate it. Can you give the steps?
- Sign up for a phpList managed mailing list with an @outlook.com account.
- Wait.
BingPreview bot will visit the link within a few minutes of the email being delivered. You should see the welcome message being sent to the Outlook email too. Opening the email in the Outlook.com email interface (don’t click the confirmation link!) seems to reduce the time you have to wait.
A similar issue with bots crawling campaign links was recently described on relation to Yahoo and unsubscribe links here: https://discuss.phplist.org/t/users-reporting-unauthorized-unsubscribes/4417
It would be good to add user agent checking and blocking for both Microsoft and Yahoo mail bots.
I wasn't able to replicate it. But I think that in general we should add anti-bot headers (no-follow, no-index) to all pages and also document the robots.txt lines to use. The issue with robots.txt is that it lives in the site root and therefore we have no access to it from the application.
Just confirming OP's issue. Note that I'm not here because of phpList, but of a related issue with Office 365.
Our organization uses Office 365 and it auto visits every link that's sent to our emails with that BingPreview user agent mentioned by the OP. This has been an issue for us because not just phpList, but many other web apps out there also sends out one-click URLs in confirmation emails.
