cookieconsent icon indicating copy to clipboard operation
cookieconsent copied to clipboard
verified publisher icon orestbida

[Feat]: Extend hideFromBots function

Open HemiCZ opened this issue 1 year ago • 3 comments

Description

The current function only checks bots with user-agent like /bot|crawl|spider|slurp|teoma/i.

However, Google uses other bots with different names, such as:

  • Google-InspectionTool
  • GoogleOther
  • Google-Extended
  • Chrome-Lighthouse

https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers#googlebot-desktop https://stackoverflow.com/a/78137919

Proposed solution

No response

Additional details

No response

HemiCZ avatar May 29 '24 20:05 HemiCZ

All the user agents strings for the services you listed contain the "bot" key, which is already included in the regex.

orestbida avatar Jun 03 '24 17:06 orestbida

@orestbida I'm not sure if I understand it, but in "Google-InspectionTool" is no "bot" key, or?

HemiCZ avatar Jun 04 '24 07:06 HemiCZ

My bad, I didn't check the second half of the services (the ones you actually listed). You are right, the plugin won't detect these bots.

With that said I think it's fine keeping it as is since the main point of this option is to prevent the indexing of the modal's content (indexing is done by other bots afaik).

orestbida avatar Jun 05 '24 11:06 orestbida

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jul 06 '24 08:07 stale[bot]