ThrawnCA

> [@wardi](https://github.com/wardi) is it ok if I work on it? He marked it "Good for Contribution" and "Beginner Friendly" so I'm sure it's fine.

Presumably this should also be applied to `/group/` searches. There should also be a way for plugins like `ckanext-datarequests`, which add new types of searches, to easily exclude them from...

> Over the years we've tried various means to instruct bots not to get confused over faceted search results, some bots adhere the instructions, but some don't, especially now when...

> How about we generate a sitemap for crawlers (paginated for sites with thousands of datasets) and block the search, group dataset search and organization dataset search pages from being...

> These changes would help with preventing indexing the pages. They don't prevent crawlers from wasting resources generating all sorts of combinations of search results and facets just to find...