django-robots
django-robots copied to clipboard
jazzband
A Django app for managing robots.txt files following the robots exclusion protocol
Used in the following places: - https://github.com/jazzband/django-robots/blob/be2e781b46aeceb728303f46da585d237e079409/src/robots/settings.py#L11 - https://github.com/jazzband/django-robots/blob/be2e781b46aeceb728303f46da585d237e079409/src/robots/views.py#L22
In Django 4, an error occurs when the cache time is set AttributeError: 'TemplateResponse' object has no attribute 'headers'
Firefox console shows this error: "The character encoding of the plain text document was not declared. The document will render with garbled text in some browser configurations if the document...
This package is configured properly on my local machine. However when I pushed it on our staging server (apache + uwsgi), it is behaving weirdly. When I go to https://test.mysite.com/robots.txt,...
Hello! I am using django + nginx + https if set `ROBOTS_USE_SCHEME_IN_HOST = True` I get this result in the robots.txt: `Host: http://site.com` But expected: `Host: https://site.com` Maybe it happens...
Allowing to set the order of allow and disallow rules would be helpful. As far as I know google does not care for the order, but other do, as speicied...
Hi! If anyone is able to help out with maintenance, QA, writing tests, reviewing the pull request list, and triaging issues, this would be helpful! At this time I'm a...
To repeat this. 1) Make sure django goversion is 2.X or above 2) Make sure SITE_ID is not set. 3) Make sure ROBOTS_SITE_BY_REQUEST is not set on settings. /robots.txt endpoint...
Hi, I use https on project. In my robots.txt file I cant get sitemap url to have https scheme. In site admin I have domain name without scheme, and name...
With robots 3.0 and Django 1.11 with deprecation warnings enabled, I see robots/views.py:3: RemovedInDjango20Warning: Importing from django.core.urlresolvers is deprecated in favor of django.urls.
