「Forbidden (403) CSRF verification failed. Request aborted」in Django administration page

[Pan-will]

My actions before raising this issue

I used K8S to deploy all the services that cvat depends on in the cluster. The deployment is as follows:

Then, I went to the backend-server pod and created a superuser account with the command python3 ~/manage.py createsuperuser

Next, I went back to the Web page, logged in the superuser account with the browser, and entered the background management page, as shown below:

Steps to Reproduce (for bugs)

In the Django administration page, all my Post requests are getting errors like this:

Possible Solution

After Google, I found a workable solution: https://github.com/doccano/doccano/issues/1820

Your Environment

As mentioned above, I used the official ymal file to deploy the various services in the K8S cluster. We haven't used Django before, so we don't know how to add annotations to our code. I was looking for a way to fix this by adding an environment variable to the backend-server deploy file.

Thanks for thinking about this!！

[Zanz2]

I encountered the same error, but only on file uploads. PR that would (likely) fix this: #6322

[Pan-will]

I encountered the same error, but only on file uploads. PR that would (likely) fix this: #6322

Thank you very much for your solution! My upload SDK does not have this issue. Due to the addition of some authentication logic, I wrapped the official upload tool and my new code to provide upload service in the form of SDK.

[Pan-will]

Depending on the result of the query, the CSRF question requires the domain name to be configured as an environment variable. But I'm not sure what environment variables should be configured for the backend-server container. Later, Google learned that it should add its own new domain name in CORS_ORIGIN_WHITELIST, so the following figure shows three environment variables configured, but to no effect:

[Zanz2]

As far as i know those settings are only applied when you use the development mode, not the production mode? Assuming a normal "vanilla" helm install:

1. You run helm install, it uses the normal cvat docker image, this one has the DJANGO_CONFIGURATION env variable set to production, not development. https://github.com/opencv/cvat/blob/a43477af8eea946cf6498a6b1d12888840925bcc/Dockerfile#L76
2. The manage.py file then uses "production" since the DJANGO_CONFIGURATION variable is set (otherwise if it wasnt it would use "development") https://github.com/opencv/cvat/blob/a43477af8eea946cf6498a6b1d12888840925bcc/manage.py#L12
3. You then use the "production.py" file and not the "development.py" file that you linked, this one has no such configuration options ( https://github.com/opencv/cvat/blob/develop/cvat/settings/production.py )

So if you would like to use the development.py options, you have to add a ENV variable to your helm chart called DJANGO_CONFIGURATION with the value of "development".

[baudneo]

I have CVAT behind a cloudflared tunnel and am getting the 403 forbidden CSRF on django admin page POST. The provided fix says to use the CSRF_TRUSTED_ORIGINS env var but, Adding it to all the containers doesnt seem to solve the issue.

export CSRF_TRUSTED_ORIGINS="htttp://myserversIP,http://some other ip"

in the docker-compose.yml where do I add the ENV var?

    environment:
      CSRF_TRUSTED_ORIGINS: ${CSRF_TRUSTED_ORIGINS:-}

I put it in cvat-server, cvat-ui, cvat-utils and I am still getting CSRF 403 forbidden

[geometrikal]

@baudneo have you found a fix for this yet? I'm running into same thing

[baudneo]

@geometrikal not yet. I took it down and hosted it locally so I could actually import datasets and annotations.

One fix I was thinking of is adding the CVAT_HOST env var to every container that is running cvat-server image. The CVAT_HOST fix works in the cvat-server container but, several of the worker containers are based on cvat-server image.

My guess is the worker containers need that env var so they know what to trust, that's just my.guess though.

[kelbstf]

Hello, i am facing the exact same POST error in the administration page, when attempting to "SAVE". But am hosting it locally, nevertheless having the problem. I reproduced all methods that have been reported to help others so far, based on tuning "CSRF_TRUSTED_ORIGINS" with the accessing IP addresses, but the problem persists.

@baudneo: do you have an idea why going local solved the problem for you?

Just in case it matters: our CVAT is installed exactly according to the official "Quick installation guide", running on 8080 with CVAT_HOST set to the LAN IP, behind a reverse proxy that terminates HTTPS.

I tried different stable release including the latest and then the latest dev, but see no difference.

Any input would be great, since currently i seem to be stuck.

[baudneo]

Behind a proxy = no good. Accessing direct local is what worked for me. Good luck getting help from cvat team.

[kelbstf]

@baudneo: ahh, i see, thank you for letting me know! I would be glad to get it working with "direct 443" at least in order to make it available soon. But on the long, a proxy would be important because we have to host many different applications, and since it's docker based... It's weired that only POSTs seem to be affected, but that's apparently a side effect of the underlying security concept.

[baudneo]

@kelbstf I did a quick deep dive and couldn't get it figured out. My situation was using cloudflared proxy for public access and I could not get it to work.

My guess is that when the tus or whatever lib it is gets init'd, it needs an option to disable CORS or configure it for CVAT_HOST. At the time I was super busy so couldn't take the time to fix the issue and now I've moved on to other software.

IIRC, it was issue after issue after issue and I had gotten fed up. Cvat team doest reply to non trivial issues and when they do, they usually say "it works on our cloud hosted service, you should stop self hosting and subscribe to our cloud service" instead of doing any meaningful troubleshooting and fixing.

[kelbstf]

@baudneo thank you for sharing your experiences. So it's not just me and i get a perspective. I am totally new to this software, but "issue after issue after issue" is exactly my experience so far. I have quite a lot of experience with implementing a variety of web applications behind reverse proxies, but here i feel i get lost in a maze of fractions of potential reasons and fixes, sometimes even for the same problem, so it's difficult to understand what is going wrong actually. Apparently the integration of django, traefik and CVAT has not a consolidated configuration pattern for the self hosted use case.

Since this is just about getting able to login and creating users, and uploading files seems to be the next challenge, i wonder how this will behave when it comes to the real meat of this application later on.

I still hope someone would share a brief guide, because this application is actually the primary choice for our users. If someone out there managed to get this super generic 90% use case (HTTPS terminsation, reverse proxy, institutional certificates) implemented in a robust fashion and can share the wisdom, our users would be really happy.

I keep digging and will call back if i find a solution.

[Zanz2]

I got it to work by pinning the cvat-server image to v2.4.5 in the helm-chart file. (tag attribute of cvat/server). However I can imagine that for this to work you have to checkout your project to an older commit (around where v2.4.5 got released).

[marc31]

@kelbstf This is the solution I came up with. Although it may not be perfect, it's functional.

1. Clone the CVAT repository
2. Create a docker-compose.override.yml file. This file allows you to extend or override settings from the original docker-compose.yml.

services:
  cvat_server:
    volumes:
       # Resolve CRSF token error by mounting production.py
      - ./production.py:/home/django/cvat/settings/production.py:ro
    environment:
      # Make CVAT_HOST accessible INSIDE the Docker container
      CVAT_HOST: ${CVAT_HOST:-localhost}

3. Duplicate the production.py file located at ./cvat/settings/production.py into a new file named ./production.py.
4. Append the following line to the end of the ./production.py file:

CSRF_TRUSTED_ORIGINS = [origin.rstrip('/') for origin in os.getenv('CSRF_TRUSTED_ORIGINS', f'http://{CVAT_HOST},https://{CVAT_HOST}').split(',')]

5. Once everything is set up, export the CVAT_HOST variable with your desired domain and bring up the Docker containers:

export CVAT_HOST="mydomain.com"
docker compose up -d

[alif-munim]

@kelbstf This is the solution I came up with. Although it may not be perfect, it's functional.

1. Clone the CVAT repository
2. Create a docker-compose.override.yml file. This file allows you to extend or override settings from the original docker-compose.yml.

services:
  cvat_server:
    volumes:
       # Resolve CRSF token error by mounting production.py
      - ./production.py:/home/django/cvat/settings/production.py:ro
    environment:
      # Make CVAT_HOST accessible INSIDE the Docker container
      CVAT_HOST: ${CVAT_HOST:-localhost}

3. Duplicate the production.py file located at ./cvat/settings/production.py into a new file named ./production.py.
4. Append the following line to the end of the ./production.py file:

CSRF_TRUSTED_ORIGINS = [origin.rstrip('/') for origin in os.getenv('CSRF_TRUSTED_ORIGINS', f'http://{CVAT_HOST},https://{CVAT_HOST}').split(',')]

5. Once everything is set up, export the CVAT_HOST variable with your desired domain and bring up the Docker containers:

export CVAT_HOST="mydomain.com"
docker compose up -d

This was the only thing that worked after hours of searching - thank you so much!

[marc31]

I have not tried it yet, but take a look at: https://github.com/cvat-ai/cvat/pull/6322#issuecomment-2257131513

[hiseulgi]

@kelbstf This is the solution I came up with. Although it may not be perfect, it's functional.

1. Clone the CVAT repository
2. Create a docker-compose.override.yml file. This file allows you to extend or override settings from the original docker-compose.yml.

services:
  cvat_server:
    volumes:
       # Resolve CRSF token error by mounting production.py
      - ./production.py:/home/django/cvat/settings/production.py:ro
    environment:
      # Make CVAT_HOST accessible INSIDE the Docker container
      CVAT_HOST: ${CVAT_HOST:-localhost}

3. Duplicate the production.py file located at ./cvat/settings/production.py into a new file named ./production.py.
4. Append the following line to the end of the ./production.py file:

CSRF_TRUSTED_ORIGINS = [origin.rstrip('/') for origin in os.getenv('CSRF_TRUSTED_ORIGINS', f'http://{CVAT_HOST},https://{CVAT_HOST}').split(',')]

5. Once everything is set up, export the CVAT_HOST variable with your desired domain and bring up the Docker containers:

export CVAT_HOST="mydomain.com"
docker compose up -d

Thank you so much for this solution!