Error: Request failed with status code 502. "Bad Gateway".

[isagastiberri]

My actions before raising this issue

• [x] Read/searched the docs
• [x] Searched past issues

I had CVAT running on a machine that unexpectedly shut down. Now, when I do the docker-compose up and I try to access it I get this errors We are not sure what is happening, maybe the app lost connection to the database but we don't understand why. Except for the up and down we have not changed anything from the docker-compose that was working earlier today.

I hope you can help us. Thanks a lot in advance

[leeyh20]

Hello, I am also facing the same issue. We have not changed any settings，in fact traefik reports that everything is fine. We checked the logs and there's nothing unusual with the logs. But no matter what settings we change, cvat still reports this error today.

It only started happening today.

[azhavoro]

@isagastiberri Hi, please provide output of docker ps command and log file from cvat container docker logs cvat > cvat.log

[isagastiberri]

@azhavoro The output from docker ps:

Also, find attached the logs from cvat cvat_server.log

Looking a bit at them I noticed that the last logs are from before it broke, I'm not sure why but there are no logs showing for after the shut down

[azhavoro]

@isagastiberri your log looks very strange... I only see a lot of decoding exceptions and no messages about server startup, try to run docker-compose down and docker-compose up -d. If that doesn't help, collect the logs again. Also I find no space left on device error, please make sure that you have some free space on your disk.

[isagastiberri]

Sorry, I realized I wasn't looking at the correct logs, here you can find attached the newest ones, I just did a docker.compose down and up again.

cvat_restart.log

I do have enough space in the disk now, we solved that mistake. I've also done docker compose down and up several times before, as whenever we had this same error that was the way to solve it, but it's not working this time.

[azhavoro]

Seems something wrong with your database, please attach logs from postgres container as well: cvat logs cvat_db > cvat_db.log Thanks.

[nrudakov]

@azhavoro I'm facing similar issue, here is my logs from cvat_db

docker logs cvat_db --details

 PostgreSQL Database directory appears to contain a database; Skipping initialization

 2021-12-01 13:29:22.723 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
 2021-12-01 13:29:22.723 UTC [1] LOG:  listening on IPv6 address "::", port 5432
 2021-12-01 13:29:22.725 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
 2021-12-01 13:29:22.738 UTC [21] LOG:  database system was shut down at 2021-12-01 13:29:13 UTC
 2021-12-01 13:29:22.744 UTC [1] LOG:  database system is ready to accept connections
 2021-12-01 13:29:24.766 UTC [28] LOG:  incomplete startup packet

[azhavoro]

@azhavoro I'm facing similar issue, here is my logs from cvat_db

docker logs cvat_db --details

 PostgreSQL Database directory appears to contain a database; Skipping initialization

 2021-12-01 13:29:22.723 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
 2021-12-01 13:29:22.723 UTC [1] LOG:  listening on IPv6 address "::", port 5432
 2021-12-01 13:29:22.725 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
 2021-12-01 13:29:22.738 UTC [21] LOG:  database system was shut down at 2021-12-01 13:29:13 UTC
 2021-12-01 13:29:22.744 UTC [1] LOG:  database system is ready to accept connections
 2021-12-01 13:29:24.766 UTC [28] LOG:  incomplete startup packet

Your database is ok, please attach cvat container logs as well

[northanapon]

I have the same 502 Bad Gateway issue when I tried to upgrade CVAT today. I tried removing the old images and volumes before reinstalling them, but still 502. The Traefik dashboard works though.

This is my docker ps:

CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                                                                                          NAMES
9a74c6dba542   openvino/cvat_ui       "/docker-entrypoint.…"   20 minutes ago   Up 20 minutes   80/tcp                                                                                         cvat_ui
879d7ffd53c5   openvino/cvat_server   "/usr/bin/supervisord"   20 minutes ago   Up 20 minutes   8080/tcp                                                                                       cvat
b84d44a9a853   traefik:v2.4           "/entrypoint.sh --pr…"   20 minutes ago   Up 20 minutes   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 80/tcp, 0.0.0.0:8090->8090/tcp, :::8090->8090/tcp   traefik
a6bca68f8b3c   postgres:10-alpine     "docker-entrypoint.s…"   20 minutes ago   Up 20 minutes   5432/tcp                                                                                       cvat_db
72da1f3d7f70   redis:4.0-alpine       "docker-entrypoint.s…"   20 minutes ago   Up 20 minutes   6379/tcp                                                                                       cvat_redis

Here are the log files:

• cvat_db.log
• cvat.log
• traefik.log
• cvat_ui.log

[northanapon]

My issue was resolved.

My machine is behind an organization's proxy. Removing proxy setting from .docker/config.json and then restarting the docker (down and up) solves the problem.

[DP1701]

Hello all, i also get this error message, but I can still log in and use CVAT.

To install CVAT, I followed the installation instructions for macOS Mojave (link). The difference is that I modified the docker-compose.yml file (starting at line 110) and added the docker-compose.override.yml file. The goal is to store the CVAT data on an external disk and to have a share path for uploading. I have attached both files. Unfortunately I don't know why the error appears before the login.

I currently have version 1.7.0 installed.

docker-compose.yml.txt docker-compose.override.yml.txt

cvat.log cvat_db.log cvat_ui.log

[sinnud]

I met the same error for develop branch. Then I changed to CVAT with tag v2.0.0-alpha and the error disappeared. Looks the develop branch have extra code to raise this issue.

[yustiks]

I couldn't find 'v2.0.0-alpha', I assume it is 'release-2.0.0'?

[sinnud]

@yustiks Looks v2.0.0-alpha is gone because release-2.0.0 is ready. Wish this bug is gone.

[yustiks]

@yustiks Looks v2.0.0-alpha is gone because release-2.0.0 is ready. Wish this bug is gone.

I still have this issue with release-2.0.0

[nmanovic]

@yustiks , please attach your logs.

[yustiks]

@yustiks , please attach your logs.

[yustiks]

@yustiks , please attach your logs.

and 'docker ps'

[nmanovic]

@yustiks , could you please create a separate issue? I see an exception in UI but I believe it is not 502 bad gateway error. Right?

@bsekachev , could you please look at the exception? What can be a reason for that?

[bsekachev]

@nmanovic

There was recent issue with similar exception #4388 The instance was incorrectly configured there. Need to work with client-side logs and requests details, to say more.

[bsekachev]

I will close the issue for now. Do not hesitate to reopen if you have any updates.