label-studio-ml-backend
label-studio-ml-backend copied to clipboard
Provided examples won't start
I am using 7a4e3153d682c33e92740b6e0b24e1dc9e6bc6b6 and trying to spin up simple text classifier. The docker compose works OK, but I can't connect to http://localhost:9090/health.
>docker compose up
[+] Running 2/0
- Container redis Running 0.0s
- Container server Created 0.0s
Attaching to redis, server
server | => LABEL STUDIO HOSTNAME = http://localhost:8082
Nothing happens if I open http://localhost:9090/health
or http://localhost:9090
.
Version 1.0.3 and commit a1dc0bc runs (however gives internal server error
).
Could you please use the latest version from the master? (It's 1.0.7 as I remember)
Hi! 7a4e315 is the latest one. I just used an older one for comparison.
Hi @fralik Could you please find logs from your ML backend? It should be somewhere near with docker file in logs directory.
There are unfortunately no logs. I see logs
folder. Sometimes when I enter it, I see files popping up with name like uwsgi-2123a
. However, the files appear and then they are deleted immediately after, so I have no chance to look into. This happens first time I enter logs folder after I bring up the container. After this initial log appearance/disappearance, the logs folder stays empty. In order to see these strange files, I need to restart the container.
It looks to me that there is some misconfiguration in how uswgi is used. However, I do not see any obvious problems.
After playing with CMD command in docker file I can say two thins:
- starting a development server works fine.
- Changing cmd command to smth like
CMD ["gunicorn", "--bind", "0.0.0.0:9090", "--workers", "1", "--timeout", "0", "_wsgi:app"]
resulted in more logs in the console. The gunicorn keeps restarting:
server | [2022-06-04 08:15:33 +0000] [1] [INFO] Starting gunicorn 20.1.0
server | [2022-06-04 08:15:33 +0000] [1] [INFO] Listening at: http://0.0.0.0:9090 (1)
server | [2022-06-04 08:15:33 +0000] [1] [INFO] Using worker: sync
server | [2022-06-04 08:15:33 +0000] [9] [INFO] Booting worker with pid: 9
server | => LABEL STUDIO HOSTNAME = http://localhost:8082
server | After class
server | [2022-06-04 08:15:34 +0000] [24] [INFO] Booting worker with pid: 24
server | => LABEL STUDIO HOSTNAME = http://localhost:8082
server | After class
server | [2022-06-04 08:15:36 +0000] [39] [INFO] Booting worker with pid: 39
server | => LABEL STUDIO HOSTNAME = http://localhost:8082
server | After class
server | [2022-06-04 08:15:37 +0000] [54] [INFO] Booting worker with pid: 54
server | => LABEL STUDIO HOSTNAME = http://localhost:8082
server | After class
server | [2022-06-04 08:15:38 +0000] [69] [INFO] Booting worker with pid: 69
and so on..
@fralik Did you modify the example? How much memory is allocated to docker service?
@fralik
Do you run it on Windows?
Could you please replace last command in docker file to:
CMD exec python3 _wsgi.py
and check one more time?
Hello, I am also having this issue using Docker with WSL. Using CMD exec python3 _wsgi.py
works.
This is really weird because there shouldn't be any difference between Windows and other platforms as gunicorn is running in a Docker container.
Hi @phungthomas Could you please add --preload to gunicorn cmd and try to execute it?
Using CMD exec python3 _wsgi.py works.
You can leave as it is if you don't have requirement to use gunicorn
Thank you for your response.
Adding "--preload" unfortunately does not work (it is already included with the generated Dockerfile). The server doesn't crash but it just doesn't respond.
Using something else like "waitress" does work, and I don't really need gunicorn so it's fine for me. It just seems weird that it doesn't work through Docker.