label-studio-ml-backend icon indicating copy to clipboard operation
label-studio-ml-backend copied to clipboard

Published 20 hours ago • verified publisher icon HumanSignal

I couldn't pass the validation when add Model.

Open midi79 opened this issue 3 years ago • 5 comments

Hi, I'd like to test the label studio backend so I installed MM detection on my Mac with docker-compose. The mmdetection server run very well but I couldn't use it in my label studio project.

The server log is below.

**(labelstudio) (base) midi79@Cheolkyuui-iMac mmdetection % docker-compose up
redis is up-to-date
server is up-to-date
Attaching to redis, server
server    | 2022-06-27 13:05:51,845 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
server    | 2022-06-27 13:05:51,852 INFO RPC interface 'supervisor' initialized
server    | 2022-06-27 13:05:51,852 CRIT Server 'inet_http_server' running without any HTTP authentication checking
server    | 2022-06-27 13:05:51,852 INFO supervisord started with pid 1
server    | 2022-06-27 13:05:52,858 INFO spawned: 'rq_00' with pid 10
server    | 2022-06-27 13:05:52,861 INFO spawned: 'wsgi' with pid 11
server    | 2022-06-27 13:05:53,951 INFO success: rq_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
server    | 2022-06-27 13:05:53,951 INFO success: wsgi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
redis     | 1:C 27 Jun 2022 13:05:51.427 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis     | 1:C 27 Jun 2022 13:05:51.427 # Redis version=7.0.2, bits=64, commit=00000000, modified=0, pid=1, just started
redis     | 1:C 27 Jun 2022 13:05:51.427 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis     | 1:M 27 Jun 2022 13:05:51.428 * monotonic clock: POSIX clock_gettime
redis     | 1:M 27 Jun 2022 13:05:51.428 * Running mode=standalone, port=6379.
redis     | 1:M 27 Jun 2022 13:05:51.428 # Server initialized
redis     | 1:M 27 Jun 2022 13:05:51.428 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis     | 1:M 27 Jun 2022 13:05:51.429 * Ready to accept connections
스크린샷 2022-06-27 오후 10 23 43

As you can see upper image, a validation error has occurred. How can I pass this validation?

The server was running as below image. 스크린샷 2022-06-27 오후 10 25 49

I will wait for your help. Thank you!

midi79 avatar Jun 27 '22 13:06 midi79

It works when I run with the python directly as below. "pip install -U label-studio" "label-studio"

스크린샷 2022-06-28 오후 10 08 30

However, I don't know how to use "Auto-annotation" when I use the labeling function. Can you show me any instructions or guides?

midi79 avatar Jun 28 '22 13:06 midi79

@midi79 https://labelstud.io/guide/ml.html#Get-interactive-preannotations check this article please. And maybe this - https://labelstud.io/blog/release-130.html#Selective-preannotation-with-images

makseq avatar Jun 29 '22 00:06 makseq

First, I really appreciate your reply. However, despite I have set a "smart=true" option in the Label interface like below, it didn't work.

<View>
  <Image name="image" value="$image"/>
  <RectangleLabels name="label" toName="image" smart="true">    
  <Label value="Dog" predicted_values="dog" background="#FFA39E"/></RectangleLabels>
</View>

And then I could see the error log at the label studio console like below.

[2022-06-30 13:31:26,214] [ml.models::interactive_annotating::278] [WARNING] Prediction not created for project  (id=1, url=http://localhost:9090): 500 Server Error: INTERNAL SERVER ERROR for url: http://localhost:9090/predict
[30/Jun/2022 13:31:26] "POST /api/ml/1/interactive-annotating HTTP/1.1" 200 93
[2022-06-30 13:31:26,251] [ml.models::interactive_annotating::278] [WARNING] Prediction not created for project  (id=1, url=http://localhost:9090): 500 Server Error: INTERNAL SERVER ERROR for url: http://localhost:9090/predict
[30/Jun/2022 13:31:26] "POST /api/ml/1/interactive-annotating HTTP/1.1" 200 93`

I also saw uwsgi.log as below in the "mmdetection" backend server.

[2022-06-30 13:38:31,466] [ERROR] [label_studio_ml.exceptions::exception_f::53] Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 175, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 187, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f384d153a50>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 499, in send
    timeout=timeout,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 786, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /data/upload/1/c1d71d61-pexels-brixiv-6527933.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f384d153a50>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/label_studio_ml/api.py", line 32, in _predict
    predictions, model = _manager.predict(tasks, project, label_config, force_reload, try_fetch, **params)
  File "/usr/local/lib/python3.7/site-packages/label_studio_ml/model.py", line 582, in predict
    predictions = m.model.predict(tasks, **kwargs)
  File "./mmdetection.py", line 92, in predict
    image_path = self.get_local_path(image_url)
  File "/usr/local/lib/python3.7/site-packages/label_studio_ml/model.py", line 322, in get_local_path
    return get_local_path(url, project_dir=project_dir, hostname=self.hostname, access_token=self.access_token)
  File "/usr/local/lib/python3.7/site-packages/label_studio_tools/core/utils/io.py", line 91, in get_local_path
    r = requests.get(url, stream=True, headers=headers)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /data/upload/1/c1d71d61-pexels-brixiv-6527933.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f384d153a50>: Failed to establish a new connection: [Errno 111] Connection refused'))

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 175, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 187, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f384d153a50>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 499, in send
    timeout=timeout,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 786, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /data/upload/1/c1d71d61-pexels-brixiv-6527933.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f384d153a50>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/label_studio_ml/api.py", line 32, in _predict
    predictions, model = _manager.predict(tasks, project, label_config, force_reload, try_fetch, **params)
  File "/usr/local/lib/python3.7/site-packages/label_studio_ml/model.py", line 582, in predict
    predictions = m.model.predict(tasks, **kwargs)
  File "./mmdetection.py", line 92, in predict
    image_path = self.get_local_path(image_url)
  File "/usr/local/lib/python3.7/site-packages/label_studio_ml/model.py", line 322, in get_local_path
    return get_local_path(url, project_dir=project_dir, hostname=self.hostname, access_token=self.access_token)
  File "/usr/local/lib/python3.7/site-packages/label_studio_tools/core/utils/io.py", line 91, in get_local_path
    r = requests.get(url, stream=True, headers=headers)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /data/upload/1/c1d71d61-pexels-brixiv-6527933.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f384d153a50>: Failed to establish a new connection: [Errno 111] Connection refused'))

[pid: 25|app: 0|req: 120/120] 172.20.0.1 () {34 vars in 471 bytes} [Thu Jun 30 13:38:31 2022] POST /predict => generated 5523 bytes in 6 msecs (HTTP/1.1 500) 2 headers in 92 bytes (1 switches on core 0)

Could you give me some advice to fix this problem? Thank you!

midi79 avatar Jun 30 '22 13:06 midi79

Looks like http://localhost:9090/ is not accessible on the LS side. Can you open this url in the projects - http://localhost:9090/health or http://localhost:9090/setup ?

makseq avatar Jul 04 '22 23:07 makseq

@makseq could you kindly look here #245 I think I have the same problem. I've created a separate issue because @midi79 did not follow with docker compose solution.

TrueWodzu avatar Apr 09 '23 19:04 TrueWodzu