cvat
cvat copied to clipboard
Published
20 hours ago •
cvat-ai
cvat-ai
Build or Run CVAT on Mac M1 with Rosetta
My actions before raising this issue
- [Y] Read/searched the docs
- [Y] Searched past issues
I'm using a Mac with M1 processor and unable to run the docker images built for AMD platform. After some reading, I came across a blog which mentioned that having Rosetta and latest version of docker will enable me to run images built for AMD on ARM.
Steps to Reproduce (for bugs)
- Followed the installation steps to run
docker compose up -dbut the cvat/server image kept restarting with logs on the lines of "Tensorflow is not built for this platform" - Then tried building the image on laptop using
docker compose -f docker-compose.yml -f docker-compose.dev.yml buildand evendocker build .but both result in errors while building stuck at the step where the wheel packages are being installed
Expected Behaviour
I expected the AMD image to run having Rosetta installed.
Your Environment
- Docker version (Docker Client 24.0.5, Docker Desktop 4.22.0)
- Are you using Docker Swarm or Kubernetes? No
- Operating System and version: MacOS
Is there anyone who has successfully built and run CVAT on Apple Silicon? I'm also open to running it without Tensorflow or any prebuilt ML models. I just need to use this as an annotation tool and make some updates on the source code.
I've gotten it to build with docker compose by setting DOCKER_DEFAULT_PLATFORM=linux/amd64 and checking out a version CVAT version <= 2.4.5.
I tried using buildx to build after modifying the config files. Sometimes I got it to appear to build completely, but ran into issues when I tried to run it.
@kaixi-wang is there such version 1.4.5? I'm trying to docker-compose up with that version but it's complaining cvat_server doesn't have that version.
I'm having issues as well. i can build on my m1, but running does not work.
Likewise it runs but the UI displays
In the logs:
redis.exceptions.ConnectionError: Error 111 connecting to cvat_redis:6379. Connection refused.
Likewise it runs but the UI displays
In the logs:
redis.exceptions.ConnectionError: Error 111 connecting to cvat_redis:6379. Connection refused.
@robmarkcole @euan-cowie Pull the latest changes in master. There was a commit a few days ago that reverted the keydb version from v6.3.4 to v6.3.2 which fixed the issue.
@BeardedUnicorn pulled latest and docker compose up now:
and logs
traefik | {"ClientAddr":"192.168.65.1:35976","ClientHost":"192.168.65.1","ClientPort":"35976","ClientUsername":"-","DownstreamContentSize":145,"DownstreamStatus":500,"Duration":38100500,"OriginContentSize":145,"OriginDuration":37996916,"OriginStatus":500,"Overhead":103584,"RequestAddr":"localhost:8080","RequestContentSize":0,"RequestCount":17,"RequestHost":"localhost","RequestMethod":"GET","RequestPath":"/api/server/health/?format=json\u0026org=","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"cvat@docker","ServiceAddr":"172.18.0.14:8080","ServiceName":"cvat@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"172.18.0.14:8080","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2023-12-15T15:57:55.248264087Z","StartUTC":"2023-12-15T15:57:55.248264087Z","entryPointName":"web","level":"info","msg":"","time":"2023-12-15T15:57:55Z"}
cvat_server | 2023-12-15 15:57:58,432 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 172.18.0.16:0 - "GET /api/auth/rules HTTP/1.0" 304 Not Modified
cvat_server |
cvat_server | 2023-12-15 15:57:59,265 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-12-15 15:57:59,263] ERROR health-check: Unexpected Error!
cvat_server | Traceback (most recent call last):
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 698, in connect
cvat_server | sock = self.retry.call_with_retry(
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
cvat_server | return do()
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 699, in <lambda>
cvat_server | lambda: self._connect(), lambda error: self.disconnect(error)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 955, in _connect
cvat_server | for res in socket.getaddrinfo(
cvat_server | File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo
cvat_server | for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
cvat_server | socket.gaierror: [Errno -2] Name or service not known
cvat_server |
cvat_server | During handling of the above exception, another exception occurred:
cvat_server |
cvat_server | Traceback (most recent call last):
cvat_server | File "/opt/venv/lib/python3.10/site-packages/health_check/backends.py", line 30, in run_check
cvat_server | self.check_status()
cvat_server | File "/home/django/cvat/apps/health/backends.py", line 44, in check_status
cvat_server | cache.set("djangohealtcheck_test", "itworks")
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/core/cache/backends/redis.py", line 191, in set
cvat_server | self._cache.set(key, value, self.get_backend_timeout(timeout))
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/core/cache/backends/redis.py", line 108, in set
cvat_server | client.set(key, value, ex=timeout)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/commands/core.py", line 2302, in set
cvat_server | return self.execute_command("SET", *pieces, **options)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/client.py", line 1255, in execute_command
cvat_server | conn = self.connection or pool.get_connection(command_name, **options)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 1442, in get_connection
cvat_server | connection.connect()
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 704, in connect
cvat_server | raise ConnectionError(self._error_message(e))
cvat_server | redis.exceptions.ConnectionError: Error -2 connecting to cvat_redis:6379. Name or service not known.
cvat_server |
cvat_server | 2023-12-15 15:57:59,290 DEBG 'uvicorn-1' stderr output:
cvat_server | [2023-12-15 15:57:59,289] ERROR django.request: Internal Server Error: /api/server/health/
cvat_server | Traceback (most recent call last):
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 698, in connect
cvat_server | sock = self.retry.call_with_retry(
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
cvat_server | return do()
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 699, in <lambda>
cvat_server | lambda: self._connect(), lambda error: self.disconnect(error)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 955, in _connect
cvat_server | for res in socket.getaddrinfo(
cvat_server | File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo
cvat_server | for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
cvat_server | socket.gaierror: [Errno -2] Name or service not known
cvat_server |
cvat_server | During handling of the above exception, another exception occurred:
cvat_server |
cvat_server | Traceback (most recent call last):
cvat_server | File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 534, in thread_handler
cvat_server | raise exc_info[1]
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 42, in inner
cvat_server | response = await get_response(request)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
cvat_server | response = await wrapped_callback(
cvat_server | File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 479, in __call__
cvat_server | ret: _R = await loop.run_in_executor(
cvat_server | File "/opt/venv/lib/python3.10/site-packages/asgiref/current_thread_executor.py", line 40, in run
cvat_server | result = self.fn(*self.args, **self.kwargs)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 538, in thread_handler
cvat_server | return func(*args, **kwargs)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 104, in view
cvat_server | return self.dispatch(request, *args, **kwargs)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 143, in dispatch
cvat_server | return handler(request, *args, **kwargs)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/utils/decorators.py", line 46, in _wrapper
cvat_server | return bound_method(*args, **kwargs)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/views/decorators/cache.py", line 62, in _wrapper_view_func
cvat_server | response = view_func(request, *args, **kwargs)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/health_check/views.py", line 91, in get
cvat_server | status_code = 500 if self.errors else 200
cvat_server | File "/opt/venv/lib/python3.10/site-packages/health_check/mixins.py", line 16, in errors
cvat_server | self._errors = self.run_check()
cvat_server | File "/opt/venv/lib/python3.10/site-packages/health_check/mixins.py", line 44, in run_check
cvat_server | for plugin in executor.map(_run, self.plugins):
cvat_server | File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
cvat_server | yield _result_or_cancel(fs.pop())
cvat_server | File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
cvat_server | return fut.result(timeout)
cvat_server | File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
cvat_server | return self.__get_result()
cvat_server | File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
cvat_server | raise self._exception
cvat_server | File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
cvat_server | result = self.fn(*self.args, **self.kwargs)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/health_check/mixins.py", line 35, in _run
cvat_server | plugin.run_check()
cvat_server | File "/opt/venv/lib/python3.10/site-packages/health_check/backends.py", line 30, in run_check
cvat_server | self.check_status()
cvat_server | File "/home/django/cvat/apps/health/backends.py", line 44, in check_status
cvat_server | cache.set("djangohealtcheck_test", "itworks")
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/core/cache/backends/redis.py", line 191, in set
cvat_server | self._cache.set(key, value, self.get_backend_timeout(timeout))
cvat_server | File "/opt/venv/lib/python3.10/site-packages/django/core/cache/backends/redis.py", line 108, in set
cvat_server | client.set(key, value, ex=timeout)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/commands/core.py", line 2302, in set
cvat_server | return self.execute_command("SET", *pieces, **options)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/client.py", line 1255, in execute_command
cvat_server | conn = self.connection or pool.get_connection(command_name, **options)
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 1442, in get_connection
cvat_server | connection.connect()
cvat_server | File "/opt/venv/lib/python3.10/site-packages/redis/connection.py", line 704, in connect
cvat_server | raise ConnectionError(self._error_message(e))
cvat_server | redis.exceptions.ConnectionError: Error -2 connecting to cvat_redis:6379. Name or service not known.
cvat_server |
cvat_server | 2023-12-15 15:57:59,290 DEBG 'uvicorn-1' stdout output:
cvat_server | INFO: 192.168.65.1:0 - "GET /api/server/health/?format=json&org= HTTP/1.0" 500 Internal Server Error
@robmarkcole Did you run docker-compose down before you ran docker-compose up? Can you docker-compose ps and post the output?
@BeardedUnicorn yes same with down first.
> docker-compose ps
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
cvat_clickhouse clickhouse/clickhouse-server:23.11-alpine "/entrypoint.sh" cvat_clickhouse 7 seconds ago Up 6 seconds 8123/tcp, 9000/tcp, 9009/tcp
cvat_db postgres:15-alpine "docker-entrypoint.s…" cvat_db 7 seconds ago Up 6 seconds 5432/tcp
cvat_grafana grafana/grafana-oss:10.1.2 "sh -euc 'mkdir -p /…" cvat_grafana 7 seconds ago Up 6 seconds 3000/tcp
cvat_opa openpolicyagent/opa:0.45.0-rootless "/opa run --server -…" cvat_opa 7 seconds ago Up 6 seconds
cvat_redis eqalpha/keydb:x86_64_v6.3.2 "docker-entrypoint.s…" cvat_redis 7 seconds ago Up Less than a second 6379/tcp
cvat_server cvat/server:dev "./backend_entrypoin…" cvat_server 7 seconds ago Up 5 seconds 8080/tcp
cvat_ui cvat/ui:dev "/docker-entrypoint.…" cvat_ui 7 seconds ago Up 5 seconds 80/tcp
cvat_utils cvat/server:dev "./backend_entrypoin…" cvat_utils 7 seconds ago Up 5 seconds 8080/tcp
cvat_vector timberio/vector:0.26.0-alpine "/usr/local/bin/vect…" cvat_vector 7 seconds ago Up 5 seconds
cvat_worker_analytics_reports cvat/server:dev "./backend_entrypoin…" cvat_worker_analytics_reports 7 seconds ago Up 6 seconds 8080/tcp
cvat_worker_annotation cvat/server:dev "./backend_entrypoin…" cvat_worker_annotation 7 seconds ago Up 6 seconds 8080/tcp
cvat_worker_export cvat/server:dev "./backend_entrypoin…" cvat_worker_export 7 seconds ago Up 6 seconds 8080/tcp
cvat_worker_import cvat/server:dev "./backend_entrypoin…" cvat_worker_import 7 seconds ago Up 5 seconds 8080/tcp
cvat_worker_quality_reports cvat/server:dev "./backend_entrypoin…" cvat_worker_quality_reports 7 seconds ago Up 6 seconds 8080/tcp
cvat_worker_webhooks cvat/server:dev "./backend_entrypoin…" cvat_worker_webhooks 7 seconds ago Up 5 seconds 8080/tcp
traefik traefik:v2.10 "/entrypoint.sh trae…" traefik 7 seconds ago Up 6 seconds 0.0.0.0:8080->8080/tcp, 80/tcp, 0.0.0.0:8090->8090/tcp
I have the similar issue to compose the docker:
CVAT_VERSION=dev DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose up -d
the response to the above command line:
Error response from daemon: image with reference traefik:v2.10 was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64/v8
Error 1: Platform-related Error
If you encounter the error:
Error response from daemon: image with reference IMAGE was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64/v8
Follow these steps:
- Remove all existing CVAT-related Docker images.
docker-compose down --rmi all
- Set the Docker default platform and start the containers:
export DOCKER_DEFAULT_PLATFORM=linux/amd64
docker compose up
- If you still see this error:
Error response from daemon: image with reference <IMAGE> was found but does not match the specified platform: wanted linux/amd64, actual: linux/arm64. Remove the problematic image and restart the compose process:
docker image rmi traefik:v2.10
docker compose up
- Continue this process until there are no more platform-related errors.
Error 2: CVAT_OPA Error
If CVAT still doesn't run successfully and you encounter errors related to cvat_opa:
cvat_opa | {"level":"error","msg":"Bundle load failed: request failed: Get \"http://cvat-server:8080/api/auth/rules\": dial tcp 192.168.0.17:8080: connect: connection refused","name":"cvat","plugin":"bundle","time":"2024-04-27T08:37:17Z"}
or
File "/home/django/cvat/apps/iam/views.py", line 168, in _etag_func
with open(file_path, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/django/static/opa/bundle.tar.gz'
Try the following:
- Bring down all services:
docker compose down
- Attempt to start cvat_opa separately:
docker compose up cvat_opa -d
- Wait for 15s then comment out the cvat_opa service in the docker-compose.yml file.
services:
...
cvat_server:
container_name: cvat_server
image: cvat/server:${CVAT_VERSION:-dev}
restart: always
depends_on:
<<: *backend-deps
# cvat_opa:
# condition:
# service_healthy
...
# cvat_opa:
# container_name: cvat_opa
# image: openpolicyagent/opa:0.63.0
# restart: always
# networks:
# cvat:
# aliases:
# - opa
# command:
# - run
# - --server
# - --log-level=error
# - --set=services.cvat.url=http://cvat-server:8080
# - --set=bundles.cvat.service=cvat
# - --set=bundles.cvat.resource=/api/auth/rules
# - --set=bundles.cvat.polling.min_delay_seconds=5
# - --set=bundles.cvat.polling.max_delay_seconds=15
Restart all services in detached mode:
docker compose up -d
These steps should help you resolve common issues when setting up CVAT on Mac M1.
