cvat
cvat copied to clipboard
cvat-ai
Helm Chart Install Showing "Could not check authorization on the server"
My actions before raising this issue
- [x] Read/searched the docs
- [x] Searched past issues
Installing CVAT using the helm chart within the repo produces an instance of CVAT that does not allow for login due to the error "Could not check authorization on the server"
Context
I followed the directions under the helm-chart README file, and was able to successfully run the command helm upgrade cvat -i --create-namespace ./helm-chart -f ./helm-chart/values.yaml I elected not to use a values-override and have made no changes to the values.yaml file - my understanding is that as I am naming the release "cvat", the secret created by "cvat-postgres-secret.yaml" will be used as the existing secret defined by the global: postgresql: existingsecret portion of values.yaml
After running the command to install the helm chart, each of the respective parts of the kubernetes cluster show up as healthy and running. I run two commands: kubectl port-forward service/cvat-backend-service 8080:8080 and kubectl port-forward service/cvat-frontend-service 8090:80, and navigate to localhost:8090. Although the login screen shows up mostly fine, there is immediately an error in the top right before I try to interact with it at all - I have included the console page of inspect page as well:
Clicking on "Create a user" takes me to this page with the following more detailed error:
TypeError n.map is not a function
TypeError: n.map is not a function at Mi (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:372:438341) at Ko (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:57930) at Ri (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:66791) at Di (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:66610) at Ai (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:66293) at gs (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:107698) at cl (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:96717) at sl (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:96642) at Qs (http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:93672) at http://localhost:8090/assets/cvat-ui.1cf4c5e4c7b29df87885.min.js:346:45314
I also have included the networking tab in this screenshot:
I have tried uncommenting the ingress part of values.yaml and enabling it, but that does not seem to solve my issue either. I am struggling at what to try now. I have seen that "n.map is not a function" error in some previous issues related to Kubernetes deployments, but from the related PR's and comments on those issues it appears that that problem should have been solved by the updated Helm Chart that is currently there, as it was related to OPA not being included in the helm chart. I assume that I am not properly setting something in values.yaml or otherwise messing something up but I would love to get some advice on where to go from here.
@se-wo, you mentioned an issue containing this exact same error in your PR https://github.com/openvinotoolkit/cvat/pull/4448. Do you have any tips here, or is there any other change that I should have made to values.yaml (or other associated files) to fix this?
Hello, your issue is related to improper ingress configuration, your need to modify lines with angle brackets at least (i.e. - host: <your_domain>, etc).
We have the exact same issue. Enabling the ingress part with proper values or not doesn't make a difference unfortunately. We installed it exactly the way it is supposed to be and still receive the error. Creating a super user doesn't solve it either, it simply gives us a login error then. The helm chart just seems to be broken at this moment and definitely needs another update.
Are there any plans for this in the nearest future?
@JulianLMIS I was able to solve this error by installing ingress locally from https://github.com/kubernetes/ingress-nginx, and then changing my ingress and tls rules in values.yaml to the following - I needed to remove the "kubernetes.io/ingress.class: nginx" line from annotations to get it to install correctly, as well as change a few other values around. With this going to localhost:80 shows the working CVAT tool with local installation.
ingress: enabled: true annotations: kubernetes.io/tls-acme: "true" ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/secure-backends: "true" nginx.ingress.kubernetes.io/proxy-body-size: "0" nginx.ingress.kubernetes.io/proxy-send-timeout: "120" nginx.ingress.kubernetes.io/proxy-read-timeout: "120" cert-manager.io/cluster-issuer: issuers.example.io hosts: - host: localhost paths: - path: "/api/.|git/.|tensorflow/.|auto_annotation/.|analytics/.|static/.|admin|admin/.|documentation/.|dextr/.|reid/." pathType: "Prefix" service: name: cvat-backend-service port: 8080 - path : "/" pathType: "Prefix" service: name: cvat-frontend-service port: 80
tls: - hosts: - localhost secretName: ingress-tls-cvat
@jgorel Thanks for your help so far! I should've been more precise: We're trying to run cvat in an Openshift cluster, but we're receiving the same error there. I tried using your port forward method combined with your ingress values, but it didn't work out for us unfortunately. Though I do have the same intuition as you that there might be some kind of error regarding the nginx controller, so I already deployed it manually beforehand on our cluster.
As of now, I'm really not sure anymore whether it's an error due to cvat or due to the controller.
@azhavoro: is there any other hint that you could give us?
Facing the same issue while trying to deploy in RKE2 Kubernetes environment. Tried creating ingress and still facing same issue.
@Julian-Marco @jgorel Did you guys face error mentioned in https://github.com/openvinotoolkit/cvat/issues/4674 ?
@jgorel Could you tell me which values have you changed ? I still facing the same issue, thanks !
Facing a similar issue and it's related to having a wrong password to the postgres db. I've tried few options from values.yaml directly on the postres pod but nothing worked so far. Has anyone solved the issue?
Same issue here, even the createsuperuser don't work.
?: (urls.W005) URL namespace 'v1' isn't unique. You may not be able to reverse all URLs in this namespace
Traceback (most recent call last):
File "/opt/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/opt/venv/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/opt/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect
self.connection = self.get_new_connection(conn_params)
File "/opt/venv/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/opt/venv/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection
connection = Database.connect(**conn_params)
File "/opt/venv/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: password authentication failed for user "cvat"
Same issue here, even the createsuperuser don't work.
The issue here is that helm-charts use a new postgres (>=14) which by default uses a different password hashing method. In docker-compose postgres:10 is used. That's why it's not working.
I haven't found a way to get it working so far and have gone back to using k8s files instead.
I updated and now scripts could create superuser:
- in "Chart.yaml", update postgresql chart version.
version: "10.16.*"
- update values
postgresql:
#See https://github.com/bitnami/charts/blob/master/bitnami/postgresql/ for more info
enabled: true # false for external db
external:
host: 127.0.0.1
port: 5432
user: postgres
password: postgres
dbname: cvat
# If not external following config will be applied by default
global:
postgresql:
postgresqlPassword: "cvat_postgres"
# doesn't work
# existingSecret: cvat-postgres-secret
secret:
create: true
name: postgres-secret
# doesn't work, generate randomly
# superuser postgres password.
# postgres_password: cvat_postgresql_postgres
# replication_password: cvat_postgresql_replica
postgresqlDatabase: cvat
postgresqlUsername: cvat
# postgresqlPassword: cvat_postgres
# servicePort: 5432
service:
port: 5432
- update cat-postgres-secret.yml
{{- if .Values.postgresql.secret.create }}
apiVersion: v1
kind: Secret
metadata:
name: "{{ .Release.Name }}-{{ .Values.postgresql.secret.name }}"
namespace: {{ .Release.Namespace }}
labels:
{{- include "cvat.labels" . | nindent 4 }}
type: generic
stringData:
postgresql-hostname: "{{ .Release.Name }}-postgresql"
postgresql-database: {{ .Values.postgresql.postgresqlDatabase }}
postgresql-username: {{ .Values.postgresql.postgresqlUsername }}
postgresql-password: {{ .Values.postgresql.global.postgresql.postgresqlPassword }}
# don't work here, ignore
# postgresql-postgres-password: {{ .Values.postgresql.secret.postgres_password }}
# postgresql-replication-password: {{ .Values.postgresql.secret.replication_password }}
{{- end }}
now open ingress setting, time to try @jgorel way:
ingress:
enabled: true
# Example for nginx ingress and cert manager
annotations:
# kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/proxy-send-timeout: "120"
nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
cert-manager.io/cluster-issuer: issuers.example.io
hosts:
- host: localhost
paths:
- path: "/api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.*"
pathType: "Prefix"
service:
name: cvat3-backend-service
port: 8080
- path : "/"
pathType: "Prefix"
service:
name: cvat3-frontend-service
port: 80
tls:
- hosts:
- localhost
secretName: ingress-tls-cvat
If you dont want to use ingress, use a nginx reverse proxy and route
/api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.* to cvat-backend-service and / to cvat-frontend-service
