workspaces-issues icon indicating copy to clipboard operation
workspaces-issues copied to clipboard
verified publisher icon kasmtech

[Bug] - Many errors since upgrade from 1.16.1 to 1.17.0

Open Taomyn opened this issue 8 months ago • 1 comments

Describe the bug Since the upgrade the Workspaces admin console is reporting many errors, but everything that I use seems to be working for the time being.

Screenshots Image

Workspaces Version 1.17.0

Workspaces Installation Method Single Server

Workspace Server Information (please provide the output of the following commands):

  • uname -a
Linux KODOS 6.8.12-9-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.12-9 (2025-03-16T19:18Z) x86_64 GNU/Linux```
 - `cat /etc/os-release`
```PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
  • sudo docker info
Client: Docker Engine - Community
 Version:    28.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.5.0
    Path:     /usr/local/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 15
  Running: 10
  Paused: 0
  Stopped: 5
 Images: 37
 Server Version: 28.1.1
 Storage Driver: overlay2
  Backing Filesystem: zfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: true
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan kasmweb/sidecar:1.2 macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.12-9-pve
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 6GiB
 Name: KODOS
 ID: 98c73f41-edfa-44ea-a4e6-d8f051a6c201
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
  • sudo docker ps | grep kasm
83664245cbdf   kasmweb/proxy:1.17.0               "/docker-entrypoint.…"   2 days ago   Up 5 hours             80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp   kasm_proxy
024bcfed3a02   kasmweb/rdp-https-gateway:1.17.0   "/opt/rdpgw/rdpgw"       2 days ago   Up 5 hours (healthy)                                                     kasm_rdp_https_gateway
b089ca4cd804   kasmweb/rdp-gateway:1.17.0         "/start.sh"              2 days ago   Up 5 hours (healthy)   0.0.0.0:3389->3389/tcp, [::]:3389->3389/tcp       kasm_rdp_gateway
759c0d3a87c4   kasmweb/agent:1.17.0               "python3 /src/Provis…"   2 days ago   Up 5 hours (healthy)   4444/tcp                                          kasm_agent
eb34592a9b4f   kasmweb/share:1.17.0               "python3 /src/api_se…"   2 days ago   Up 5 hours (healthy)   8182/tcp                                          kasm_share
0e9edf228691   kasmweb/api:1.17.0                 "/bin/sh -c /usr/bin…"   2 days ago   Up 5 hours (healthy)   8080/tcp                                          kasm_api
dc9b93ad8246   kasmweb/manager:1.17.0             "python3 /src/api_se…"   2 days ago   Up 4 hours (healthy)   8181/tcp                                          kasm_manager
1d9dc5d5228f   kasmweb/kasm-guac:1.17.0           "/dockerentrypoint.sh"   2 days ago   Up 5 hours (healthy)                                                     kasm_guac
3ddd467721f5   redis:5-alpine                     "docker-entrypoint.s…"   2 days ago   Up 5 hours             6379/tcp                                          kasm_redis
29e4412d4438   kasmweb/postgres:1.17.0            "docker-entrypoint.s…"   2 days ago   Up 5 hours (healthy)   5432/tcp

Additional context Exported logs: will add link to Pastebin once they unblock the log, but here's a short example:

  {
    "host": "KODOS",
    "ingest_date": "20250424114319",
    "application": "kasm_api",
    "levelname": "ERROR",
    "funcname": null,
    "kasm_user_name": "[email protected]",
    "message": "Unhandled exception occurred\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1243, in _execute_context\n    self.dialect.do_execute(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py\", line 552, in do_execute\n    cursor.execute(statement, parameters)\npsycopg2.errors.UndefinedFile: could not load library \"/usr/local/lib/postgresql/llvmjit.so\": Error loading shared library libLLVM-16.so: No such file or directory (needed by /usr/local/lib/postgresql/llvmjit.so)\nCONTEXT:  parallel worker\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cprequest.py\", line 659, in respond\n    self._do_respond(path_info)\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cprequest.py\", line 718, in _do_respond\n    response.body = self.handler()\n                    ^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/lib/encoding.py\", line 223, in __call__\n    self.body = self.oldhandler(*args, **kwargs)\n                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/lib/jsontools.py\", line 59, in json_handler\n    value = cherrypy.serving.request._json_inner_handler(*args, **kwargs)\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cpdispatch.py\", line 54, in __call__\n    return self.callable(*self.args, **self.kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"api_server/utils.py\", line 166, in C\n  File \"api_server/admin_api.py\", line 580, in get_report\n  File \"api_server/data/access_postgres.py\", line 1240, in execute_native_query\n  File \"api_server/data/access_postgres.py\", line 1239, in execute_native_query\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/orm/session.py\", line 1267, in execute\n    return self._connection_for_bind(bind, close_with_result=True).execute(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 988, in execute\n    return meth(self, multiparams, params)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py\", line 287, in _execute_on_connection\n    return connection._execute_clauseelement(self, multiparams, params)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1101, in _execute_clauseelement\n    ret = self._execute_context(\n          ^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1247, in _execute_context\n    self._handle_dbapi_exception(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1466, in _handle_dbapi_exception\n    util.raise_from_cause(sqlalchemy_exception, exc_info)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/util/compat.py\", line 383, in raise_from_cause\n    reraise(type(exception), exception, tb=exc_tb, cause=cause)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/util/compat.py\", line 128, in reraise\n    raise value.with_traceback(tb)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1243, in _execute_context\n    self.dialect.do_execute(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py\", line 552, in do_execute\n    cursor.execute(statement, parameters)\nsqlalchemy.exc.OperationalError: (psycopg2.errors.UndefinedFile) could not load library \"/usr/local/lib/postgresql/llvmjit.so\": Error loading shared library libLLVM-16.so: No such file or directory (needed by /usr/local/lib/postgresql/llvmjit.so)\nCONTEXT:  parallel worker\n\n[SQL: \n\t\t\t\tSELECT\n\t\t\t\t  failed_login,\n\t\t\t\t  success_login,\n\t\t\t\t  RPAD(y.timeslot, 12, '0')::char(12) AS _time\n\t\t\t\tFROM\n\t\t\t\t(\n\t\t\t\t  SELECT\n\t\t\t\t\t  SUM(x.failed_login) as failed_login,\n\t\t\t\t\t  SUM(x.success_login) as success_login,\n\t\t\t\t\t  CASE\n\t\t\t\t\t\t  WHEN 1440 = 1 THEN substring(log_minute, 0, 13)\n\t\t\t\t\t\t  WHEN 1440 = 10 THEN SUBSTRING(log_minute, 0, 12)\n\t\t\t\t\t\t  WHEN 1440 = 60 THEN SUBSTRING(log_minute, 0, 11)\n\t\t\t\t\t\t  WHEN 1440 = 1440 THEN SUBSTRING(log_minute, 0, 9)\n\t\t\t\t\t\t  ELSE SUBSTRING(log_minute, 0, 11) --default 1 hour resolution\n\t\t\t\t\t  END AS timeslot\n\t\t\t\t  FROM\n\t\t\t\t  (\n\t\t\t\t\t  select\n\t\t\t\t\t\tCASE WHEN (\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_invalid_password' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_invalid_user' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_locked' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_email_not_verified' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_ldap_error' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_saml_only'\n\t\t\t\t\t\t\t\t   ) THEN 1 ELSE 0 END AS failed_login,\n\t\t\t\t\t\tCASE WHEN (\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.successful'\n\t\t\t\t\t\t\t\t) THEN 1 ELSE 0 END AS success_login,\t\t\t                \n\t\t\t\t\t\tto_char(logs.ingest_date, 'YYYYMMDDHH24MI'::text) as log_minute\n\t\t\t\t\t  from logs\n\t\t\t\t\t  WHERE \t\t\t               \n\t\t\t\t\t\tingest_date < to_timestamp('20250424 11:43', 'YYYYMMDD HH24:MI') AND\n\t\t\t\t\t\tingest_date > to_timestamp('20250417 11:43', 'YYYYMMDD HH24:MI') \n\t\t\t\t\t  ORDER BY ingest_date DESC\n\t\t\t\t  ) AS x\n\t\t\t\t  GROUP BY\n\t\t\t\t\t  timeslot\n\t\t\t\t) AS y\n\t\t\t\tORDER BY _time;\n    \t\t]\n(Background on this error at: http://sqlalche.me/e/e3q8)",
    "traceback": null,
    "error_stack": null,
    "process": "root",
    "client_ip": "192.168.1.30",
    "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
    "allow": null,
    "url": null,
    "site": null,
    "domain": null,
    "category": null
  },
  {
    "host": "KODOS",
    "ingest_date": "20250424114319",
    "application": "kasm_api",
    "levelname": "ERROR",
    "funcname": null,
    "kasm_user_name": "[email protected]",
    "message": "[24/Apr/2025:11:43:19] HTTP \nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1243, in _execute_context\n    self.dialect.do_execute(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py\", line 552, in do_execute\n    cursor.execute(statement, parameters)\npsycopg2.errors.UndefinedFile: could not load library \"/usr/local/lib/postgresql/llvmjit.so\": Error loading shared library libLLVM-16.so: No such file or directory (needed by /usr/local/lib/postgresql/llvmjit.so)\nCONTEXT:  parallel worker\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cprequest.py\", line 659, in respond\n    self._do_respond(path_info)\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cprequest.py\", line 718, in _do_respond\n    response.body = self.handler()\n                    ^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/lib/encoding.py\", line 223, in __call__\n    self.body = self.oldhandler(*args, **kwargs)\n                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/lib/jsontools.py\", line 59, in json_handler\n    value = cherrypy.serving.request._json_inner_handler(*args, **kwargs)\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cpdispatch.py\", line 54, in __call__\n    return self.callable(*self.args, **self.kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"api_server/utils.py\", line 166, in C\n  File \"api_server/admin_api.py\", line 580, in get_report\n  File \"api_server/data/access_postgres.py\", line 1240, in execute_native_query\n  File \"api_server/data/access_postgres.py\", line 1239, in execute_native_query\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/orm/session.py\", line 1267, in execute\n    return self._connection_for_bind(bind, close_with_result=True).execute(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 988, in execute\n    return meth(self, multiparams, params)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py\", line 287, in _execute_on_connection\n    return connection._execute_clauseelement(self, multiparams, params)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1101, in _execute_clauseelement\n    ret = self._execute_context(\n          ^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1247, in _execute_context\n    self._handle_dbapi_exception(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1466, in _handle_dbapi_exception\n    util.raise_from_cause(sqlalchemy_exception, exc_info)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/util/compat.py\", line 383, in raise_from_cause\n    reraise(type(exception), exception, tb=exc_tb, cause=cause)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/util/compat.py\", line 128, in reraise\n    raise value.with_traceback(tb)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1243, in _execute_context\n    self.dialect.do_execute(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py\", line 552, in do_execute\n    cursor.execute(statement, parameters)\nsqlalchemy.exc.OperationalError: (psycopg2.errors.UndefinedFile) could not load library \"/usr/local/lib/postgresql/llvmjit.so\": Error loading shared library libLLVM-16.so: No such file or directory (needed by /usr/local/lib/postgresql/llvmjit.so)\nCONTEXT:  parallel worker\n\n[SQL: \n\t\t\t\tSELECT\n\t\t\t\t  failed_login,\n\t\t\t\t  success_login,\n\t\t\t\t  RPAD(y.timeslot, 12, '0')::char(12) AS _time\n\t\t\t\tFROM\n\t\t\t\t(\n\t\t\t\t  SELECT\n\t\t\t\t\t  SUM(x.failed_login) as failed_login,\n\t\t\t\t\t  SUM(x.success_login) as success_login,\n\t\t\t\t\t  CASE\n\t\t\t\t\t\t  WHEN 1440 = 1 THEN substring(log_minute, 0, 13)\n\t\t\t\t\t\t  WHEN 1440 = 10 THEN SUBSTRING(log_minute, 0, 12)\n\t\t\t\t\t\t  WHEN 1440 = 60 THEN SUBSTRING(log_minute, 0, 11)\n\t\t\t\t\t\t  WHEN 1440 = 1440 THEN SUBSTRING(log_minute, 0, 9)\n\t\t\t\t\t\t  ELSE SUBSTRING(log_minute, 0, 11) --default 1 hour resolution\n\t\t\t\t\t  END AS timeslot\n\t\t\t\t  FROM\n\t\t\t\t  (\n\t\t\t\t\t  select\n\t\t\t\t\t\tCASE WHEN (\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_invalid_password' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_invalid_user' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_locked' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_email_not_verified' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_ldap_error' OR\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.failed_saml_only'\n\t\t\t\t\t\t\t\t   ) THEN 1 ELSE 0 END AS failed_login,\n\t\t\t\t\t\tCASE WHEN (\n\t\t\t\t\t\t\t\tmetric_name = 'account.login.successful'\n\t\t\t\t\t\t\t\t) THEN 1 ELSE 0 END AS success_login,\t\t\t                \n\t\t\t\t\t\tto_char(logs.ingest_date, 'YYYYMMDDHH24MI'::text) as log_minute\n\t\t\t\t\t  from logs\n\t\t\t\t\t  WHERE \t\t\t               \n\t\t\t\t\t\tingest_date < to_timestamp('20250424 11:43', 'YYYYMMDD HH24:MI') AND\n\t\t\t\t\t\tingest_date > to_timestamp('20250417 11:43', 'YYYYMMDD HH24:MI') \n\t\t\t\t\t  ORDER BY ingest_date DESC\n\t\t\t\t  ) AS x\n\t\t\t\t  GROUP BY\n\t\t\t\t\t  timeslot\n\t\t\t\t) AS y\n\t\t\t\tORDER BY _time;\n    \t\t]\n(Background on this error at: http://sqlalche.me/e/e3q8)",
    "traceback": null,
    "error_stack": null,
    "process": "cherrypy.error.135280809415968",
    "client_ip": "192.168.1.30",
    "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
    "allow": null,
    "url": null,
    "site": null,
    "domain": null,
    "category": null
  },
  {
    "host": "KODOS",
    "ingest_date": "20250424114319",
    "application": "kasm_api",
    "levelname": "ERROR",
    "funcname": null,
    "kasm_user_name": "[email protected]",
    "message": "[24/Apr/2025:11:43:19] HTTP \nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1243, in _execute_context\n    self.dialect.do_execute(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py\", line 552, in do_execute\n    cursor.execute(statement, parameters)\npsycopg2.errors.UndefinedFile: could not load library \"/usr/local/lib/postgresql/llvmjit.so\": Error loading shared library libLLVM-16.so: No such file or directory (needed by /usr/local/lib/postgresql/llvmjit.so)\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cprequest.py\", line 659, in respond\n    self._do_respond(path_info)\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cprequest.py\", line 718, in _do_respond\n    response.body = self.handler()\n                    ^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/lib/encoding.py\", line 223, in __call__\n    self.body = self.oldhandler(*args, **kwargs)\n                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/lib/jsontools.py\", line 59, in json_handler\n    value = cherrypy.serving.request._json_inner_handler(*args, **kwargs)\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/cherrypy/_cpdispatch.py\", line 54, in __call__\n    return self.callable(*self.args, **self.kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"api_server/utils.py\", line 166, in C\n  File \"api_server/admin_api.py\", line 580, in get_report\n  File \"api_server/data/access_postgres.py\", line 1240, in execute_native_query\n  File \"api_server/data/access_postgres.py\", line 1239, in execute_native_query\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/orm/session.py\", line 1267, in execute\n    return self._connection_for_bind(bind, close_with_result=True).execute(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 988, in execute\n    return meth(self, multiparams, params)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py\", line 287, in _execute_on_connection\n    return connection._execute_clauseelement(self, multiparams, params)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1101, in _execute_clauseelement\n    ret = self._execute_context(\n          ^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1247, in _execute_context\n    self._handle_dbapi_exception(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1466, in _handle_dbapi_exception\n    util.raise_from_cause(sqlalchemy_exception, exc_info)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/util/compat.py\", line 383, in raise_from_cause\n    reraise(type(exception), exception, tb=exc_tb, cause=cause)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/util/compat.py\", line 128, in reraise\n    raise value.with_traceback(tb)\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py\", line 1243, in _execute_context\n    self.dialect.do_execute(\n  File \"/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py\", line 552, in do_execute\n    cursor.execute(statement, parameters)\nsqlalchemy.exc.OperationalError: (psycopg2.errors.UndefinedFile) could not load library \"/usr/local/lib/postgresql/llvmjit.so\": Error loading shared library libLLVM-16.so: No such file or directory (needed by /usr/local/lib/postgresql/llvmjit.so)\n\n[SQL: \n\t\t\t\tSELECT\n\t\t\t\t  kasms,\n\t\t\t\t  RPAD(y.timeslot, 12, '0')::char(12) AS _time\n\t\t\t\tFROM\n\t\t\t\t(\n\t\t\t\t  SELECT\n\t\t\t\t\t  SUM(x.kasm) as kasms,\n\t\t\t\t\t  CASE\n\t\t\t\t\t\t  WHEN 1440 = 1 THEN substring(log_minute, 0, 13)\n\t\t\t\t\t\t  WHEN 1440 = 10 THEN SUBSTRING(log_minute, 0, 12)\n\t\t\t\t\t\t  WHEN 1440 = 60 THEN SUBSTRING(log_minute, 0, 11)\n\t\t\t\t\t\t  WHEN 1440 = 1440 THEN SUBSTRING(log_minute, 0, 9)\n\t\t\t\t\t\t  ELSE SUBSTRING(log_minute, 0, 11) --default 1 hour resolution\n\t\t\t\t\t  END AS timeslot\n\t\t\t\t  FROM\n\t\t\t\t  (\n\t\t\t\t\t  select\n\t\t\t\t\t\tCASE WHEN (\n\t\t\t\t\t\t\t\tmetric_name = 'provision.create'\n\t\t\t\t\t\t\t\t   ) THEN 1 ELSE 0 END AS kasm,\t                \n\t\t\t\t\t\tto_char(logs.ingest_date, 'YYYYMMDDHH24MI'::text) as log_minute\n\t\t\t\t\t  from logs\n\t\t\t\t\t  WHERE \t\t\t               \n\t\t\t\t\t\tingest_date < to_timestamp('20250424 11:43', 'YYYYMMDD HH24:MI') AND\n\t\t\t\t\t\tingest_date > to_timestamp('20250417 11:43', 'YYYYMMDD HH24:MI') \n\t\t\t\t\t  ORDER BY ingest_date DESC\n\t\t\t\t  ) AS x\n\t\t\t\t  GROUP BY\n\t\t\t\t\t  timeslot\n\t\t\t\t) AS y\n\t\t\t\tORDER BY _time;\n    \t\t]\n(Background on this error at: http://sqlalche.me/e/e3q8)",
    "traceback": null,
    "error_stack": null,
    "process": "cherrypy.error.135280809415968",
    "client_ip": "192.168.1.30",
    "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36",
    "allow": null,
    "url": null,
    "site": null,
    "domain": null,
    "category": null
  },

Taomyn avatar Apr 24 '25 12:04 Taomyn

Thank you for reporting. We are looking into it

j-travis avatar Apr 25 '25 00:04 j-travis

Any idea when a fix will be released?

Taomyn avatar Aug 27 '25 07:08 Taomyn

Yes, please try the following:

Stop Kasm services on the database VM

sudo /opt/kasm/bin/stop

Pull the latest database image

docker pull kasmweb/postgres:1.17.0

Start Kasm services

sudo /opt/kasm/bin/start

j-travis avatar Aug 27 '25 13:08 j-travis

Thanks, that fixed it and I no longer see new errors - I did it yesterday and left it overnight.

Taomyn avatar Aug 28 '25 08:08 Taomyn