Yacht icon indicating copy to clipboard operation
Yacht copied to clipboard

[Bug Report] yacht did not respond, website gui did not respond

Open SkalCore opened this issue 3 years ago • 24 comments

Describe the bug yacht did not respond, website gui did not respond, process uvicorn generate high CPU Usage on Qnap NAS

To Reproduce Steps to reproduce the behavior: using yacht , after strarting yacht via Docker on qnap Nas Process uvicorn generate high CPU usage Restarting container did not help. Start/Stop Container did not help

Expected behavior using Yacht normal to control differnt docker container, website GUi for yacht did not respond, using Firefox last version, nothing changed in settings on yacht

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: qnap NAS OS/ Linux
  • Browser [e.g. chrome, safari] Firefox last version
  • Yacht Version [e.g. v0.0.6-alpha] last docker release

Additional context Add any other context about the problem here.

** Logs ** Add any relevant logs (from the yacht docker container or from your browser console)

SkalCore avatar Jun 01 '21 08:06 SkalCore

Can you post logs from the Yacht container on your host using the docker logs command? It’s fairly light cpu wise so should definitely not cause issues with CPU usage.

SelfhostedPro avatar Jun 01 '21 13:06 SelfhostedPro

here are the necessary logs: https://pastebin.com/NqBBQZH1

SkalCore avatar Jun 01 '21 13:06 SkalCore

Looks like the dashboard isn’t able to get the CPU usage. Is it just the dashboard that’s frozen or is it the whole GUI? It may be a good idea to revert to v0.0.7 instead of the latest hotfix. I’ll take a look at this shortly and workout a solution for you.

SelfhostedPro avatar Jun 01 '21 14:06 SelfhostedPro

https://i.imgur.com/8zmhSpT.png, its the gui and my webinterface from my Nas is very slowly...so i think i switch to an older version

SkalCore avatar Jun 01 '21 14:06 SkalCore

ok, i tried it with two older versions. the cpu load is now back to normal, but i don't get any cpu info displayed in yacht anymore. Before that everything worked fine. Until with the last release the described errors occurred.

SkalCore avatar Jun 01 '21 14:06 SkalCore

What is the current version you’re on?

SelfhostedPro avatar Jun 01 '21 14:06 SelfhostedPro

What is the current version you’re on?

v0.0.6-alpha-hf1 and v0.0.7-alpha...but with both version no CPU Info are shown on dashboard.

in the logs of both versions these error messages appear

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found`

SkalCore avatar Jun 01 '21 19:06 SkalCore

Can you try this version? It should show cpu and memory but may not show all the apps you’re running. v0.0.7-alpha-2021-05-07--12

SelfhostedPro avatar Jun 01 '21 19:06 SelfhostedPro

Can you try this version? It should show cpu and memory but may not show all the apps you’re running. v0.0.7-alpha-2021-05-07--12

sorry, no success with that either, not a single container is shown in the dashboard and no info regarding CPU is shown, in the logs the same message appears again and again

2021-06-01T19:43:47.304749846Z INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

2021-06-01T19:43:47.309330037Z INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

2021-06-01T19:43:47.310807368Z INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

2021-06-01T19:43:47.315275870Z INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

2021-06-01T19:43:47.356606717Z INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

2021-06-01T19:43:47.360347980Z INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

2021-06-01T19:43:47.362106477Z INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

2021-06-01T19:43:47.581235152Z INFO: - "GET /api/apps/stats HTTP/1.1" 404 Not Found

SkalCore avatar Jun 01 '21 19:06 SkalCore

ok, I found the error, it was due to the cookies, after I deleted the cookies, it now works with the last version

SkalCore avatar Jun 01 '21 20:06 SkalCore

Just trying out the newest Firefox (typically, I'm a Chrome user), and I've just experienced this as well.

SteveDinn avatar Jun 09 '21 14:06 SteveDinn

Ah, I'm a fool. It's still doing a request for each app even though the one endpoint serves stats for all apps. I will get some work done to fix this shortly.

SelfhostedPro avatar Jun 09 '21 16:06 SelfhostedPro

same here. current release has this problem. All I did was to either change the email from [email protected], or the password, or both. Didn't happen with the last published v0.0.6 alpha. Also you're not a fool, shit happens. Thanks for creating this!

''' version: '3.3' services: yacht: ports: - '8888:8000' volumes: - '/var/run/docker.sock:/var/run/docker.sock' - './config:/config' image: selfhostedpro/yacht container_name: yacht '''

and the docker-compose log:

yacht    | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
yacht    | [s6-init] ensuring user provided files have correct perms...exited 0.
yacht    | [fix-attrs.d] applying ownership & permissions fixes...
yacht    | [fix-attrs.d] done.
yacht    | [cont-init.d] executing container initialization scripts...
yacht    | [cont-init.d] 01-envfile: executing...
yacht    | [cont-init.d] 01-envfile: exited 0.
yacht    | [cont-init.d] 10-adduser: executing...
yacht    | usermod: no changes
yacht    |
yacht    | -------------------------------------
yacht    |           _         ()
yacht    |          | |  ___   _    __
yacht    |          | | / __| | |  /  \
yacht    |          | | \__ \ | | | () |
yacht    |          |_| |___/ |_|  \__/
yacht    |
yacht    |
yacht    | Brought to you by linuxserver.io
yacht    | -------------------------------------
yacht    |
yacht    | To support LSIO projects visit:
yacht    | https://www.linuxserver.io/donate/
yacht    | -------------------------------------
yacht    | GID/UID
yacht    | -------------------------------------
yacht    |
yacht    | User uid:    911
yacht    | User gid:    911
yacht    | -------------------------------------
yacht    |
yacht    | [cont-init.d] 10-adduser: exited 0.
yacht    | [cont-init.d] 30-config: executing...
yacht    | [cont-init.d] 30-config: exited 0.
yacht    | [cont-init.d] 31-migrate: executing...
yacht    | INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
yacht    | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
yacht    | INFO  [alembic.autogenerate.compare] Detected removed index 'ix_jwt_token_blacklist_jti' on 'jwt_token_blacklist'
yacht    | INFO  [alembic.autogenerate.compare] Detected removed table 'jwt_token_blacklist'
yacht    | INFO  [alembic.autogenerate.compare] Detected removed index 'ix_secret_key_key' on 'secret_key'
yacht    | INFO  [alembic.autogenerate.compare] Detected removed table 'secret_key'
yacht    | --- MODELS ---
yacht    | Generating /alembic/versions/66dd50aedca1_automated_db_upgrade.py ...  done
yacht    | INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
yacht    | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
yacht    | INFO  [alembic.runtime.migration] Running upgrade  -> 66dd50aedca1, automated db upgrade
yacht    | --- MODELS ---
yacht    | [cont-init.d] 31-migrate: exited 0.
yacht    | [cont-init.d] 32-env: executing...
yacht    | Replacing env constants in JS
yacht    | Processing /app/js/app.1d365aa7.js ...
yacht    | Processing /app/js/app.1d365aa7.js.map ...
yacht    | Processing /app/index.html ...
yacht    | [cont-init.d] 32-env: exited 0.
yacht    | [cont-init.d] 99-custom-files: executing...
yacht    | [custom-init] no custom files found exiting...
yacht    | [cont-init.d] 99-custom-files: exited 0.
yacht    | [cont-init.d] done.
yacht    | [services.d] starting services
yacht    | [services.d] done.
yacht    | INFO:     Started server process [314]
yacht    | INFO:     Waiting for application startup.
yacht    | INFO:     Application startup complete.
yacht    | INFO:     Uvicorn running on unix socket /tmp/gunicorn.sock (Press CTRL+C to quit)
yacht    | Secret key generated
yacht    | Secret key exists
yacht    | DISABLE_AUTH = False (<class 'bool'>)
yacht    | Users Exist
yacht    | Template Variables Exist
yacht    | INFO:      - "GET /api/auth/me HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/ HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "POST /api/auth/refresh HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "POST /api/auth/refresh HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/auth/me HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "POST /api/auth/login HTTP/1.1" 400 Bad Request
yacht    | INFO:      - "POST /api/auth/login HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/ HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:     Shutting down
yacht    | INFO:     Waiting for application shutdown.
yacht    | INFO:     Application shutdown complete.
yacht    | INFO:     Finished server process [314]
yacht    | [cont-finish.d] executing container finish scripts...
yacht    | [cont-finish.d] done.
yacht    | [s6-finish] waiting for services.
yacht    | [s6-finish] sending all processes the TERM signal.
yacht    | [s6-finish] sending all processes the KILL signal and exiting.
yacht    | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
yacht    | [s6-init] ensuring user provided files have correct perms...exited 0.
yacht    | [fix-attrs.d] applying ownership & permissions fixes...
yacht    | [fix-attrs.d] done.
yacht    | [cont-init.d] executing container initialization scripts...
yacht    | [cont-init.d] 01-envfile: executing...
yacht    | [cont-init.d] 01-envfile: exited 0.
yacht    | [cont-init.d] 10-adduser: executing...
yacht    | usermod: no changes
yacht    |
yacht    | -------------------------------------
yacht    |           _         ()
yacht    |          | |  ___   _    __
yacht    |          | | / __| | |  /  \
yacht    |          | | \__ \ | | | () |
yacht    |          |_| |___/ |_|  \__/
yacht    |
yacht    |
yacht    | Brought to you by linuxserver.io
yacht    | -------------------------------------
yacht    |
yacht    | To support LSIO projects visit:
yacht    | https://www.linuxserver.io/donate/
yacht    | -------------------------------------
yacht    | GID/UID
yacht    | -------------------------------------
yacht    |
yacht    | User uid:    911
yacht    | User gid:    911
yacht    | -------------------------------------
yacht    |
yacht    | [cont-init.d] 10-adduser: exited 0.
yacht    | [cont-init.d] 30-config: executing...
yacht    | [cont-init.d] 30-config: exited 0.
yacht    | [cont-init.d] 31-migrate: executing...
yacht    | INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
yacht    | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
yacht    | ERROR [alembic.util.messaging] Target database is not up to date.
yacht    | --- MODELS ---
yacht    | FAILED: Target database is not up to date.
yacht    | INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
yacht    | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
yacht    | INFO  [alembic.runtime.migration] Running upgrade  -> 66dd50aedca1, automated db upgrade
yacht    | --- MODELS ---
yacht    | [cont-init.d] 31-migrate: exited 0.
yacht    | [cont-init.d] 32-env: executing...
yacht    | Replacing env constants in JS
yacht    | Processing /app/js/app.1d365aa7.js ...
yacht    | Processing /app/js/app.1d365aa7.js.map ...
yacht    | Processing /app/index.html ...
yacht    | [cont-init.d] 32-env: exited 0.
yacht    | [cont-init.d] 99-custom-files: executing...
yacht    | [custom-init] no custom files found exiting...
yacht    | [cont-init.d] 99-custom-files: exited 0.
yacht    | [cont-init.d] done.
yacht    | [services.d] starting services
yacht    | [services.d] done.
yacht    | INFO:     Started server process [309]
yacht    | INFO:     Waiting for application startup.
yacht    | INFO:     Application startup complete.
yacht    | INFO:     Uvicorn running on unix socket /tmp/gunicorn.sock (Press CTRL+C to quit)
yacht    | Secret key generated
yacht    | Secret key exists
yacht    | DISABLE_AUTH = False (<class 'bool'>)
yacht    | Users Exist
yacht    | Template Variables Exist
yacht    | INFO:      - "GET /api/apps/ HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/auth/logout HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/auth/me HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "POST /api/auth/login HTTP/1.1" 400 Bad Request
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "POST /api/auth/login HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/ HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | ERROR:    Exception in ASGI application
yacht    | Traceback (most recent call last):
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 390, in run_asgi
yacht    |     result = await app(self.scope, self.receive, self.send)
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
yacht    |     return await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
yacht    |     await super().__call__(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/applications.py", line 111, in __call__
yacht    |     await self.middleware_stack(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
yacht    |     await self.app(scope, receive, _send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
yacht    |     await self.app(scope, receive, sender)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 566, in __call__
yacht    |     await route.handle(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 227, in handle
yacht    |     await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 44, in app
yacht    |     await response(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 176, in __call__
yacht    |     await run_until_first_complete(
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in run_until_first_complete
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in <listcomp>
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 199, in stream_response
yacht    |     async for data in self.body_iterator:
yacht    |   File "/./api/actions/apps.py", line 528, in all_stat_generator
yacht    |     async for event in merged:
yacht    |   File "/usr/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine
yacht    |     result = task.result()
yacht    |   File "/./api/actions/apps.py", line 503, in stat_generator
yacht    |     current_stats = await process_app_stats(line, app_name)
yacht    |   File "/./api/actions/apps.py", line 551, in process_app_stats
yacht    |     cpu_percent = await calculate_cpu_percent(line)
yacht    |   File "/./api/utils/apps.py", line 228, in calculate_cpu_percent
yacht    |     system_delta = float(d["cpu_stats"]["system_cpu_usage"]) - float(
yacht    | KeyError: 'system_cpu_usage'
yacht    | ERROR:    Exception in ASGI application
yacht    | Traceback (most recent call last):
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 390, in run_asgi
yacht    |     result = await app(self.scope, self.receive, self.send)
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
yacht    |     return await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
yacht    |     await super().__call__(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/applications.py", line 111, in __call__
yacht    |     await self.middleware_stack(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
yacht    |     await self.app(scope, receive, _send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
yacht    |     await self.app(scope, receive, sender)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 566, in __call__
yacht    |     await route.handle(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 227, in handle
yacht    |     await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 44, in app
yacht    |     await response(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 176, in __call__
yacht    |     await run_until_first_complete(
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in run_until_first_complete
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in <listcomp>
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 199, in stream_response
yacht    |     async for data in self.body_iterator:
yacht    |   File "/./api/actions/apps.py", line 528, in all_stat_generator
yacht    |     async for event in merged:
yacht    |   File "/usr/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine
yacht    |     result = task.result()
yacht    |   File "/./api/actions/apps.py", line 503, in stat_generator
yacht    |     current_stats = await process_app_stats(line, app_name)
yacht    |   File "/./api/actions/apps.py", line 551, in process_app_stats
yacht    |     cpu_percent = await calculate_cpu_percent(line)
yacht    |   File "/./api/utils/apps.py", line 228, in calculate_cpu_percent
yacht    |     system_delta = float(d["cpu_stats"]["system_cpu_usage"]) - float(
yacht    | KeyError: 'system_cpu_usage'
yacht    | ERROR:    Exception in ASGI application
yacht    | Traceback (most recent call last):
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 390, in run_asgi
yacht    |     result = await app(self.scope, self.receive, self.send)
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
yacht    |     return await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
yacht    |     await super().__call__(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/applications.py", line 111, in __call__
yacht    |     await self.middleware_stack(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
yacht    |     await self.app(scope, receive, _send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
yacht    |     await self.app(scope, receive, sender)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 566, in __call__
yacht    |     await route.handle(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 227, in handle
yacht    |     await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 44, in app
yacht    |     await response(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 176, in __call__
yacht    |     await run_until_first_complete(
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in run_until_first_complete
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in <listcomp>
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 199, in stream_response
yacht    |     async for data in self.body_iterator:
yacht    |   File "/./api/actions/apps.py", line 528, in all_stat_generator
yacht    |     async for event in merged:
yacht    |   File "/usr/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine
yacht    |     result = task.result()
yacht    |   File "/./api/actions/apps.py", line 503, in stat_generator
yacht    |     current_stats = await process_app_stats(line, app_name)
yacht    |   File "/./api/actions/apps.py", line 551, in process_app_stats
yacht    |     cpu_percent = await calculate_cpu_percent(line)
yacht    |   File "/./api/utils/apps.py", line 228, in calculate_cpu_percent
yacht    |     system_delta = float(d["cpu_stats"]["system_cpu_usage"]) - float(
yacht    | KeyError: 'system_cpu_usage'
yacht    | ERROR:    Exception in ASGI application
yacht    | Traceback (most recent call last):
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 390, in run_asgi
yacht    |     result = await app(self.scope, self.receive, self.send)
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
yacht    |     return await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
yacht    |     await super().__call__(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/applications.py", line 111, in __call__
yacht    |     await self.middleware_stack(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
yacht    |     await self.app(scope, receive, _send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
yacht    |     await self.app(scope, receive, sender)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 566, in __call__
yacht    |     await route.handle(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 227, in handle
yacht    |     await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 44, in app
yacht    |     await response(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 176, in __call__
yacht    |     await run_until_first_complete(
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in run_until_first_complete
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in <listcomp>
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 199, in stream_response
yacht    |     async for data in self.body_iterator:
yacht    |   File "/./api/actions/apps.py", line 528, in all_stat_generator
yacht    |     async for event in merged:
yacht    |   File "/usr/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine
yacht    |     result = task.result()
yacht    |   File "/./api/actions/apps.py", line 503, in stat_generator
yacht    |     current_stats = await process_app_stats(line, app_name)
yacht    |   File "/./api/actions/apps.py", line 551, in process_app_stats
yacht    |     cpu_percent = await calculate_cpu_percent(line)
yacht    |   File "/./api/utils/apps.py", line 228, in calculate_cpu_percent
yacht    |     system_delta = float(d["cpu_stats"]["system_cpu_usage"]) - float(
yacht    | KeyError: 'system_cpu_usage'
yacht    | ERROR:    Exception in ASGI application
yacht    | Traceback (most recent call last):
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 390, in run_asgi
yacht    |     result = await app(self.scope, self.receive, self.send)
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
yacht    |     return await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
yacht    |     await super().__call__(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/applications.py", line 111, in __call__
yacht    |     await self.middleware_stack(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
yacht    |     await self.app(scope, receive, _send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
yacht    |     await self.app(scope, receive, sender)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 566, in __call__
yacht    |     await route.handle(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 227, in handle
yacht    |     await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 44, in app
yacht    |     await response(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 176, in __call__
yacht    |     await run_until_first_complete(
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in run_until_first_complete
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in <listcomp>
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 199, in stream_response
yacht    |     async for data in self.body_iterator:
yacht    |   File "/./api/actions/apps.py", line 528, in all_stat_generator
yacht    |     async for event in merged:
yacht    |   File "/usr/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine
yacht    |     result = task.result()
yacht    |   File "/./api/actions/apps.py", line 503, in stat_generator
yacht    |     current_stats = await process_app_stats(line, app_name)
yacht    |   File "/./api/actions/apps.py", line 551, in process_app_stats
yacht    |     cpu_percent = await calculate_cpu_percent(line)
yacht    |   File "/./api/utils/apps.py", line 228, in calculate_cpu_percent
yacht    |     system_delta = float(d["cpu_stats"]["system_cpu_usage"]) - float(
yacht    | KeyError: 'system_cpu_usage'
yacht    | ERROR:    Exception in ASGI application
yacht    | Traceback (most recent call last):
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 390, in run_asgi
yacht    |     result = await app(self.scope, self.receive, self.send)
yacht    |   File "/usr/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
yacht    |     return await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
yacht    |     await super().__call__(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/applications.py", line 111, in __call__
yacht    |     await self.middleware_stack(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
yacht    |     await self.app(scope, receive, _send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
yacht    |     raise exc from None
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
yacht    |     await self.app(scope, receive, sender)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 566, in __call__
yacht    |     await route.handle(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 227, in handle
yacht    |     await self.app(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/routing.py", line 44, in app
yacht    |     await response(scope, receive, send)
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 176, in __call__
yacht    |     await run_until_first_complete(
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in run_until_first_complete
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/starlette/concurrency.py", line 18, in <listcomp>
yacht    |     [task.result() for task in done]
yacht    |   File "/usr/lib/python3.8/site-packages/sse_starlette/sse.py", line 199, in stream_response
yacht    |     async for data in self.body_iterator:
yacht    |   File "/./api/actions/apps.py", line 528, in all_stat_generator
yacht    |     async for event in merged:
yacht    |   File "/usr/lib/python3.8/site-packages/aiostream/stream/advanced.py", line 59, in base_combine
yacht    |     result = task.result()
yacht    |   File "/./api/actions/apps.py", line 503, in stat_generator
yacht    |     current_stats = await process_app_stats(line, app_name)
yacht    |   File "/./api/actions/apps.py", line 551, in process_app_stats
yacht    |     cpu_percent = await calculate_cpu_percent(line)
yacht    |   File "/./api/utils/apps.py", line 228, in calculate_cpu_percent
yacht    |     system_delta = float(d["cpu_stats"]["system_cpu_usage"]) - float(
yacht    | KeyError: 'system_cpu_usage'
yacht    | error while getting new CPU stats: %r, falling back
yacht    | error while getting new CPU stats: %r, falling back
yacht    | error while getting new CPU stats: %r, falling back
yacht    | error while getting new CPU stats: %r, falling back
yacht    | error while getting new CPU stats: %r, falling back
yacht    | error while getting new CPU stats: %r, falling back
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/settings/variables HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/settings/check/update HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "GET /api/auth/me HTTP/1.1" 401 Unauthorized
yacht    | INFO:      - "POST /api/auth/refresh HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/auth/me HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/auth/me HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/auth/me HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/auth/me HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/ HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK
yacht    | INFO:      - "GET /api/apps/stats HTTP/1.1" 200 OK

C-Fu avatar Jul 02 '21 06:07 C-Fu

I'm hitting this as well. I have ~60+ containers and loading Yacht freezes my browser.

image

It looks like there's a /api/apps/stats request for each container, but each /api/apps/stats EventStream gives data about all containers.

nicjansma avatar Jul 08 '21 16:07 nicjansma

I've noticed that just having Yacht open in my browser makes a noticeable impact on my docker host's CPU temperature. I'd like to be able to leave it open in a tab in the background, but it makes my office noisy :)

SteveDinn avatar Jul 08 '21 18:07 SteveDinn

Everyone who’s having these issues. Can you try the devel tag to see if that resolved it? (Just spin up another instance of yacht on another port so it doesn’t affect your current one)

skluthe avatar Jul 08 '21 18:07 skluthe

With :devel, I no longer see tons of /api/app/stats requests:

image

However, the page still locks up on all the browsers I've tried (Chrome, FF, Safari). I can't even open the Profiler in any of the browsers because the profilers hang too.

nicjansma avatar Jul 12 '21 13:07 nicjansma

I was finally able to get a Profile with Safari.

Here's a 30.37-second trace with ~29 seconds of JavaScript CPU:

image

Obviously this is minified code but it seems to be Vue updates.

nicjansma avatar Jul 12 '21 13:07 nicjansma

I am seeing this issue as well. Installed through OMV5 OMV-Extras. Lots of intermittent lag loading things and getting a "KeyError: system_cpu_usage" in the logs.

Spencer-Dawson avatar Sep 28 '21 03:09 Spencer-Dawson

Yeah Yacht is unusable for me until this is fixed. Unfortunately I don't have any experience debugging Vue, hopefully there's a simple place it can be optimized?

nicjansma avatar Sep 28 '21 14:09 nicjansma

Yeah Yacht is unusable for me until this is fixed. Unfortunately I don't have any experience debugging Vue, hopefully there's a simple place it can be optimized?

Unfortunately this kind of issue is hard to optimize which is why it’s taken so long. I may wind up having to just add a few seconds of delay to the dashboards in order for it not to be so resource intensive. Also is intermittent and not affecting everyone so it’s hard to nail down.

skluthe avatar Sep 29 '21 12:09 skluthe

Honestly, even requiring a manual refresh is perfectly acceptable to me. Anything is worth making it not use so many resources on both the hosting server and the client.

SteveDinn avatar Sep 29 '21 12:09 SteveDinn

@skluthe thanks for taking a look. It's 100% repro for me, given enough docker instances (I have 58 on my host).

Again I know very little about Vue, but in the past when I've seen issues like this it can be caused by the UI framework not batching UI updates together (e.g. trying to apply container 1, draw, then 2, draw, etc instead of applying container 1-58 changes together then draw). Looks like Vue batches updates for each Tick, but maybe the way these components are being updated is confusing Vue a bit?

If it's a challenge to fix the perf issues here, I'd also be happy with what @SteveDinn says: either a control to decide how often to update (e.g. never, 1s, 2s, 5s, 10s, 30s, 60s) or a "Update Now" button.

nicjansma avatar Sep 29 '21 16:09 nicjansma

Dev branch resolved the performance issue for me.

Spencer-Dawson avatar Sep 30 '21 00:09 Spencer-Dawson