[Bug] GPU container does not report GPU memory usage accurately

[mmartial]

Description of the bug

The dashdot:nvidia container does not report memory usage accurately.

I am attaching screenshots of memory consumption using a ComfyUI workflow. The workflow run uses close to 80% memory, but Dashdot reports only 30% VRAM use (despite recognizing the correct GPU model and memory). One image has the value of nvidia-smi to show the total memory usage.

After: Dashdot reports that 0% of the memory is in use when it is closer to 50%

A few minutes after (while I type this), the numbers are still the same 0% for Dashdot, about 50% for nvidia-smi

I note that ComfyUI is also running as a GPU container.

How to reproduce

compose.yaml file:

services:
  dash:
    image: mauricenino/dashdot:nvidia
    container_name: dashdot-nvidia
    restart: unless-stopped
    privileged: true
    deploy:
      resources:
        reservations:
          devices:
            - capabilities:
                - gpu
    ports:
      - 3001:3001
    volumes:
      - /:/mnt/host:ro
    environment:
      DASHDOT_WIDGET_LIST: os,cpu,storage,ram,network,gpu

Relevant log output

No response

Info output of dashdot cli

INFO
=========
Yarn: 3.7.0
Node: v20.16.0
Dash: 5.8.6

Cwd: /app
Hash: a0e2ee941c37092804cb11acc7252ada877b9772
Platform: Linux 97d6bae71c48 6.8.0-41-generic #41-Ubuntu SMP PREEMPT_DYNAMIC Fri Aug  2 20:41:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Docker image: nvidia
In Docker: true
In Docker (env): true
In Podman: false

What browsers are you seeing the problem on?

Firefox, Chrome

Where is your instance running?

Linux Server, Desktop PC (Linux)

Additional context

No response