Cant limit cpu usage

[bruno-1337]

When trying to limit cpu usage following https://www.baeldung.com/ops/docker-memory-limit the model stops working Tried it on the cuda yml file

I am running a R7 5700 at 4.5ghz + GTX 1070

would be nice if there was a easy way to limit cpu usage

[M0E-lnx]

Try editing the docker-compose.yml file to add this to your llama-gpt-api service.

deploy:
  resources:
    limits:
      cpus: N # Number of cpu cores to allow.
      memory: NNNM # Your ram limit in MB followed by M, ex: 1024M

After that, execute ./run.sh as usual See here for more https://docs.docker.com/compose/compose-file/compose-file-v3/#resources

[bruno-1337]

i did everything on the link i provided, which includes the option you gave me, just doesnt work

[naufalso]

Try to modify n_threads in the run.sh

# Get the number of available CPU threads
n_threads=$(grep -c ^processor /proc/cpuinfo)

The script tries to use all CPUs in our current machine