How can one set the parameters via CLI when starting a server?

Open syddharth opened this issue 2 years ago • 2 comments

Is this already possible?

If not, could this be a feature? For example:

 .\nitro 4 127.0.0.1 5000 --ngl 20

on a Windows11 machine with NVIDIA GPU.

Jan 06 '24 00:01 syddharth

hi friend you can have a look at the docs https://nitro.jan.ai/docs

Jan 06 '24 05:01 tikikun

https://nitro.jan.ai/features/load-unload

This section specifically, we don't support at runtime right now, mostly during the curl to load model

Jan 06 '24 06:01 tikikun