hiro
hiro
**Describe the bug** - There is a bug reported for using `Mistral Instruct 7B Q4` model that has text strikethrough: https://discord.com/channels/1107178041848909847/1192366847446753330/1192371090123665419 - After careful investigation, me and @louis-jan found out...
## Describe Your Changes - Monitor: NVIDIA total VRAM and current load (GPU and VRAM) ## Fixes Issues - #286 - #1428 ## Self Checklist - [x] Added relevant comments,...
Feature for https://github.com/janhq/nitro/issues/175 - [x] Load multiple models - [ ] Add GET `models` to return models list - [ ] CUDA support for multiple model request at the same...
**Problem** - The current implementation for `chat/completion` with only support for base64 as `image_url.url` makes it hard for using curl to test out quickly. Using something like `file://` makes it...
I would like to run TTS and STT supported in Nitro for my personal usage (self hosted Otter.ai) For now nitro supports "think" - running LLM This new feature is...
Follow up on #205 - #59 - API: https://platform.openai.com/docs/api-reference/audio/createTranscription - Whisper.cpp: https://github.com/ggerganov/whisper.cpp.git
**Problem** - It's tricky to call nitro with `ai_prompt`, `system_prompt` and `user_prompt` as there are many existing and well-known chat templates out there. **Success Criteria** - API call in `loadModel`...
**Problem** - As an model user day to day, I find it hard to explain and share to my friends which model is good to use, especially with the help...
**Problem** - Currently the installation of Nitro is quite manual (I have to download binaries from release) - No universal way to install **Success Criteria** - `pip install nitro` -...
**Problem** - Sometime it's interesting to have the LLM to suggest question for me to ask it - Sometime I have to think a lot more after having some turns...