automaticcat

icon indicating a website link alandao.net icon indicating an email [email protected]

icon company @janhq icon location Nononnono

Results 98 comments of automaticcat

feat: Threads can override and inherit default params from Assistants, Models, and Engine

We will have 2 main modes: LOCAL and REMOTE LOCAL REMOTE

feat: Threads can override and inherit default params from Assistants, Models, and Engine

Link to excalid draw: https://excalidraw.com/#json=kOBPg9OoLTCLAm3JO7FHn,qV29wMh7fLvGkFXf5HRYNA

epic: Jan supports multi Inference Engines

Stale, rn we're doing this in cortex with different ticket

feat: enable `continue` button for when LLM responses exceed token limit

This is not a bug since i don't have any hard coded token limit

feat: enable `continue` button for when LLM responses exceed token limit

should be transfered to jan

feat: Apple Ferret and MLX

blocked by Python runtime

bug: very large prompts cause nitro to hang

This is a purely performance issue. We can try to mitigate this with some warning @imtuyethan

feat: nitro hibernation/model unload after X time

This should be on nitro inference plugin level

epic: Nitro supports speech/hear capabilities

need to resolve the difference in ggml model + file between whisper and llama cpp

feat: GPU Docker Image?

hi @Elsayed91 i have a working CUDA example dockerfile in my homelab, i will update that for everyone to try also