Prateek Gupta

Did you try to reproduce the issue on Apple M1? I reinstalled the Jan application, added the model again. The issue is restricted to this model. I can use codeqwen-1_5...

> I think there is a problem with the downloaded model. The stats are unreal; I'm using an M2 Pro with 32 GB of RAM, but the token speed is...

Bumping this necro thread. Is it possible to get callback facility using SCIP?

> Sorry the inference is taking so long. I could not figure this out, does Jan.ai use ollama to run the models? Or are you talking about two different cases?...

But this will always bottleneck when you have the most requirement for using the assistant. If I have 10 notes, I can manage them without the plugin. If I increase...

facing a similar issue. "Difference between Mistral-T and transformer" starts repeating after 4 points.

> > I think we should add something like this to the guide: > > Perhaps "notable" instead of "important"? But yes. Also specifically https://github.com/koreader/contrib Bumping a necro thread. Anyone...

> * If you are talking about on-device inference, most of the devices KOReader works on don't have the CPU/GPU/RAM/battery budget to provide a reasonable experience. Newest Android phones/tablets can...

> @SmokeShine You can run llama.cpp's server + a python script to get OpenAI compatible API. I think all you need to do is change > > ```lua > local...

> If you want to use KoboldCpp API, you can try my fork of askGPT here: https://github.com/Topping1/AskKobold/tree/main Thanks @Topping1 . Is it possible to get options like - summary of...