Saifeddine ALOUI comments

Results 313 comments of


                                            Saifeddine ALOUI

Any plans to add a queue status endpoint?

I guess I have to handle this on my end then. I'll add a proxy that counts the connections and route them to multiple servers.

Any plans to add a queue status endpoint?

Ok, it is done, I have created a separate repository for it. it also handles permissions and user authentication using a KEY (just like open ai api): https://github.com/ParisNeo/ollama_proxy_server

Support for windows

Is it crucial to use uvloop? This is the library that is preventing this too lfrom being using in windows.

Edges are hard to select

Hi there. I will try to fix this issue as soon as I have time. Thanks for pointing out the problem.

When select downloaded model, it show error "DLL load failed while importing exllamav2_ext"

this is like the red ring of death of exllamav2. It hapens when sometghing went wrong wile installing it. I never actually managed to find why this hapens on some...

When select downloaded model, it show error "DLL load failed while importing exllamav2_ext"

You can try reinstalling exllamav2 manually by opening the lollms conda environment and doing a pip install

from lollms.com import NotificationType, NotificationDisplayType ModuleNotFoundError: No module named 'lollms'

Ah, if you install using the script this should work. Here you have clearely not installed lollms. The script is supposed to install lollms when it is run. There is...

AttributeError: 'ExLlamaV2Attention' object has no attribute 'device_idx'

I spent the day trying to fix thix. Reinstall lollms then reinstall exllamav2 binding. It should work now.

Error when trying to use Petals.

Hi. If you are on windows, then you need to use WSL and it should work. If you are on linux, this looks like you did not install the binding...

AI not able to read documents uploaded

Hi, what kind of document did you send to the AI?