Günter
Günter
Thanks a ton for your take on this. I appreciate it! I fear you are right that SLI is too little general purpose and just for splitting up graphics rendering...
Could you please share some details what sizes (model, parameter count,..) the numbers in the graph relate to!? Guess it must be pretty small if results for 6GB (and ancient...
@bkutasi Hey Balázs, any idea how to pimp things to make longer prompts work? Got 256GB RAM and 2x 20GB VRAM RTX 4000. Sorta waste. ;-) Thought I had a...
> wish I had time to fork/contribute Just too many fronts. @ current pace give Parler coupla weeks to mature & use alts 2 do tts. stt Whisper pretty good...
@bkutasi Been following the whole AI craze, but lost track what's free & local (and really open needed?) in TTS. Not yet decided what to think of LocalAI.io, but just...
Thanks for the light-speed response on Easter Sunday. Was in bed seconds after bug rep. (Vienna time ;-) Just tried the bin with your curl on Ubuntu 22.04 latest and...
Please forgive if I mix Windows & Docker and Ubuntu and bin, but didn't get to start Ubuntu and fell into Windows: docker run -p 8088:8080 --gpus all --name local-ai...
Ubuntu 22.04 bin > it seems it does not find gpt-4 model definition. can you post the content of /home/gy/local-ai/models directory and also the output from curl http://127.0.0.1:8080/models? As expected...
**BINGO!** `gy@gnu:~/dl$ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ "model": "gpt-4", "messages": [{"role": "user", "content": "How are you doing?", "temperature": 0.1}] }'` `{"created":1712000959,"object":"chat.completion","id":"6d6f0d5e-29ae-4d21-b822-2ee2f4458687","model":"gpt-4","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"I'm doing well, thank you for asking! How...
You seem to have overlooked that only a few of the curl sample commands work, but http://127.0.0.1:8080/ throws `{"error":{"code":404,"message":"Cannot GET /","type":""}}` on any platform, Windows Docker AiO and AVX-512 binary...