adi-lb-phoenix

Results 9 comments of adi-lb-phoenix

Documentation for installing AdaptiveCpp on macbook Pro M1

@illuhad Thank you for the reply . I think its better if documentation related to macOS can be replicated for how its done with windows as you suggested , because...

unable to convert llama2 7b model

I have been getting an error when trying to convert Meta-Llama-3-8B-Instruct.Q4_0.gguf to .bin format `python3 export.py llama2_7b.bin --meta-llama /home/####/llm_inferences/llama.cpp/models/meta Traceback (most recent call last): File "/home/####t/llm_inferences/llama2.c/export.py", line 559, in model...

failed to convert llama_2 7B model in .gguf to .bin format

Yes noted. I did not follow the instructions carefully.

AttributeError: 'NoneType' object has no attribute 'Redis' when using Celery with Redis

> pip install redis @swcasimiro why do I need to `pip install redis` when I have a redis as `docker run -d -p 6379:6379 redis`

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

@fanalyaen that does not work

Garbage output on serving 4 parallel users.

Hello @sgwhat . So I have installed podman and distrobox on kde neon, on which I have created a ubuntu distro using distrobox. Ipex llm is deployed inside the ubuntu...

Garbage output on serving 4 parallel users.

@sgwhat Thank you for picking this up. It has been observed not just for `codellama ` but for other models as well.

Garbage output on serving 4 parallel users.

https://github.com/ggerganov/llama.cpp/issues/9505#issuecomment-2352561991 Here llama.cpp does not output garbage values

Garbage output on serving 4 parallel users.

When serving just one user Ipex llm has better speed than llama.cpp Result from ipex-llm: ``` llama_print_timings: load time = 7797.13 ms llama_print_timings: sample time = 30.64 ms / 400...