llama_index_ray icon indicating copy to clipboard operation
llama_index_ray copied to clipboard

Published 20 hours ago • verified publisher icon amogkam
→

Metadata

Using LlamaIndex with Ray for productionizing LLM applications

Results 2 llama_index_ray issues
Sort by recently updated
recently updated
newest added

Error while loading the model from Huggingface with Ray Serve

@amogkam I was trying to implement ray serve to enhance the inference speed.But during implementation got the error about loading the model saying load_in_8_bit requires accelerate and bitsandbytes which I...

pranavbhat12 avatar pranavbhat12

Hardware Issue

1 comment

ctyler9 avatar ctyler9

← Metadata

70
Stars
11
Forks
Watchers

Owner

verified publisher icon amogkam

Metadata

Using LlamaIndex with Ray for productionizing LLM applications

Back