Pravin Gadakh

SSL handshake failed with `ssl.ca.pem` option

1

Description =========== I've been unable to connect to secured kafka broker using `ssl.ca.pem` option whereas `ssl.ca.location` works just fine. I am new to go as well as kafka, so pardon...

0.9.4 docker image using cuda 11.7 instead of cuda 11.8

16

### System Info We are trying to run llama2-70B model and have noticed that with huggingface/text-generation-inference:0.9.1 docker image it is using cuda 11.8 ``` >>> import torch >>> torch.version.cuda '11.8'...

Unable to run distributed inference on ray with tensor parallel size > 1

6

I am referring to https://github.com/vllm-project/vllm/blob/main/examples/offline_inference_distributed.py example. The suggestion in the example is not to set num_gpus when tensor parallel is used. However with that I ran into following issue: Ray...

[Core|Dataset] Ray job stuck with idle actors with no tasks

4

### What happened + What you expected to happen **What happened** Our ray job intermittently gets stuck. The Ray job is submitted using the RayJob CRD. We use ray data...