sam

Same issue for me after I updated to 0.2.2 with Pytorch '2.1.1+cu121'. 0.2.1-post1 with '2.0.1+cu11.8' was working.

Thanks @WoosukKwon for the suggestion. I think I am able to make it work now. I was confused with AliBi as in the HF implementation there is **only one AliBi...

Hey guys, thanks for enthusiasm. I am currently working on it. Currently facing some issues with CUDA.

Getting the below issue with latest `vLLM-0.2.7`, although Jais works with `vLLM-0.2.1-post1`. ``` Traceback (most recent call last): File "/path_to_vllm_env/vllm/lib/python3.9/site-packages/vllm/worker/model_runner.py", line 581, in capture hidden_states = self.model( File "/path_to_vllm_env/vllm/lib/python3.9/site-packages/torch/nn/modules/module.py", line...

Hey @7ossam81 , I have the code ready, will make it public by EOD today. You can just directly run it. Please wait for some time.