tristan279

Hi @githubzuoyi also interested in this issue, tried installing unsloth with that commit but getting the "TypeError: LlamaRotaryEmbedding.__init__() got an unexpected keyword argument 'config' " error again, how did you...

I'm getting this error as well! following the colab instructions, with the meta-llama/Meta-Llama-Llama3.1-8B-Instruct model. when doing it with the model from unsloth, it works fine  ==((====))== Unsloth 2024.8: Fast...

issue resolved! Thanks for the fix ::)

with vllm=0.5.3.post, only thing i changed was enabling prefix caching, crashed with illegal cuda access errors