LongLM issues

What effect on qwen1.5 will be if i use self-extend trick?

4

Thanks for your contribution on accommondating qwen on self-extend. Qwen1.5 has already been 32k context length. I'm wondering if i can use self-extend to make it to about 100K? Have...

WeixuanXiong

llama3 is not working.

1

I followed your direction like the below to apply selfextend to llama3 """ [04/19/2024]:💡 We added the support for LLama-3 with transformers==4.40. To use it with transformers==4.40, you may change...

rayjang

Support with vLLM

5

Hello! Thank you for your great work, its amazing how much hard work you put for this algorithm. I just had one question is it possible to integrate this with...

Aniketto16

Question about equation 4 and Table 5 caption in paper

3

Hi! I have a question that may seem simple, but I think I'm overlooking something. Assume Phi-2's context window is 2K. When we apply a group size ($G_s$) of 4...

MarsJacobs