Chris Cates

What do you guys need to actually make this happen?

@leafyshark this implementation wouldn't work for me. What's frustrating is the complete lack of responsiveness from the Prisma team. We all know they aren't interested in our PRs and will...

@Ph0rk0z, @turboderp, Hey guys, just bumping this issue. I brought up the discussion about RoPE Base and RoPE Frequency in a previous issue. I'm not sure if it's possible to...

@turboderp understood. Here is my proposal. Instead of replacing the current rotary embedding calculation. We have optionality for two. Utilizing `rope_alpha` and `rope_theta` for the first calculation and `rope_base` and...

@Ph0rk0z thanks man! I was wondering why I couldn't find the relevant source. But, just found it. https://github.com/turboderp/exllama/blob/21f4a12be5794692f66410ad4fb78ffaad508d00/model.py#L126-L127

As per discussion in issue #270. This issue is being reopened. The following is a fairly informal proposal for @turboderp to review: Instead of replacing the current rotary embedding calculation....