rerope
rerope copied to clipboard
Excellent Idea!!!!
Could you support llama2 with rerope?
The current open-source code is based on the llama implementation (both llama1 and llama2 are supported, as long as transformers==3.41.0)
@bojone Great Job! Can this method be directly applied to Alibi?
@xs1997zju I haven't experimented with alibi, but my intuition tells me alibi can directly use Position Interpolation.