ALiBi positional encoding

[aamir-gmail]

Based on the paper from Bloomberg GPT, the reference here https://arxiv.org/pdf/2303.17564v1.pdf They mention sequence length of more than 2048 can be used during inference https://paperswithcode.com/method/alibi Can the LLAMA model be modified to use this?