felixzhu555

[Misc] Add attention sinks

5

### Overview This PR adds experimental support for attention sinks (#1304), based on this [paper](https://arxiv.org/abs/2309.17453) and [repo](https://github.com/mit-han-lab/streaming-llm). Support is currently limited to RoPE and ALiBi models (e.g. Llama, Mistral/Mixtral, Falcon,...