SmallShark

Fix Bug: Gemma2 the past_key_value.update() function has added a new parameter "sliding_window" to support the _sliding_update function.

1

# What does this PR do? System Info transformers 4.42.3 Now gemma2 model generates long text that exceeds the window size (>4096), it will report a CUDA error, which seems...

[Feature]: ROPE scaling supported by vLLM gemma2

1

### 🚀 The feature, motivation and pitch Now vLLM gemma2 does not support ROPE scaling, and I sincerely hope that support for it will be added in the future.