rjmehta1993

Results 7 comments of rjmehta1993

Feature request：support ExLlama

This will be the biggest release for vllm to support exllamav2. +1

ExLlamaV2: exl2 support

+1. EXl quants is unbeatable

[Bug] Unrecognized keys in `rope_scaling` for 'rope_type'='yarn': {'original_max_position_embeddings'}

Changing key from type to rope_type matches to what transformers expect and it removes warning. But does this confirms that the YaRN is implemented and working?

rh.order_sell_market => 'Invalid Good Til Canceled order.'

Same issue

Remove Conda from Docker Installation

Any updates?

[Bug]: Can't use yarn rope config for long context in Qwen2 model

+1. Changing key from type to rope_type matches to what transformers expect and it removes warning. But does this confirms that the YaRN is implemented and working?

[Feature] support mistral small vlm

is this supported yet? @zhyncs