junyan-zg comments

Repositories
Issues
Comments

Results 2 comments of


                                            junyan-zg

Can't swap tokens

Successfully verified on v0.17.0 !

[Feature]: Application support for the Qwen2.5-VL-72B-Instruct-unsloth-bnb-4bit、Qwen2.5-VL-72B-Instruct-bnb-4bit series models.

@bbss Is the dynamic quant one working now ? ` >>> from vllm import LLM INFO 03-20 12:43:57 [__init__.py:256] Automatically detected platform cuda. >>> import torch >>> model_id = "/opt/Qwen2.5-VL-72B-Instruct-unsloth-bnb-4bit"...