vllm icon indicating copy to clipboard operation
vllm copied to clipboard

Published 20 hours ago • verified publisher icon vllm-project

[Bugfix] Fixed error in slice_lora_b for MergedQKVParallelLinearWithLora

Open FurtherAI opened this issue 1 year ago • 2 comments

FIX #3524 Fix #4605

MergedQKVParallelLinearWithLora slice_lora_b throws an error when the passed in lora_b contains None.

FurtherAI avatar May 05 '24 21:05 FurtherAI

Ping @Yard1.

FurtherAI avatar May 05 '24 21:05 FurtherAI

Good point, it is done. Also updated the fully sharded layers to match.

FurtherAI avatar May 06 '24 04:05 FurtherAI

can you merge this plz

tejeshbhalla avatar May 07 '24 17:05 tejeshbhalla