lmdeploy icon indicating copy to clipboard operation
lmdeploy copied to clipboard
Published 1 month ago • verified publisher icon InternLM

[dlinfer][ascend]set blcoksize default to 128 for 310P device

Open JackWeiw opened this issue 7 months ago • 2 comments

Motivation

Dlinfer Ascend 310P device will trigger error when using lmdeploy chat api, since the original default block size is 64. Set block size to 128 will prevent this error

JackWeiw avatar May 07 '25 09:05 JackWeiw

waiting for checking on 910B

jinminxi104 avatar May 09 '25 05:05 jinminxi104

better to set block_size=128 on 910b

jinminxi104 avatar May 09 '25 16:05 jinminxi104