vllm icon indicating copy to clipboard operation
vllm copied to clipboard

[Bugfix] Fix none seed sampling in rejection_sampler

Open TopIdiot opened this issue 2 months ago • 8 comments

This PR fixes https://github.com/vllm-project/vllm/issues/11123

When parts of seeds are None in a batch, list slice of a tensor return a new tensor rather than a view of the original tensor. We can simply confirm it by .storage().data_ptr(). So these parts of the original tensor will not be updated.

TopIdiot avatar Dec 12 '24 09:12 TopIdiot