text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

feat(server): avoid manipulating position_ids for non-applicable models

Open njhill opened this issue 2 years ago • 0 comments

Currently, position_ids are always maintained/updated in the CausalLM case but this is unnecessary for models like BLOOM which don't use them.

njhill avatar Apr 18 '23 03:04 njhill