openvino icon indicating copy to clipboard operation
openvino copied to clipboard
verified publisher icon openvinotoolkit

Hetero support continuous batching

Open WeldonWangwang opened this issue 11 months ago • 0 comments

Details:

  • Add remote context and remote tensor to support continuous batching
  • Fix the the reshape op shape in subgraph with paged attention, the output shape of reshape ops should not be [-1,-1] image

Tickets:

  • CVS-164805

WeldonWangwang avatar Apr 29 '25 07:04 WeldonWangwang