openvino icon indicating copy to clipboard operation
openvino copied to clipboard

Published 20 hours ago • verified publisher icon openvinotoolkit

TP flow in CPU plugin

Open sunxiaoxia2022 opened this issue 9 months ago • 0 comments

Details:

  • two sub streams run in parallel when modelDistributionPolicy=TENSOR_PARALLEL in latency mode

Tickets:

  • ticket-id

sunxiaoxia2022 avatar May 05 '24 05:05 sunxiaoxia2022