FasterTransformer icon indicating copy to clipboard operation
FasterTransformer copied to clipboard

Does FasterTransformer support multi-stream pipeline parallelism ?

Open FlyingPotatoZ opened this issue 1 year ago • 0 comments

Hello guys: Because there is no dependence on computation and communication, I think multi-stream pipeline parallelism can hide communication time to improve performance. I didn't find how to configure the multi-stream feature in the code. Can anyone help? Thank you so much~ ftNcclRecv(sequence_lengths_ + id_offset, local_batch_size * beam_width, pipeline_para_.world_size_ - 1, pipeline_para_, stream_);

FlyingPotatoZ avatar Nov 21 '23 01:11 FlyingPotatoZ