b4b4o

In TF_Blade, we found TRT_SUPPORT_LIST is not match the Onnx-Tensorrt Op Support List. In our opinion, this will lead to the TRT Engine Convert Failed. Because some Op(like 'FusedBatchNormV3') is...

fix typo.

[QUESTION] Why we need modify the LinearWithGradAccumulationAndAsyncCommunication's backward function to support SP?

1

Thanks to authors for the great work! As shown in this commit https://github.com/sail-sg/zero-bubble-pipeline-parallelism/commit/a84d634a5c0597b694f214ed8a72a8fdb0010bdb, It just seems to add a flag reshard_for_sequence_parallel for the Row Major Linear, The forward RS and...