Introduce PipelineParallelMicroStepCallback for Custom Hooks in Pipeline Parallelism

[eee4017]

PR types

New features

PR changes

APIs

Description

This PR introduces PipelineParallelMicroStepCallback in PaddlePaddle, allowing enhanced management of hooks within pipeline parallel processes. Unlike existing register_forward_post_hook and register_forward_pre_hook methods for nn.Layer, this feature facilitates registering callbacks at specific pipeline stages, such as forward_begin, forward_end, backward_begin, and backward_end. This is particularly relevant for PipelineParallel where layers are divided into multiple chunks. The addition aims to support a variety of tasks including logging, monitoring, and dynamic parameter adjustments during pipeline execution. I am proposing this feature to align with PaddlePaddle's design principles and address the specific needs of pipeline parallelism. Feedback on the API's design and functionality would be greatly appreciated to ensure this enhancement meets community standards and expectations.

[paddle-bot[bot]]

你的PR提交成功，感谢你对开源项目的贡献! 请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。 Your PR has been submitted. Thanks for your contribution! Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-ci-bot[bot]]

Sorry to inform you that 3cfa108's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[paddle-ci-bot[bot]]

Sorry to inform you that b672fff's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.