PiPPy
PiPPy copied to clipboard
[SPMD][Fusion] Add unit tests for fusion
After PR https://github.com/pytorch/tau/pull/631 lands, add unit testing.
Simple tests would involved fusion based on a set policy and verifying output gradients and inspecting the graph.