satflow
satflow copied to clipboard
Try out PipeTransformer
PyTorch recently came out with this https://github.com/Distributed-AI/PipeTransformer that sped up training transformers and potentially using less GPU resources, or at least train faster.