Refactor StableDiffusion example scripts to use Optimum / BetterTransformer optimisations

[0xdevalias]

Describe the solution you'd like

StableDiffusion example scripts are updated to make use of the Optimum / BetterTransformer optimisations.

Describe alternatives you've considered

That the examples can't be integrated, or that it won't provide a speedup/benefits.

Additional context

• https://huggingface.co/docs/optimum/bettertransformer/overview

  • Optimum provides an integration with BetterTransformer, a stable API from PyTorch to benefit from interesting speedups on CPU & GPU through sparsity and fused kernels.

  • Since its 1.13 version, PyTorch released the stable version of BetterTransformer in its library. You can benefit from interesting speedup on most consumer-type devices, including CPUs, older and newer versions of NIVIDIA GPUs. You can now use this feature in 🤗 Optimum together with Transformers and use it for major models in the Hugging Face ecosystem.

• https://huggingface.co/docs/optimum/bettertransformer/tutorials/contribute

  • Adding BetterTransformer support for new architectures

• https://twitter.com/huggingface/status/1594783600855158805

  • A collaboration with @PyTorch to make transformer-based models faster using optimum library! Up to 4.5x speedup for text, vision and audio models using a one liner! 🔥

• https://twitter.com/PyTorch/status/1594766050851102720

  • Better Transformer for #PyTorch out of the box performance on 🤗 @huggingface models now available! Want to know more about the collaboration? 👀 the blog

    • https://medium.com/pytorch/bettertransformer-out-of-the-box-performance-for-huggingface-transformers-3fbe27d50ab2

See also

• https://github.com/huggingface/optimum/issues/512
• https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/5015

[patrickvonplaten]

I think we would like to add support for stable diffusion directly into Optimum cc @echarlaix

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.