diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Modification on the PAG community pipeline

Open HyoungwonCho opened this issue 10 months ago • 7 comments

What does this PR do?

Perturbed-Attention Guidance

This implementation is based on Diffusers. StableDiffusionPAGPipeline is a modification of StableDiffusionPipeline to support Perturbed-Attention Guidance (PAG). For more information about PAG, please refer to Project / arXiv / GitHub

Hi! We have updated some of the implementations of the Perturbed-Attention Guidelines, which we previously contributed to community pipelines.

Fixes # (issue)

Before submitting

  • [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [x] Did you read the contributor guideline?
  • [x] Did you read our philosophy doc (important for complex PRs)?
  • [ ] Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
  • [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • [ ] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

HyoungwonCho avatar Apr 17 '24 06:04 HyoungwonCho

can we run make quality? will merge once the tests pass

yiyixuxu avatar May 01 '24 16:05 yiyixuxu

cc @asomoza here

yiyixuxu avatar May 01 '24 16:05 yiyixuxu

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

HuggingFaceDocBuilderDev avatar May 01 '24 16:05 HuggingFaceDocBuilderDev

Thanks for your work, this looks great, I didn't have the chance to be for the first PR so I'm just trying this one out.

I really need to sit down and test the effects of PAG because I find them really interesting, for example some of the blocks I often use for LoRAs:

normal scale 5.0, "m0" scale=5.0, "d4", "d5", "m0" scale=7.0, "d4", "d5", "m0"
20240501160255_2177944627 20240501160317_2177944627 20240501160343_2177944627 20240501160706_2177944627

Still I think it would be better to use SDXL to showcase it, this one is supposed to be a "steam powered mechanic spider robot" but at least PAG makes it more of a "spider".

asomoza avatar May 01 '24 20:05 asomoza

@yiyixuxu Thank you, I removed the white space in the code for code quality check.

HyoungwonCho avatar May 02 '24 05:05 HyoungwonCho

@asomoza Thank you for your interest in our work!

Thanks to multimodalart, users can try PAG on SDXL on Hugging Face Spaces.

For more information, please visit our github page.

HyoungwonCho avatar May 02 '24 05:05 HyoungwonCho

I am struggling with build error. If this is resolved, I will open a new pull request.

HyoungwonCho avatar May 07 '24 07:05 HyoungwonCho