diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard
Published 1 month ago • verified publisher icon huggingface

NPU Adaption for FLUX

Open leisuzz opened this issue 1 year ago • 4 comments

What does this PR do?

Implement Flash Attention for NPU machine Add cleaning memory function for NPU machine

Before submitting

  • [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [x] Did you read the contributor guideline?
  • [x] Did you read our philosophy doc (important for complex PRs)?
  • [x] Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
  • [x] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • [x] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

leisuzz avatar Oct 23 '24 07:10 leisuzz

@yiyixuxu @sayakpaul Please let me know if there is any improvement for this PR, and thanks for your help!

leisuzz avatar Oct 24 '24 15:10 leisuzz

can you run make style and make fix-copies to the CI would pass?

yiyixuxu avatar Oct 25 '24 22:10 yiyixuxu

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

HuggingFaceDocBuilderDev avatar Oct 25 '24 22:10 HuggingFaceDocBuilderDev

@yiyixuxu I've fixed the code quality issue, please approve the workflow. Thanks

leisuzz avatar Oct 26 '24 00:10 leisuzz

@yiyixuxu Sorry about my mistake, could you please approve the workflow? Thanks

leisuzz avatar Oct 28 '24 01:10 leisuzz

hi @leisuzz if you installed our dev environment, you can run make style and make fix-copeis from the terminal that will automatically adjust the style for you so the CI will pass

see here https://huggingface.co/docs/diffusers/en/conceptual/contribution#how-to-open-a-pr

yiyixuxu avatar Oct 28 '24 18:10 yiyixuxu

@yiyixuxu Thanks for the information!

leisuzz avatar Oct 29 '24 01:10 leisuzz

Hi @yiyixuxu , Sorry for the trouble, I think there is something wrong with the ruff in my env as there is no code quality in my end. But I think I've fix it manually. Please approve the workflow. Thanks for your help

leisuzz avatar Oct 31 '24 00:10 leisuzz

@yiyixuxu Could you please redo the approval of the workflow? Thanks for your help and sorry for all the troubles!

leisuzz avatar Oct 31 '24 01:10 leisuzz

Hi @yiyixuxu , There was a copyright / consistency issue with "def fuse_qkv_projections" as it has to match models.unets.unet_2d_condition.UNet2DConditionModel.fuse_qkv_projections. Does that mean I have to change both of these at the same time? For now, please accept this workflows as fuse_qkv_projections can be added later in another PR.

leisuzz avatar Oct 31 '24 06:10 leisuzz

@yiyixuxu @sayakpaul Could you please approve the workflow? Thanks for your help!

leisuzz avatar Nov 01 '24 02:11 leisuzz

Failing tests are unrelated.

sayakpaul avatar Nov 01 '24 03:11 sayakpaul

@yiyixuxu I'm not completely sure why we merged this PR with the following changes:

image

I believe we should default to just the normal SDPA processors otherwise there is inconsistencies introduced. Users can already set custom attention processors, no?

a-r-r-o-w avatar Feb 23 '25 21:02 a-r-r-o-w

@a-r-r-o-w yeah we should revert this change - it was my fault, I realized we should not do that here https://github.com/huggingface/diffusers/pull/10465/files

yiyixuxu avatar Feb 24 '25 17:02 yiyixuxu