diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

Fix key error for dictionary with randomized order in convert_ldm_unet_checkpoint

Open yunseongcho opened this issue 10 months ago • 6 comments

What does this PR do?

I found that for some model weights, the key values in the dictionary are not applied in order, and this can cause a potential KeyError.

For example, on line 560 of convert_from_ckpy.py, the key value of ["conv.bias", "conv.weight"] in output_block_list is "2", but if the order is first ({"2": ["conv.bias", "conv.weight"], "0": [...], ...}), then on line 563 the index is "0" instead of "2". This causes a key error.

Because we get the value of unet_state_dict through index, not the key, a KeyError occur if the order of the output_block_list and index do not match.

To prevent this from happening, this PR ensures that index and key are in the same order.

Fixes # (issue)

Before submitting

  • [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [x] Did you read the contributor guideline?
  • [x] Did you read our philosophy doc (important for complex PRs)?
  • [ ] Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
  • [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • [ ] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR. @sayakpaul @yiyixuxu @DN6

yunseongcho avatar Apr 15 '24 05:04 yunseongcho

@yunseongcho Would you mind sharing a checkpoint we could use to test with?

DN6 avatar Apr 15 '24 06:04 DN6

@DN6 I tested with the PastelMix model https://civitai.com/models/5414/pastel-mix-stylized-anime-model

The model weights link: https://civitai.com/api/download/models/6297 I used "wget" command with above link

yunseongcho avatar Apr 15 '24 07:04 yunseongcho

This is not an error that happens all the time, it happens in some cases and not in others.

yunseongcho avatar Apr 15 '24 07:04 yunseongcho

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@DN6 Hello, Could I get your feedback? I'd love to hear your feedback

yunseongcho avatar Apr 30 '24 05:04 yunseongcho

Hi @yunseongcho sorry for the delay. Yes I'm okay to merge this.

DN6 avatar May 01 '24 12:05 DN6