mmagic icon indicating copy to clipboard operation
mmagic copied to clipboard

[Feature] Support SDXL training

Open okotaku opened this issue 1 year ago • 3 comments

Motivation

Support SDXL training.

Related PR: https://github.com/open-mmlab/mmengine/pull/1361

Use cases (Optional)

mim train mmagic configs/stable_diffusion_xl/stable-diffusion_xl_pokemon_blip.py

Results

stable-diffusion_xl_pokemon_blip

fake_img_10

weight: https://drive.google.com/file/d/1fzJwdW2yBjEqBDdQ1ILvQ23eVBK43jLO/view?usp=sharing

stable-diffusion_xl_lora_pokemon_blip

fake_img_10

Checklist

Submitting this pull request means that,

Before PR:

  • [x] I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • [x] Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • [x] Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • [x] New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • [x] The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • [x] If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects.
  • [x] CLA has been signed and all committers have signed the CLA in this PR.

okotaku avatar Sep 21 '23 07:09 okotaku

Why do so many cpu tests fail, including some that have not been changed?

https://github.com/open-mmlab/mmagic/actions/runs/6258709202/job/16993265761?pr=2040#step:10:4026

okotaku avatar Sep 21 '23 08:09 okotaku

Codecov Report

Attention: 39 lines in your changes are missing coverage. Please review.

Files Coverage Δ
mmagic/datasets/__init__.py 100.00% <100.00%> (ø)
mmagic/datasets/transforms/__init__.py 100.00% <100.00%> (ø)
mmagic/engine/hooks/__init__.py 100.00% <100.00%> (ø)
mmagic/models/archs/lora.py 15.06% <100.00%> (ø)
...gic/models/data_preprocessors/data_preprocessor.py 97.41% <100.00%> (ø)
mmagic/engine/hooks/visualization_hook.py 95.53% <85.71%> (+1.31%) :arrow_up:
mmagic/datasets/transforms/loading.py 91.10% <78.12%> (-2.04%) :arrow_down:
mmagic/datasets/transforms/sdxl.py 93.06% <93.06%> (ø)
...magic/engine/hooks/lora_checkpoint_to_save_hook.py 56.25% <56.25%> (ø)
...editors/stable_diffusion_xl/stable_diffusion_xl.py 13.20% <0.00%> (-0.27%) :arrow_down:
... and 1 more

:loudspeaker: Thoughts on this report? Let us know!.

codecov[bot] avatar Sep 22 '23 10:09 codecov[bot]

I am fixing the test error on mmengine PR. https://github.com/open-mmlab/mmengine/pull/1365

okotaku avatar Sep 23 '23 05:09 okotaku