HelloWorldBeginner

I train controlnet_sdxl in bf16 datatype, got unsupported ERROR in datasets

1

### Describe the bug ``` Traceback (most recent call last): File "train_controlnet_sdxl.py", line 1252, in main(args) File "train_controlnet_sdxl.py", line 1013, in main train_dataset = train_dataset.map(compute_embeddings_fn, batched=True, new_fingerprint=new_fingerprint) File "/home/miniconda3/envs/mhh_df/lib/python3.8/site-packages/datasets/arrow_dataset.py", line...

Add Ascend NPU support for SDXL fine-tuning and fix the model saving bug when using DeepSpeed.

11

# What does this PR do? Added support for SDXL finetune on AscendNPU and fixed the bug causing the hang out when saving models using the deepspeed distributed framework. DeepSpeed...

Add NPU support for Llava

7

[NPU] Support Llava training and inference for Ascend NPU. I've modified some codes to add support for NPU, allowing LLAVA to perform both training and inference on NPU. It works...

Add Ascend NPU support for SDXL and fix bugs

1

# What does this PR do? 1. Adds NPU flash attention support for NPU, similar to #7816. 2. Fixes a bug related to saving the model when using deepspeed, also...

I use image_finetune.yaml to fine-tune the model, and I found that the first step of each epoch would get stuck for 5 minutes

5

I changed the train_step parameter inside image_finetune.yaml to 2000 steps, which will train multiple epochs, but the machine gets stuck for five minutes at the start of each epoch. ...

Add AscendNPU support for Animatediff

1

I've modified some codes to add support for Ascend NPU, allowing Animatediff to train and inference on NPU. It works fine on NPU. NPU training  NPU Inference 

### Question LLava is a great work, I have adapted llava to Ascend NPU hardware, enabling pre-training, inference, and evaluation on the Ascend NPU. I'm wondering if NPU is also...