[Tracker] use the new model card utilities for saving model cards from the training script

[sayakpaul]

With https://github.com/huggingface/diffusers/pull/6729, we shifted to a more resilient and robust way to generate model cards from the DreamBooth training scripts.

So, we're keen for the community to help us do this for the other training scripts that save model cards:

• [x] T2I
• [x] T2I LoRA
• [x] T2I SDXL
• [x] T2I SDXL LoRA
• [x] ControlNet
• [x] ControlNet SDXL
• [x] T2I Adapter SDXL
• [x] Textual inversion
• [x] Textual inversion SDXL
• [ ] Advanced diffusion training (cc: @linoytsaban)
• [ ] Advanced diffusion training SDXL (cc: @linoytsaban)

While creating PRs, please also supply a notebook that generates the model card for a respective example. An example is available here: https://github.com/huggingface/diffusers/pull/6729#issue-2103496495. Use https://github.com/huggingface/diffusers/pull/6729 as a reference (only refer to the changes made in the training scripts). Please also target ONLY ONE example at a time in your PRs.

Please also tag me for a review when you open the PRs.

[sayakpaul]

All yours!

[bamps53]

Hi @sayakpaul, I checked #6729 and attempted to run your test notebook (https://huggingface.co/sayakpaul/test-model-card-template-dreambooth/blob/main/test_dreambooth_model_card.ipynb). However, it appears to not work for #6729 as it uses the instance_prompt keyword arguments in load_or_create_model_card. Changing it to prompt seems to fix the issue. My question is about the maintenance of these notebooks. Is it ok to create a notebook just to verify that my PR is working at the time?

[bamps53]

Oh, it seems not only notebooks, but also training scripts in examples are not workings now. I made PR #6895 to fix these before this issue.

[Bhavay-2001]

Hi @sayakpaul, I would love to work on this issue. I read the #6729 and based on my understanding what we have to do is -

1. We need to add a training script for these models where we append the changes you mentioned in #6729. Basically all those changes that you made to this file examples/dreambooth/train_dreambooth_lora.py
2. We need to use your notebook and generate model_cards for different models as well right?

[Bhavay-2001]

If this is the case then I would be happy to work on ControlNet and ControlNetSDXL.

[sayakpaul]

@bamps53 I have fixed the example notebook to clear the errors. So, that should be fine. Have reviewed your PR: https://github.com/huggingface/diffusers/pull/6895 as well. It looks solid.

I have also updated the description of this issue, clearly mentioning that contributors should open one PR at a time, targeting only one example. Hope that helps.

[sayakpaul]

@Bhavay-2001 I don't think you need to add any training script. We need to make use of the utilities (as shown in my example notebook and also in the PR) to create the model cards of the examples I have listed in the description. Is it more clear?

[bamps53]

@sayakpaul Thanks, I got it! I'll split the PR for each.

[bamps53]

Sorry for the mess, but I created 3 PRs for each controlnet training scripts! #6908 #6909 #6910

By the way it was more tadious to create notebooks than modifying code itself... Any better way?

[sayakpaul]

I think you misunderstood the description. The PR should target the actual training scripts. I asked for a notebook example to be able to quickly verify if the changes lead to expected README. The notebook can be a part of the PR description but not a part of the PR changes. If you check the reference PR, this is exactly what I have done.

Not sure how I can convey this more thoroughly.

[bamps53]

Yes, I believe I understand that. I didn't include the notebook in the PR. Just attached it to the description of each PR.

[cosmo3769]

Hello @sayakpaul, I would love to work on all the T2I ones. Thank you!

[sayakpaul]

Yes, feel free to!

[chiral-carbon]

hi @sayakpaul could I claim Advanced Diffusion Training and Advanced Diffusion SDXL?

[Stepheni12]

I'll take Textual Inversion SDXL if it's available!

[sayakpaul]

@chiral-carbon, @Stepheni12 feel free to proceed :)

[Bhavay-2001]

Hi @sayakpaul, I have also some pending PRs from a previous issue #6585. Can you please check them as well?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[chiral-carbon]

Hi sorry, I’m going to push a PR by the end of the week and was delayed due to some heavy backlog in my work, sorry about it.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[chiral-carbon]

Hi @sayakpaul sorry for the delay! I have opened 2 new PRs for the tasks I had picked up.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.