MotionAdapter.from_single_file

[eps696]

Model/Pipeline/Scheduler description

Animatediff is a great technique with exploding interest and advancements; additional custom models start popping around in ckpt format, which cannot be loaded to diffusers in the meantime. is it possible to implement from_single_file function for motion adapter, similar to how it's done now for say controlnet?

Open source status

• [X] The model implementation is available.
• [X] The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

No response

[DN6]

Hi @eps696 this is definitely something we can consider. We're working on a clean up of from_single_file at the moment so it might take a bit of time. Could you point me to these new checkpoints? I can see if there's a way to convert them to diffusers format as an intermediate solution.

[eps696]

thanks for a prompt response! here is few examples: https://github.com/Lightricks/LongAnimateDiff https://huggingface.co/guoyww/animatediff/tree/main [see v3 versions]

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[rishabhjain]

Agree that from_single_file method will be helpful. Since animatediff was added to diffusers, new versions for SD as well as SDXL has been released. But unable to use them, as from_single_file isn't supported and those new models are usually a single .ckpt files.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[vladmandic]

@yiyixuxu @DN6 not sure why gh bot chose to close this as stale label was removed?

[eps696]

+1, just yesterday tried fresh release - from_single_file not implemented yet for motion adapter

[yiyixuxu]

ohh sorry 😱 I didn't know it does that

not sure why gh bot chose to close this as stale label was removed?

[DN6]

Hi @eps696 have you tried using the AnimateDiff conversion scripts with those checkpoints?

https://github.com/huggingface/diffusers/blob/main/scripts/convert_animatediff_motion_module_to_diffusers.py https://github.com/huggingface/diffusers/blob/main/scripts/convert_animatediff_motion_lora_to_diffusers.py

[eps696]

Hi @eps696 have you tried using the AnimateDiff conversion scripts with those checkpoints?

no, missed it somehow, thanks for a hint!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[vladmandic]

ping to remove stale status.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[yiyixuxu]

@DN6 I think this is not supported yet with the new from_single_file - do we plan to support it? should we ask community to help?

[DN6]

Yes we can 👍🏽 Was planning on adding the support this week, but definitely a good one for the community as well.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[vladmandic]

ping to remove stale status.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[vladmandic]

ping to remove stale status.

[DN6]

Hi @vladmandic we merged #8819 with single file support for MotionAdapter a little while back.

[DN6]

Closing since this has been completed.