generative-models
generative-models copied to clipboard
Stability-AI
How to train Stable Video Diffusion model?
sorry，ask me Hello！ Do you have a training script for Stable Video Diffusion? Could you please contact me
Our team has released the SVD training script: https://github.com/mindspore-lab/mindone/tree/master/examples/svd It's still under development, but you can use it for reference.
What are the GPU requirements for fine tuning @hadipash ?
Currently a lot (64 GB), but we're working on reducing VRAM usage.
@hadipash thats not bad really. You tested it with sharding across multiple GPU's (eg multiple GPU's with <64gb of VRAM)?
@hadipash thats not bad really. You tested it with sharding across multiple GPU's (eg multiple GPU's with <64gb of VRAM)?
Currently, a sequence of 4 frames can be trained on a single 64GB GPU. We are working on 1) optimizing memory usage on a single device and 2) implementing distributed training to allow for longer sequences (e.g., 30+ frames).
@hadipash hello，I tried using LoRA to fine-tune the U-Net with SVD, and even with a batch size of 1, memory overflow occurs on the A100 GPU when the dataset consists of 25-frame videos. Does this mean that model parallel training must be employed, distributing the model parameters across multiple GPUs?
@DataAIPlayer Not sure about LoRA, haven't integrated it yet. However, for vanilla training - yes, need distributed training as a single 64GB GPU can only fit 4 frames.