video-diffusion-pytorch
video-diffusion-pytorch copied to clipboard
Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch
After updating this commitment, the color of the sampled video fades(not know why); i am using the ucf101 dataset, unconditional training with a 10k step warmup.
Hi, Awesome work first of all. Is there a reason why you would combine both rotational as well as relative positional embedding in your Attention class? I would assume one...
The **_"name text_use_bert_cls is not defined"_** error occurs when trying to use explicit texts as mentioned in the 3rd example. The error occurs as the variable is not directly linked...
Thanks for your effort in implementation. I did not find any code blocks using autograd package to compute gradient as shown in Eq(6) of the paper. Have you implemented this...
Thank you for quickly implementing this model @lucidrains ! Maybe you already have or are planning to do this -- "To manage the computational requirements of training our models, we...
Looking for pointers to get started on modifying the conditioning code below to include conditioning on an image along with text. ``` videos = torch.randn(2, 3, 5, 32, 32) #...
Hi, thanks for this fantastic work! Could I ask what is the training dataset for text-to-video generation? And any training code provided for this task? Thank you so much
Hey @lucidrains, thanks for keeping these models implemented. In line 88 https://github.com/lucidrains/video-diffusion-pytorch/blob/f55f1b0824b1be7d2bb555ed7a5d612eff8ad5d0/video_diffusion_pytorch/video_diffusion_pytorch.py#L84-L88 you have `max_exact` as the half of `num_buckets`, whose value was already halved in line 84. I think...
Is it as simple as adding cond="" on line 944?
https://github.com/THUDM/CogVideo : short high quality videos https://plai.cs.ubc.ca/2022/05/20/flexible-diffusion-modeling-of-long-videos/ : long low quality videos diffusion ans transformers