video-diffusion-pytorch

video-diffusion-pytorch copied to clipboard

the commitment: one more residual

23

After updating this commitment, the color of the sampled video fades(not know why); i am using the ucf101 dataset, unconditional training with a 10k step warmup.

liangbingzhao

Reason for combining rotary and relative positional embedding?

1

Hi, Awesome work first of all. Is there a reason why you would combine both rotational as well as relative positional embedding in your Attention class? I would assume one...

oxjohanndiep

Noisy output & "text_use_bert_cls" error

31

The **_"name text_use_bert_cls is not defined"_** error occurs when trying to use explicit texts as mentioned in the 3rd example. The error occurs as the variable is not directly linked...

GoutamKelam

Gradient method for conditional sampling

4

Thanks for your effort in implementation. I did not find any code blocks using autograd package to compute gradient as shown in Eq(6) of the paper. Have you implemented this...

LuckyDC

Generating longer videos at test time

46

Thank you for quickly implementing this model @lucidrains ! Maybe you already have or are planning to do this -- "To manage the computational requirements of training our models, we...

mrkulk

Conditioning on image + text embedding

4

Looking for pointers to get started on modifying the conditioning code below to include conditioning on an image along with text. ``` videos = torch.randn(2, 3, 5, 32, 32) #...

ChintanTrivedi

Training Dataset

2

Hi, thanks for this fantastic work! Could I ask what is the training dataset for text-to-video generation? And any training code provided for this task? Thank you so much

Mq-Zhang1

Duplicate dividing in relative positional encoding

1

Hey @lucidrains, thanks for keeping these models implemented. In line 88 https://github.com/lucidrains/video-diffusion-pytorch/blob/f55f1b0824b1be7d2bb555ed7a5d612eff8ad5d0/video_diffusion_pytorch/video_diffusion_pytorch.py#L84-L88 you have `max_exact` as the half of `num_buckets`, whose value was already halved in line 84. I think...

songweige

Is it as simple as adding cond="" on line 944?

samlhuillier

related recent work

1

https://github.com/THUDM/CogVideo : short high quality videos https://plai.cs.ubc.ca/2022/05/20/flexible-diffusion-modeling-of-long-videos/ : long low quality videos diffusion ans transformers

rom1504