Songwei Ge

Hi Kang, good point. I think you are right. I suspect that if you mask out the loss on the initial and last frames, it shouldn't affect the model performance.

Thank you for your interest in our project! We are working on the final steps of conditional training scripts as well as a colab notebook. The notebook is almost done...

We have added the relevant instructions on how to train a conditional transformer and sample videos with it. Please check the ReadMe file and feel free to let me know...

Hi, if you want to fine tune from a checkpoint, you can use the `--resume_from_checkpoint=your_checkpoint_file` when using any training scripts. If you are using pyotch-lightning > 2.0, this should be...

We used an old, internal version of the MUGEN dataset. I will be working on the improvements to support the new official dataloader. I don't have any experience with the...

Hi, thanks for your interests! (a) For FVD and KVD metrics, please check https://github.com/SongweiGe/TATS#synthesis for the scripts. For IS, we generally generated 10K samples and compute it with the code...

It looks like you have successfully generated samples but failed when loading the real videos. You need to specify the `data_path` to be `./TATS/data/sky_timelapse/` with two subfolders called `train/` and...

Nice. Your Taichi command looks good to me except that you are missing `--sample_every_n_frames 4``. This should only affect the real embeddings. To get a quick sanity check to see...

Can you try to add `--resolution 128 --sequence_length 64` to your command?

I'm more than happy to help. This looks like a dataloader issue. Usually it happens when your cache file has a different path from where your videos are really stored....