TokenFlow
TokenFlow copied to clipboard
omerbt
Official Pytorch Implementation for "TokenFlow: Consistent Diffusion Features for Consistent Video Editing" presenting "TokenFlow" (ICLR 2024)
Hi, thank you for your nice work. Is the part of th code 'NN field compute & warp' is this code? def prepare_depth_maps(self, model_type='DPT_Large', device='cuda'): depth_maps = [] midas =...
Thank you for your paper. Is joint editing in paper use pretrained 'Tune-A-Video' model?
Hi @omerbt , Great work on `TokenFlow` and congratulations on the ICLR acceptance! This pull request makes it possible to run `TokenFlow` on Replicate (https://replicate.com/cjwbw/TokenFlow) and via API (https://replicate.com/cjwbw/TokenFlow/api). Also...
test cmd: python preprocess.py https://github.com/omerbt/TokenFlow/assets/48466610/3fee547d-f65c-4af0-bee7-5712229c582d
There are some confusing parts between the usage of the two pairs in tokenflow_utils.py: 1 reshape_heads_to_batch_dim and head_to_batch_dim 2 reshape_batch_dim_to_heads and batch_dim_to_head For example, in tokenflow_utils.py, head_to_batch_dim appears in two...
Hi, May I ask for the script for getting the Warp-error metric? Or is there any code base that I can refer to? Thanks a lot. Best,
This is an excellent project, but I'm curious, does each video need preprocessing before editing? Thank you very much.
As discussed previously, I implemented [edit-friendly DDPM inversion](https://arxiv.org/abs/2304.06140) drastically reducing runtime for the inversion step. Additionally, the edit quality/robustness benefits from the inversion guarantees of DDPM, thus eliminating any inversion...
@MichalGeyer @duongna21 @omerbt If I use any other SD versions other than the 1.5, 2.0 -base, 2.1-base I'm getting random images after preprocess. I have used 2.1-unclip with image conditioning...
@MichalGeyer Can we use SD v2.1 Unclip architecture to condition on Image and Text to replace the subject with another subject similar to Make-A-Protagonist (https://github.com/HeliosZhao/Make-A-Protagonist)?