Extend CogVideoX2B to 5B I2V

[OrangeSodahub]

Hi, thanks for your excellent contributions! I noticed you directly extend 2B model to 5B and finetune it, resulting from t2v to ti2v. But is there any more details about that? How did you manage the pretrained parameters and the untrained paramters? For example, the input channels of transformer block will be doubled in 5B model, how did you place the pretrained half-channel parameters?

Any suggestions would be appreciated!

[OrangeSodahub]

@zRzRzRzRzRzRzR Hi, I believe your train 2B and 5B model both from scratch. However I was wondering if it is possible I directly extend the 2B T2V model to a I2V model by finetuning?

[zRzRzRzRzRzRzR]

Of course, it is supported, but this requires you to implement it yourself. You can use the doubled channels to store the image lantent as the condition for training.