Xiaodong Wang

icon indicating a website link https://wang-xiaodong1899.github.io/

icon company Peking University icon location PhD student & MS at PKU, Ex intern @Microsoft Asia, @ SenseTime, @megvii-research

Results 5 issues of Xiaodong Wang

Why the video does not pass through the encoder?

Hi! lucidrains. Thanks for providing a great repo which is convenient to understand the NUWA paper. I have a question as follows: In the NUWA paper, we can see that...

[feat] Add CLIP support

1 comment

add clip support to train_t2v.py. This PR does that.

Update README.md

update readme.md

About the input video frames

1 comment

Hi! Thanks for your great work! I have looked into the code about video processing, but I did not figure out **how many frames** of a video are encoded into...

fix prepare_inputs_labels_for_multimodal in llava_arch

if `image_idx not in video_idx_in_batch`, `image_feature` will be added into `new_image_features` repeatedly, which should be avoided. That's what this PR does.