MoE-LLaVA icon indicating copy to clipboard operation
MoE-LLaVA copied to clipboard

Published 20 hours ago • verified publisher icon PKU-YuanGroup

is support video ?

Open awzhgw opened this issue 1 year ago • 3 comments

is support video train?

awzhgw avatar Feb 02 '24 08:02 awzhgw

Sure. Our code supports multi-image training, multi-video training, and even image-video training together.

LinB203 avatar Feb 02 '24 09:02 LinB203

@LinB203 is there any video predict code in the repo to test it on a mp4 file? Or a preprocessing script showing how to sample the frames from a video?

fcakyon avatar Feb 04 '24 11:02 fcakyon

This repo support training video but do not release checkpoint about video. So you may need to train a new model to support video. For video predict code or how to adapt video encoder into MoE-LLaVA, you can refer to Video-LLaVA.

LinB203 avatar Feb 04 '24 11:02 LinB203