VBench

VBench copied to clipboard

Hi, I know vbench has supported the evaluation on customized videos. However, it seems vbench_i2v has not yet, right? If so, how soon it would be support that? Thanks! Thanks!

motor-x

Can not download pretrained model for "overall consistency" evaluation

4

I've tried the following command to download the pretrained model. `wget https://pjlab-gvm-data.oss-cn-shanghai.aliyuncs.com/umt/single_modality/l16_ptk710_ftk710_ftk400_f16_res224.pth -P ~/.cache/vbench/umt_model` But I have encountered the following error messages: ``` > --2024-08-09 21:47:30-- https://pjlab-gvm-data.oss-cn-shanghai.aliyuncs.com/umt/single_modality/l16_ptk710_ftk710_ftk400_f16_res224.pth > Resolving pjlab-gvm-data.oss-cn-shanghai.aliyuncs.com...

haoyuhsu

When evaluate aesthetic_quality, 'CUDA out of memory' always occur for multiple videos

2

When evaluating aesthetic_quality for my own videos, I always get the error 'CUDA out of memory', which leading me not be able to batch test. Is there any problem? And...

Crazy-wu-20

add support for multi-gpu parallel evaluation

1

TiankaiHang

VBench-Long Leaderboard

3

Dear authors @ziqihuangg @MqLeet , thank you for your excellent work. I am curious if there is a leaderboard for VBench-Long to facilitate easy comparison of long video generation models?

Epiphqny

lmxyy

failed to get the output result

1

I ran this command: `vbench evaluate --videos_path "/home/notebook/code/group/hkx/video_tasks/dover/DOVER/demo" --dimension "motion_smoothness"` then it worked like this below: `args: Namespace(func=, output_path='./evaluation_results/', full_json_dir='./VBench_full_info.json', videos_path='/home/notebook/code/group/hkx/video_tasks/dover/DOVER/demo', dimension='motion_smoothness', load_ckpt_from_local=None, read_frame=None) start evaluation Evaluation meta data saved...

mamianshusheng

Difference between "i2v_subject" and "subject_consistency" dimensions

6

AshkanTaghipour

Vbench Leaderboard

2

Hi, How can I get the Total Score, Quality Score, Semantic Score and Selected Score locally?

BeachWang

error with torch.distributed for multiprocesssing

6

Hi, I want to use VBench with `torch.distributed` for multiprocessing evaluation, however, I found only the first process can finish while all the rest processes cannot successfully finish. Here is...

quantumiracle