Evaluation issue:

[Mei0211]

I wrote a script for evaluation,but I encoutered a issue then: 1、script: from vbench import VBench import torch device = torch.device("cuda") my_VBench = VBench(device, "VBench_full_info.json", "evaluation_results") my_VBench.evaluate( videos_path = "./videocrafter/spatial_relationship", #there are several videos in this directory name = "spatial_relationship", dimension_list = ["spatial_relationship"], ) 2、issue: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount)

3、evaluate results:

[JustinYuu]

Hi, thanks for your interest in our work! The issue may be attributed to the wrong sampled video format. As we mentioned in the How to Sample Videos for Evaluation section, the names of videos should be in the form of $prompt-$index.mp4. You can refer to the $dimension_full_info.json in the evaluation_reuslts/ folder to check if the sampled videos exist in the “video_list" key of each prompt.

[mei1127]

Thanks for your information！I‘ll try it later！

[ziqihuangg]

Hi, we're closing this issue as it appears your questions have been addressed. However, feel free to open a new issue or reopen this one, if you have further questions or if anything else comes up related to this issue.