Vchitect
prompt&video
Great work, I would like to know does the name of the video have to be the same as the prompt If the prompt is exceptionally long and exceeds the maximum value of the video name how should it be executed?
The vbench standard prompt in here will not exceed the maximum value of file name length. But in case you have your own prompt and that is longer than 255, you can use these commands:
... --videos_path <path/to/video.mp4> --mode=custom_input --prompt <your prompt>... --videos_path <path/to/video_dir> --mode=custom_input --prompt_file <path/to/json file that map filenames to prompts>
Thank you for your reply, but I have a new problem, I tried to run it and found that there are some metrics with negative scores, like temporal_style, is this as expected and what does the negative number mean?
hi，my evaluation instructions are as follows： vbench evaluate --dimension subject_consistency --videos_path v1.mp4 --mode=custom_input --prompt A man is dancing
evaluation framework returns error: vbench: error: unrecognized arguments: man is dancing
Are there any plans in the future to support custom prompts and video reviews in the Controllability dimension?
Thank you for your interest. Custom video evaluation can only be done on the dimension without auxiliary_info and Controllability is not satisfied. We update this feature for five dimensions. Please refer to https://github.com/Vchitect/VBench/tree/master/VBench-2.0#new-evaluating-single-dimension-of-your-own-videos
hi，my evaluation instructions are as follows： vbench evaluate --dimension subject_consistency --videos_path v1.mp4 --mode=custom_input --prompt A man is dancing
evaluation framework returns error: vbench: error: unrecognized arguments: man is dancing
Seems like you didn't "quote" your input prompt
Closing the issue as all questions appear to be addressed. Feel free to reopen it if you believe anything remains unresolved.