VBench
VBench copied to clipboard
add support for multi-gpu parallel evaluation
Hi, thanks for the PR and sorry for late response. I think there are some bugs with spatial_relationship and overall_consistency. I think you need to change these lines.
--- a/vbench/overall_consistency.py
+++ b/vbench/overall_consistency.py
- video_results = all_gather(video_results)
+ video_results = gather_list_of_dict(video_results)
--- a/vbench/spatial_relationship.py
+++ b/vbench/spatial_relationship.py
- all_results = sum([d['frame_results'] for d in video_results]) / len(video_results)
+ all_results = sum([d['video_results'] for d in video_results]) / len(video_results)
merged