VBench icon indicating copy to clipboard operation
VBench copied to clipboard

add support for multi-gpu parallel evaluation

Open TiankaiHang opened this issue 1 year ago • 1 comments

TiankaiHang avatar Jul 13 '24 15:07 TiankaiHang

Hi, thanks for the PR and sorry for late response. I think there are some bugs with spatial_relationship and overall_consistency. I think you need to change these lines.

--- a/vbench/overall_consistency.py
+++ b/vbench/overall_consistency.py
-        video_results = all_gather(video_results)
+        video_results = gather_list_of_dict(video_results)

--- a/vbench/spatial_relationship.py
+++ b/vbench/spatial_relationship.py
-        all_results = sum([d['frame_results'] for d in video_results]) / len(video_results)
+        all_results = sum([d['video_results'] for d in video_results]) / len(video_results)

NattapolChan avatar Aug 02 '24 10:08 NattapolChan

merged

NattapolChan avatar Aug 20 '24 11:08 NattapolChan