Haibin Wang

I ran the 65B model in 8 * A100 (80G). But I found that it stuck in allreduce and reported the following error with my own edited prompt. `RuntimeError: NCCL...

[Usage] Inference with batches

1

### Describe the issue I try to run inference with batches to accelerate the process. However, I find it takes just as much time as inferring each instance individually.

Add tools for FVD, KVD, ISV, PRV, FID, KID, IS, PR eval for videos.

Hi, Can you release the code for the computation of KL reduction in Figure 3 in the paper? Thank you very much!

Reproduction of experiments

10

Hi, We follow the training pipeline in `experimental` to replicate the DSIR results. However, our average performance reached only 81.05, significantly below the reported benchmark of 82.30. Are there any...

1. add data executor hook 2. add data partition executor

Problems Encountered during Installation

20

Hi, I install VBench with pip, but I encountered some problems. 1. I encountered the following error when evaluating motion_smoothness: 2. At line 61 in `vbench/dynamic_degree.py`, line 110 in `vbench/motion_smoothness.py`...

About the size and number of frames

2

Hi, Dose Vbench have limitation on the video size and the number of frames for generated videos? What is the setting of the video size and the number of frames...

Eval on MMVET

1

Hi, I try to eval the model in MMVET. But there is no such file `scripts/eval_gpt_mmvet.py` in your code.

Vbench Leaderboard

2

Hi, How can I get the Total Score, Quality Score, Semantic Score and Selected Score locally?