freq

How to evaluate on llama3-8b-instruct?

1

How to evaluate on llama3-8b-instruct? Please add the function, thanks!

Add long context evaluation benchmarks such as LongBench and LEval.

2

Add long context evaluation benchmarks such as LongBench and LEval.

How you evaluate reasoning models like QwQ-32B, since the response time and token length is very long?

5

Any adjustments to the hyperparameters in pred.py?

Questions on calculating FVD, FID and IS scores

1

When calculating FVD, FID and IS scores, how many fake videos (sample.mp4 ) need to be generated? Whether you use all real video frames when calculating these scores? CUDA_VISIBLE_DEVICES=gpu_id python...

how to use a local LLM to evaluate prediction quality? For example, Llama-3-70B-Instruct?

1

### Feature request / 功能建议 how to use a local LLM to evaluate prediction quality? For example, Llama-3-70B-Instruct? ### Motivation / 动机 how to use a local LLM to evaluate...