Songyang Zhang
Songyang Zhang
We can consider adding this to our backlog, @liushz.
Thanks for the report, we will fix this issue. Actually, our design is compatible with `id` or relative path. Some datasets need `get_data_path(path, local_mode=True)`.
Can you launch the evaluation with --debug?
Actually, I have no idea about this bug. Can you use tmux and then try to remove --debug?
Please check the prediction to find if there is a repeat pattern in response. And reduce the --max-out-len. Also, you can remove the --debug and use four workers.
The best option is to use vllm or lmdeploy, because the math requires the model to generate the reasoning process.
Please provide the details of the bug.
Thanks for your contribution, we would like to provide essential support.
Please make sure you have installed the opencompass successfully in your environment.
Please fix the lint issue