很多评测数据集的post_check函数报错

Open LIRENDA621 opened this issue 8 months ago • 2 comments

我在用Qwen2.5-VL-Instruct-3B模型评测时，在很多数据集的post_check函数中遇到下面的警告，例如MathVision mathv.py: post_check - 128: <class 'ValueError'>: signal only works in main thread of the main interpreter

不清楚为什么会出现这个警告，以及这个警告是否会影响评测的精度？

Apr 09 '25 02:04 LIRENDA621

Apr 12 '25 07:04 kooWZ

Changing the line to @timeout_decorator.timeout(30, use_signals=False) at https://github.com/open-compass/VLMEvalKit/blob/7c47fcaedcd9e3eff1546cb41da84c090a64679c/vlmeval/dataset/utils/mathv.py#L14 fixes this error on MathVision. See https://github.com/pnpnpn/timeout-decorator#multithreading

Apr 14 '25 05:04 kooWZ