ML-Bench icon indicating copy to clipboard operation
ML-Bench copied to clipboard

Published 20 hours ago • verified publisher icon gersteinlab

The GPT3.5 result could not be reproduced

Open iiinsight opened this issue 11 months ago • 1 comments

I found that two parameters in script/run.sh were running incorrectly, where the type="quarter" parameter was not defined or used in query_gpt.py. The instructions="extend_instructions" parameter also returns KeyError: 'extend_instructions' How do I reproduce the GPT3.5 results in the paper in ML-Bench?

image

image

Processing the data. 0%| | 0/292 [00:00<?, ?it/s] Traceback (most recent call last): File "query_gpt.py", line 239, in <module> extend_instructions=line[f'{args.instruction}'] KeyError: 'extend_instructions' (py38torch1101)

iiinsight avatar Mar 25 '24 00:03 iiinsight