lm-evaluation-harness icon indicating copy to clipboard operation
lm-evaluation-harness copied to clipboard

id_column contains duplicate values

Open nicho2 opened this issue 10 months ago • 0 comments

I tried to send data to zeno visualisator but i have this error:

py ./scripts/zeno_visualize.py --data_path ./output/ --project_name "LG Model Test" Successfully updated project. Access your project at https://hub.zenoml.com/project/460c-8698-59a31942ea41/LG%20Model%20Test_gsm8k Traceback (most recent call last): File "C:\Projects\transpose\lm-evaluation-harness\scripts\zeno_visualize.py", line 219, in main() File "C:\Projects\transpose\lm-evaluation-harness\scripts\zeno_visualize.py", line 103, in main project.upload_dataset( File "C:\Python311\Lib\site-packages\zeno_client\client.py", line 117, in upload_dataset pa_table = df_to_pa(df, id_column) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Python311\Lib\site-packages\zeno_client\util.py", line 22, in df_to_pa raise ValueError("ERROR: id_column contains duplicate values") ValueError: ERROR: id_column contains duplicate values

it's because result are duplicate in the log file.

command is: lm_eval --model local-chat-completions --tasks gsm8k,babi --model_args model=sn/mistral-7b-instruct-v0.2-LG,base_url=http://10.2.42.198:1234/v1 --log_samples --output_path output/mistral-7b-instruct-v0.2-LG --limit 4

it's concerning only gsm8k task

nicho2 avatar Apr 03 '24 13:04 nicho2