lm-evaluation-harness icon indicating copy to clipboard operation
lm-evaluation-harness copied to clipboard

Published 20 hours ago • verified publisher icon EleutherAI

[New Task] CommonsenseQA

Open haileyschoelkopf opened this issue 1 year ago • 4 comments

https://github.com/jonathanherzig/commonsenseqa https://huggingface.co/datasets/commonsense_qa

Evaluated on by Llama 2 + Mistral

haileyschoelkopf avatar Nov 27 '23 16:11 haileyschoelkopf

Hi. I just want to know if there is any update on this? Thanks

Davido111200 avatar Apr 01 '24 10:04 Davido111200

Hi! There has not been yet--we'd still welcome a contribution!

I may look at it at some point in the future but have been prioritizing framework/library improvements.

haileyschoelkopf avatar Apr 01 '24 13:04 haileyschoelkopf

@haileyschoelkopf I'm interested in taking this on if no one is currently working on it.

murphybrendan avatar Apr 17 '24 19:04 murphybrendan

@haileyschoelkopf I'm interested in taking this on if no one is currently working on it.

Thank you! Please do.

StellaAthena avatar Apr 17 '24 20:04 StellaAthena