Brian Yu

@asaparov Any updates? Running into the exact same error with a private model on CUDA 11.0 on p4d.24xlarge AWS EC2 instances

I've examined a few more of these functions, some from the beginning and some from the end. Seems that the `model_answer` property is not very accurate? Please correct my ground...

Hi Fanjia! Gotcha, thanks for the response! I'm still a little confused sorry. Where is the ground truth used for the OpenFunctions test dataset if `model_answer` is not the reference?...

Hi Fanjia! Gotcha, just to confirm -- I should not evaluate using `test.json` because the ground truths may be wrong? If so, how should I reproduce the evaluation numbers in...