Wrong evaluation reply using ChatGPT

Hi,

When I use my output.json and the repo's test_eval.json, it worked the first two times. However, now I see ChatGPT replies such as:

I would rate your answer as 10.

which leads to the following error:

evaluation/0 [0]:  File "/workspace/evaluation.py", line 194, in <module>
evaluation/0 [0]:    output = evaluation.evaluation()
evaluation/0 [0]:  File "/workspace/evaluation.py", line 153, in evaluation
evaluation/0 [0]:    scores["match"] = self.eval_match()
evaluation/0 [0]:  File "/workspace/evaluation.py", line 71, in eval_match
evaluation/0 [0]:    outs2 = self.eval_chatGPT(self.match["GPT"])
evaluation/0 [0]:  File "/workspace/evaluation.py", line 46, in eval_chatGPT
evaluation/0 [0]:    scores = list(map(float, scores))
evaluation/0 [0]:ValueError: could not convert string to float: 'I would rate your answer as 10.'

Based on previous issues https://github.com/OpenDriveLab/DriveLM/issues/92, https://github.com/OpenDriveLab/DriveLM/issues/53, it seems like this should have been fixed, but I'm still facing this issue. Can someone help me out?

Thank you for your interest. Are you running eval on your local env or our test server?