How to comprehend the evaluation results?

[yaoyiyao-yao]

Hello， when I get the evaluation results,I am not sure the meaning of "predicted_parse_error" and "exact".I guess when "predicted_parse_error" is true it means the model can’t produce a predicted sql,is that right?And I found "exact" has 3 possibilities:true,0 and false,I guess when "exact" is true ,it means the predicted sql is right.But what do 0 and false mean? Thank you.

[Impavidity]

You can use the official evaluation script https://github.com/taoyds/spider to evaluate the outputs. For customized evaluation in the codebase, "predicted_parse_error": true means the model cannot produce sql. both 0 and false means the predicted sql is wrong.