spider
spider copied to clipboard
Evaluation script can't be used on train_gold labels
Command:
../evaluation.py --gold train_gold.sql --pred train_gold.sql --db database --table tables.json --etype all
Output:
Traceback (most recent call last):
File "../evaluation.py", line 866, in <module>
evaluate(gold, pred, db_dir, etype, kmaps)
File "../evaluation.py", line 505, in evaluate
g_sql = get_sql(schema, g_str)
File "/home/nikolai.korolev/workspace/spider/process_sql.py", line 552, in get_sql
_, sql = parse_sql(tokens, 0, tables_with_alias, schema)
File "/home/nikolai.korolev/workspace/spider/process_sql.py", line 504, in parse_sql
from_end_idx, table_units, conds, default_tables = parse_from(tokens, start_idx, tables_with_alias, schema)
File "/home/nikolai.korolev/workspace/spider/process_sql.py", line 389, in parse_from
idx, table_unit, table_name = parse_table_unit(tokens, idx, tables_with_alias, schema)
File "/home/nikolai.korolev/workspace/spider/process_sql.py", line 259, in parse_table_unit
key = tables_with_alias[tokens[idx]]
KeyError: 'ref_company_types'