bugfix: de-tensorized and fix index issues

[Nana2929]

Description

The correct_predictions.csv and incorrect_predictions.csv should supposedly contain qid. They do not, so I try to map back the qid using image_name and question_text in a batch (which is a valid approach because in testset.json the 2 keys do form a composite unique key).
However I find these 2 files have overlap in the mapped-back qid.
Upon reading the codes, I find that the 4 for loops in test.py from #L117 to #L144 have indexing issues (should use batch index ind to index elements in the batch loaders; use i for indexing open and close logits, the original codes kind of mix them up I guess).

Related Issue

• None

Motivation and Context

• The original prediction files have tensors inside; do not provide the decoded answers.
• The original prediction files have indexing issues.

How Has This Been Tested?

It is verified that with the test.py in this PR, the 2 correction files can map to non-overlap qid sets that can union to the full testset.

trainset = load_json(Path(data_dir)/"trainset.json") 
testset = load_json(Path(data_dir)/"testset.json")
trainset = to_dataframe(trainset)
testset = to_dataframe(testset) 

correct_df = pd.read_csv(Path(result_dir)/'correct_predictions.csv')
incorrect_df = pd.read_csv(Path(result_dir)/'incorrect_predictions.csv') 

# need to use question + image_name to map back
def qid_map(r: pd.Series):
    q = r['question']
    img = r['image_name']
    assert len(testset[(testset['question'] == q) & (testset['image_name'] == img)]) == 1
    return testset[(testset['question'] == q) & (testset['image_name'] == img)]['qid'].values[0]

qtype_map = testset[['qid', 'question_type']].set_index('qid').to_dict()['question_type']
for df in [correct_df, incorrect_df]:
    df['qid'] = df.apply(qid_map, axis=1)
    df['question_type'] = df['qid'].map(qtype_map)
# map the answer back to trainset.json, testset.json as inference results
# merge correct and incorrect
correct_df['correct'] = True
incorrect_df['correct'] = False
inference = pd.concat([correct_df, incorrect_df], axis=0)
# use question + image_name to map back to testset
correct_df.shape, incorrect_df.shape, inference.shape
# check if there is any overlap
correct_qids = set(correct_df['qid'].values)
incorrect_qids = set(incorrect_df['qid'].values)

# union 
union = correct_qids.union(incorrect_qids)
print(len(union))
# intersection
intersection = correct_qids.intersection(incorrect_qids)
print(len(intersection))

Screenshots (if appropriate):