FewRel: meta_labels[-1].item() is always 4

[miaoski]

Hi. Thank you for the implementation. It is great to learn from your code. I have observed a weird thing wrt FewRel. The answer in the training set is always 4, as I added in evaluate(),

                if meta_labels[-1].item() != 4:
                    print(closest_idx, 'vs', meta_labels[-1].item())

Nothing was printed. The predicted result closest_idx varies, so there is no problem with the model. The implementation looks correct as well. It might be caused by the implementation of meta_labels, for example,

for sample_idx, r in enumerate(sampled_relation):
...
        meta_input = meta_train_input + [torch.LongTensor(meta_test_input)]
        meta_labels = meta_train_labels + [meta_test_labels]
        print('meta_labels =', meta_labels)   # Added this line

Also, meta_labels[-1] was obtained from,

        target_idx = self.N - 1
...
        meta_test_labels = [target_idx]
...
        meta_labels = meta_train_labels + [meta_test_labels]

It is fixed to self.N-1, thus 4 in 5-way-1-shot.

meta_labels = [[0], [1], [2], [3], [4], [4]]
4 vs 4
meta_labels = [[0], [1], [2], [3], [4], [4]]
1 vs 4
meta_labels = [[0], [1], [2], [3], [4], [4]]
4 vs 4
meta_labels = [[0], [1], [2], [3], [4], [4]]
1 vs 4
meta_labels = [[0], [1], [2], [3], [4], [4]]
4 vs 4

[miaoski]

Ah. I kind of understood it. Brilliant implementation!

sampled_relation = [49, 37, 25, 9]
sampled_relation = [62, 30, 59, 2]
sampled_relation = [2, 45, 27, 28]
sampled_relation = [19, 50, 58, 25]

[plkmo]

Thanks :)