vert-papers The result of decoding BPE

Hello! Could you help me understand the following output? I passed these tags as query label to DecomposedMetaNER: ['action', 'action', 'O', 'entity', 'O', 'O', 'O', 'action', 'O', 'property', 'entity', 'O', 'O', 'property', 'entity', 'O', 'O', 'property', 'O', 'entity', 'O', 'O'] And after applying convert_bpe I have word indexes: [[0, 1], [3, 4], [7, 8], [9, 9], [10, 11], [13, 13], [14, 15], [17, 17], [19, 20]] What is the logic of pairs? Why can I get [i, i] or [i, i+1] for some single words?

Mar 23 '23 13:03 temav

The end goal was to get model prediction in conll or jsonl format

Mar 23 '23 13:03 temav

It seems you have a bug at L823 You should replace bisect.left with bisect.right, am I right?

Mar 23 '23 15:03 temav

It seems you have a bug at L823 You should replace bisect.left with bisect.right, am I right?

Hi @temav, I think you are right. However, the function convert_bpe has been deprecated. We did not use this function in our experiments. And we will fix this issue in the next PR, thank you!

Apr 18 '23 06:04 iofu728

I see, thanks!

Apr 27 '23 15:04 temav