word-embeddings-benchmarks icon indicating copy to clipboard operation
word-embeddings-benchmarks copied to clipboard

Different evaluation results for the same embeddings.

Open flyaway1217 opened this issue 6 years ago • 0 comments

Hi, there This is a very useful tool that can help us evaluate word embeddings in a consistent way. However, during the evaluation process, I found a bug.

I have two embedding files: file1.txt and file2.txt. These two files contains exactly the same word embeddings, the only difference is they have different order of words. When we evaluate these two, they should have exactly the same results. However, this evaluation tool produces completely different scores for these files. I think there must be something wrong here.

flyaway1217 avatar Dec 04 '18 03:12 flyaway1217