FlagEmbedding
FlagEmbedding copied to clipboard
Provide a normalized algorithm for compute lexical similar score
Same sentences can always get a "1" simirlar score like dense way but not a score less than 1 and change with different sentence content.
Different sentences can get an more even similar score distribution.
Same sentences results:
Different sentences results:
Thanks for your contribution! This method may change the ranking list, so we need some time to conduct experiments to evaluate its performance.
Thanks for your contribution! This method may change the ranking list, so we need some time to conduct experiments to evaluate its performance.
This approach might be more explainable for applications compared to the original method, therefore it could perhaps be considered as an additional method, but not a replacement for the original one (depending on the results of your experiments).