RediSearch icon indicating copy to clipboard operation
RediSearch copied to clipboard

how to search text like jaccard similarity

Open XufengXufengXufeng opened this issue 2 years ago • 4 comments

when I search "i hate banana", i would like to get the text "i love banana" too. I can't achieve it with any of the scoring methods, am doing it wrong? How do i achieve it?

XufengXufengXufeng avatar Feb 06 '23 03:02 XufengXufengXufeng

@XufengXufengXufeng You can add a synonym group using FT.SYNUPDATE to make love and hate equivalent. We currently support Vector Similarity Search which is suitable for binary similarity (images, text embedding, other binary data) and not for NLP.

oshadmi avatar Feb 06 '23 09:02 oshadmi

@XufengXufengXufeng You can add a synonym group using FT.SYNUPDATE to make love and hate equivalent. We currently support Vector Similarity Search which is suitable for binary similarity (images, text embedding, other binary data) and not for NLP.

thank you for the reply. I am not looking for the synonym feature actually. The phrases "I hate apple" and "I like apple" are very similar. I can achieve the search result with a bm25 python library and with the ElasticSearch tool, but not with redisearch.

by the way, the search results doesn't have a score field which is very useful.

XufengXufengXufeng avatar Feb 09 '23 10:02 XufengXufengXufeng

To add to this, I think what would be useful is a way to get the BM25 score for each document and query (and optionally specify a radius like in Vector Search). What appears to be happening is a more standard SQL-like filtering is taking place (per the docs) and then any documents that pass the filter are sorted by the scorer (e.g. BM25). It would be handy to have the ability to treat the scorers more like VSS queries and not do a hard-filter on the text itself.

Please let me know if I am missing something, as I am pretty new to learning about the Redis Search capabilities.

steve-marmalade avatar Aug 17 '23 20:08 steve-marmalade

This issue is stale because it has been open for 60 days with no activity.

github-actions[bot] avatar Nov 26 '23 01:11 github-actions[bot]