alexklibisz
Added dot product everywhere were cosine similarity was used
Related Issue
Support for inner product similarity measures
- #265
Changes
almost everywhere (except some tests) where there was a cosine reference a parallel dot function/option is added
What changed?
most files where cosine was used
Testing and Validation
Not done , needs to be done, it is pending. I'll try to generate and replace the plugin in my ES isntallation and check it. but not sure which prodedures I must follow.
Hi @alexklibisz First thanks for fast response, and the plug-in itself!! let me apologize , for sending the pull request before deeply testing it. It's my first time doing things in Scala, and I'm a bit confused on how to do some things. Finally i could compile and build the zip in my side, so I can try to run the plugin! (and check if it works and the performance) But as I said my knowledge of scala is my knowledge of Java. ... and for some of the things you ask me, i'm not able to do them. (basically testing is where I think it will take a me a while to understand everything!)
About the changes you asked. I did all of them . One I did in a different way was to ensure that the similarity was returning a positive value with max(0,1+dotProduct) In this way we don't raise an exception and negative values will have a 0 similartiy
Thanks Joan
Hi @joancf can you try adding the exceptions for scores outside [0, 2]? If you're having trouble I can try this, but probably not til the weekend or next week.
hi. @alexklibisz my company doesn't want to use it. So, I will finish it out-hours I'll do my best with exceptions and testing.
@alexklibisz Now I'm back with this code, I wanted to extract some logs, let me see If I can finish it
Hi, @alexklibisz , I added some tests (i normalized the vectors for dot product) and also I rebased the branch to main I can't compile it, because there is an error in [E008] Not Found Error: elastiknn-plugin/src/main/scala/com/klibisz/elastiknn/mapper/VectorMapper.scala:119:83 [error] 119 | new VectorMapper.FieldType(CONTENT_TYPE, context.buildFullName(thisBuilder.leafName), mapping),
maybe this a problem of the version of Elasticsearch I'm used ( I'm using 12.2 instead of 13) but I need to work on that version and I don't want to upgrade my environment.
Hi, @alexklibisz , I added some tests (i normalized the vectors for dot product) and also I rebased the branch to main I can't compile it, because there is an error in [E008] Not Found Error: elastiknn-plugin/src/main/scala/com/klibisz/elastiknn/mapper/VectorMapper.scala:119:83 [error] 119 | new VectorMapper.FieldType(CONTENT_TYPE, context.buildFullName(thisBuilder.leafName), mapping),
maybe this a problem of the version of Elasticsearch I'm used ( I'm using 12.2 instead of 13) but I need to work on that version and I don't want to upgrade my environment.
In order to get this merged, it needs to be working with the latest version. If that doesn't work, you're welcome to fork and build your own artifacts for older versions. I'm not maintaining multiple/older versions of Elasticsearch.
ok, Alex I synchronized my repo, Now I can compile and generate the plugin 8.15.0.0