Answer Semantic Similarity gives output not in range 0-1

[AnnamariaGalante]

Describe the bug The output of the AnswerSimilarity metric should be in range 0-1, but gives also negative output, and not in range 0-1. This affects also answer correctness metric. If i do not put the param "threshold" to None, the output results binary True/False. Ragas version: 0.0.22 Python version:3.10.6

Code to Reproduce answer_similarity=AnswerSimilarity(llm=ragas_bedrock_model,embeddings= bedrock_embeddings, threshold=None) answer_similarity.score(ds) output: 35.83004634

Expected behavior The score should be between 0 and 1, not specifiyng thresholds values.

[almajo]

Are your embeddings normalized? This might be one reason

[AnnamariaGalante]

I'm passing to the api the embedding model, not embeddings theirself, so the library should do this.

[shahules786]

Hi @AnnamariaGalante , thanks for reporting this issue. Does this occur in every sample or is it once in a while issue?

[AnnamariaGalante]

Hi @shahules786, It occurs in every sample, using bedrock embeddings. I think this affects answer correctness too.

[shahules786]

that's interesting, Would you like to help us find out the issue here with bedrock? I think this is due to the embedding not being normalised @AnnamariaGalante

[AnnamariaGalante]

@shahules786 how can i help you to fix this issue?