langchainrb icon indicating copy to clipboard operation
langchainrb copied to clipboard

Langchain::Evals::CosineSimilarity to calculate simple similarity between actual and expected outputs

Open andreibondarev opened this issue 1 year ago • 2 comments

andreibondarev avatar Oct 24 '24 01:10 andreibondarev

@bborn Okay, I re-packaged it a bit to a small, simple PR. I'm thinking of adding instructions to the README, something like:

# Execute the Assistant.
assistant.add_message_and_run! content: "What is 2+2?"
actual_output = assistant.messages.last.content

# Compare it's output to the expected output
Langchain::Evals::CosineSimilarity.new(llm).score(actual_output: actual_output, expected_output: "4")

( ^^^ Need a better example of course)

andreibondarev avatar Oct 24 '24 01:10 andreibondarev

@andreibondarev is this only missing the README update?

sergiobayona avatar Apr 14 '25 20:04 sergiobayona