langfuse-docs icon indicating copy to clipboard operation
langfuse-docs copied to clipboard

Updated Cookbook: Example for Fetching Scores from Langfuse

Open Sohammhatre10 opened this issue 1 year ago • 5 comments

Description

This update provides an example of using the fetch_scores() function from Langfuse to retrieve evaluation metrics. The example integrates UpTrain and Ragas for model evaluation and demonstrates how to log and fetch scores within Langfuse as mentioned in https://github.com/langfuse/langfuse/issues/3505

Key Features

  1. Evaluation with UpTrain and Ragas:

    • Provides examples for evaluating context relevance, factual accuracy, response completeness, context precision, faithfulness, and answer relevancy..
  2. Fetching Scores:

    • Shows how to retrieve and filter scores using fetch_scores_from_langfuse.
  3. Correlation Matrix Visualization:

    • Adds a section that calculates and visualizes the correlation between UpTrain and Ragas evaluation scores using a heatmap.

[!IMPORTANT] Adds an example for using Langfuse to fetch scores, evaluate models with UpTrain and Ragas, and visualize results using a correlation matrix.

  • Behavior:
    • Adds example for using fetch_scores() from Langfuse to retrieve evaluation metrics.
    • Demonstrates integration with UpTrain and Ragas for model evaluation.
    • Shows how to log and fetch scores within Langfuse.
  • Visualization:
    • Includes a section for calculating and visualizing correlation between evaluation scores using a heatmap.
  • Misc:
    • Minor whitespace changes in dspy.md, instructor.md, example-javascript.md, example-python-langgraph.md, example-python-instrumentation-module.md, example-python.md, example-vercel-ai.md, example_external_evaluation_pipelines.md, integration_dspy.md, integration_instructor.md, integration_langgraph.md, integration_llama-index_instrumentation.md, integration_llama_index_posthog_mistral.md, integration_mirascope.md, integration_mistral_sdk.md, integration_ollama.md, integration_openai_structured_output.md, example-langchain.md, js_integration_langchain.md, js_tracing_example_vercel_ai_sdk.md, prompt_management_langchain.md.

This description was created by Ellipsis for 02ebc242f05aa872fb1742d8b1514c5dd217a0aa. It will automatically update as commits are pushed.

Sohammhatre10 avatar Oct 14 '24 08:10 Sohammhatre10

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@Sohammhatre10 is attempting to deploy a commit to the langfuse Team on Vercel.

A member of the Team first needs to authorize it.

vercel[bot] avatar Oct 14 '24 08:10 vercel[bot]

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

:x: Your Name
:x: Sohammhatre10


Your Name seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar Oct 14 '24 08:10 CLAassistant

@marcklingen Yupp, this was based on a single trace, and the scores were fetched accordingly. Haven't used any specifics for traces, but this was the first trace I created, so it defaulted to the first trace. Should I add more specificity for a single trace? Apologies for the late reply.

Sohammhatre10 avatar Nov 19 '24 05:11 Sohammhatre10

@jannikmaierhoefer can you review this?

marcklingen avatar Feb 27 '25 20:02 marcklingen