Felipe Adachi

+1

Each row in the picture represents a Langkit metric, whose documentation can be found here: https://github.com/whylabs/langkit/blob/main/langkit/docs/modules.md Each column in the picture represents a whylogs metric. The following example discusses these...

It means that, within the data you profiled (with the size of 50, as shown by `counts/n`), there are approximately 17 different unique values for `prompt.aggragate_reading_level`. This is not the...

Then you can use langkit directly, like this: ```python from langkit import llm_metrics, extract import pandas as pd df = pd.DataFrame({"prompt":["Hi! how are you?"],"response":["I'm ok, thanks for asking!"]}) enhanced_df =...

Can you elaborate on your question a bit more? What is exactly that you're trying to do?

The hallucination detection will call the LLM in two distinct phases: 1) To generate the additional samples, based on the prompt passed 2) To perform the consistency check between the...

Hi @Starignus , Thank you for reporting this! We are working on this and will let you know how this advances in this thread.

Hi, @rajib76 Yes, this is exactly how `response_hallucination` works. To try to answer your questions: 1. This module was designed with the zero-resource scenario in mind - without ground truth...

Thanks for the reply @rajib76 For #1, yes, it should be possible to perform the semantic similarity based consistency check without the presence of an LLM. And #2 also makes...