Add Vectara Hallucination Detection Model
Added vectara hallucination detection model to the huggingface class
adding an exposition/model example using huggingface models end to end for demo
this is a draft PR , just need to add some text to explain the examples we have chosen
| :rocket: This PR description was created by Ellipsis for commit 4f3353e23a27528c1075b182753d15da0667ce32. |
|---|
Summary:
The PR adds a new method hallucination_evaluator to the Huggingface class in hugs.py for evaluating the hallucination score of a combined input of two statements using the Huggingface hallucination evaluation model.
Key points:
- Added
hallucination_evaluatormethod toHuggingfaceclass inhugs.py. - The method uses the
HUGS_HALLUCINATION_API_URLendpoint for the hallucination evaluation model from Huggingface. - The method takes two arguments:
model_outputandretrieved_text_chunks, combines them, and sends a POST request to the API. - The response is parsed to extract the hallucination score.
Generated with :heart: by ellipsis.dev
Check out this pull request on ![]()
See visual diffs & provide feedback on Jupyter Notebooks.
Powered by ReviewNB
Can you remove the .ipynb checkpoints from this change?
Also can you elaborate on why there are two separate notebooks to show this capability?
Thanks for all the work on this @MN-Noor @Josephrp - this is close! Just added a few small comments before it can go in
hi @MN-Noor would you like to take the last reviews above ?
@joshreini1 , thanks for the comments, should we keep both notebooks renamed or remove one or the other?
sorry to bother you but i'll be able to wrap this up :-)
@Josephrp keep the one in /models and remove the other. Thanks!
@joshreini1 thanks ! @MN-Noor wrapped it up nicely , hope that's us done - until next time !
@Josephrp @MN-Noor do you have a Twitter/x account? I’ll shoutout from the TruLens handle if so!
ha! i sure do , here's me . Noor's doesnt work well where she is , so that's why she prefers linked-in
hope that's okay :-)