guardrails icon indicating copy to clipboard operation
guardrails copied to clipboard

Added cosine similarities to metadata for ProvenanceV0

Open yadomkar opened this issue 11 months ago • 3 comments

This PR addresses issue #575, enhancing the validate_each_sentence and validate_full_text methods in the ProvenanceV0 validator to include cosine similarity values in the validation metadata, providing insights into how closely each sentence matches the source text

yadomkar avatar Mar 04 '24 19:03 yadomkar

Codecov Report

Attention: Patch coverage is 0% with 8 lines in your changes are missing coverage. Please review.

:exclamation: No coverage uploaded for pull request base (main@ce61fe6). Click here to learn what that means.

Files Patch % Lines
guardrails/validators/provenance.py 0.00% 8 Missing :warning:
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #610   +/-   ##
=======================================
  Coverage        ?   81.80%           
=======================================
  Files           ?      118           
  Lines           ?     6764           
  Branches        ?        0           
=======================================
  Hits            ?     5533           
  Misses          ?     1231           
  Partials        ?        0           
Flag Coverage Δ
unittests 81.80% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Mar 05 '24 05:03 codecov-commenter

Thanks for creating the PR @yadomkar! Can you update the tests as well? You'll need to modify this test

ShreyaR avatar Mar 05 '24 06:03 ShreyaR

@ShreyaR Thank you for the feedback! Currently, there doesn't seem to be existing test coverage for ProvenanceV0 that I could directly extend to include checks for the cosine similarity metadata. My changes were specifically for ProvenanceV0, and I haven't touched ProvenanceV1 as it uses a different method. Should I proceed with adding coverage for ProvenanceV0, or is there an existing suite I might have overlooked?

yadomkar avatar Mar 06 '24 05:03 yadomkar

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

github-actions[bot] avatar Aug 22 '24 03:08 github-actions[bot]