ragas icon indicating copy to clipboard operation
ragas copied to clipboard
verified publisher icon explodinggradients

[FIX] - Fix for summarization edge case

Open sky-2002 opened this issue 1 year ago • 1 comments

This PR adds a fix for the issue mentioned in #1108

However I have a points to discuss @shahules786 :

  • I had added conciseness_score to penalize long summaries, but I also do not want to promote very very short and skimpy summaries, need to find a middle ground.
  • Is averaging a good way to combine QA_score and conciseness_score?
  • Ranking based metrics to measure quality of summarization (as mentioned by shahul in the above issue)

Given the conclusions we reach based on these discussion points, I will push more commits, let's keep this PR open till we resolve these points.

sky-2002 avatar Aug 15 '24 13:08 sky-2002

Hey @sky-2002 thanks for the PR, my friend. I have answered your query in the here I think doing that will resolve the issue. Let me know your thoughts

When you make the change please make sure you modify and explain the same in docs too

shahules786 avatar Aug 16 '24 13:08 shahules786

@sky-2002 Sorry for the late merge. There were few important changes I had to make on this, please take a look at it so you know.

shahules786 avatar Aug 23 '24 12:08 shahules786

Hey @jjmachan Can you check what's with the lining errors on those files (not modified in this PR)

shahules786 avatar Aug 23 '24 12:08 shahules786