faithfullness checks for ragas assumes a match

[jjimenez]

https://github.com/andreibondarev/langchainrb/blob/86b7faf9c21f5402ad1a12ba02978c1716abb6a6/lib/langchain/evals/ragas/faithfulness.rb#L45

this line of code assumes that there is a match.

 def count_verified_statements(verifications)
          match = verifications.match(/Final verdict for each statement in order:\s*(.*)/)
          verdicts = match.captures.first
          verdicts
            .split(".")
            .count { |value| value.strip.to_boolean }
        end

consider changing to

 def count_verified_statements(verifications)
          match = verifications.match(/Final verdict for each statement in order:\s*(.*)/)
          verdicts = match&.captures&.first
          verdicts
            &.split(".")
            &.count { |value| value.strip.to_boolean } || 0
        end

[andreibondarev]

@jjimenez Could you please help me understand when this happens? Does the LLM hallucinate and not return the correct output? Could you please share what that output is?

[andreibondarev]

Closing due to inactivity. Feel free to reopen.

[jjimenez]

I apologize for dropping this. I really appreciate the tool, but got busy on other projects. If and when I return to this I will reopen or create a test scenario and provide additional details.