Piotr Mardziel

Hi; is the issue coming from litellm? Can you check whether litellm without the use of trulens operates as expected on the models you want to run and whether litellm's...

Looks like these models don't like having messages with only role of "system" and will complain if no "user" role is given.

That is, the problem is with the prompt we are giving to evaluate some of these eval metrics. Will look more into it.

Does not work: ```python input_dict = {'messages': [{'content': 'You are a RELEVANCE grader; providing the relevance ' 'of the given CONTEXT to the given QUESTION.\n' 'Respond only as a number...

Having trouble replicating on main. @kouskouss can you let me know if the issue persists on the main branch of trulens_eval and the versions of pip packages and python you...

Problem may be related to https://github.com/truera/trulens/issues/639 . Looking for a fix.

Do all of the attributes that end in "score" have a single consistent and agreed upon definition of how such scores are computed?