[BUG] Unable to get context_relevance_with_cot_reasons and groundedness_measure_with_cot_reasons when using Azure OpenAI

[harsh-garg98]

Bug Description I am following the Azure OpenAI LangChain Quickstart google colab notebook (https://colab.research.google.com/github/truera/trulens/blob/main/trulens_eval/examples/expositional/models/azure_openai_langchain.ipynb) to set up trulens using Azure OpenAI. I am the 'Initialize Feedback Function(s)' stage and I am getting an error:

AttributeError: 'AzureOpenAI' object has no attribute 'context_relevance_with_cot_reasons'

According to the documentation the AzureOpenAI class derives from the OpenAI class which derives from the LLMProvider base class. The LLMProvider base class documentation page shows that it has the 'context_relevance_with_cot_reasons' and the 'groundedness_measure_with_cot_reasons' methods.

To Reproduce I am following the notebook and have not changed any bit of code.

Expected behavior The expected behaviour is to get the context relevance and groundedness scores.

Relevant Logs/Tracebacks Code:

from trulens_eval.feedback.provider import AzureOpenAI
import numpy as np
from trulens_eval.app import App


# Initialize AzureOpenAI-based feedback function collection class:
provider = AzureOpenAI(
    # Replace this with your azure deployment name
    deployment_name=os.getenv("DEPLOYMENT_NAME")
)

# select context to be used in feedback. the location of context is app specific.
context = App.select_context(rag_chain)

# Question/answer relevance between overall question and answer.
f_qa_relevance = Feedback(provider.relevance, name="Answer Relevance").on_input_output()

# Question/statement relevance between question and each context chunk.
f_context_relevance = (
    Feedback(provider.context_relevance_with_cot_reasons, name="Context Relevance")
    .on_input()
    .on(context)
    .aggregate(np.mean)
)

# groundedness of output on the context
f_groundedness = (
    Feedback(provider.groundedness_measure_with_cot_reasons, name="Groundedness")
    .on(context.collect())
    .on_output()
)

Error:

AttributeError                            Traceback (most recent call last)
Cell In[88], [line 20](vscode-notebook-cell:?execution_count=88&line=20)
     [16](vscode-notebook-cell:?execution_count=88&line=16) f_qa_relevance = Feedback(provider.relevance, name="Answer Relevance").on_input_output()
     [18](vscode-notebook-cell:?execution_count=88&line=18) # Question/statement relevance between question and each context chunk.
     [19](vscode-notebook-cell:?execution_count=88&line=19) f_context_relevance = (
---> [20](vscode-notebook-cell:?execution_count=88&line=20)     Feedback(provider.context_relevance_with_cot_reasons, name="Context Relevance")
     [21](vscode-notebook-cell:?execution_count=88&line=21)     .on_input()
     [22](vscode-notebook-cell:?execution_count=88&line=22)     .on(context)
     [23](vscode-notebook-cell:?execution_count=88&line=23)     .aggregate(np.mean)
     [24](vscode-notebook-cell:?execution_count=88&line=24) )
     [26](vscode-notebook-cell:?execution_count=88&line=26) # groundedness of output on the context
     [27](vscode-notebook-cell:?execution_count=88&line=27) f_groundedness = (
     [28](vscode-notebook-cell:?execution_count=88&line=28)     Feedback(provider.groundedness_measure_with_cot_reasons, name="Groundedness")
     [29](vscode-notebook-cell:?execution_count=88&line=29)     .on(context.collect())
     [30](vscode-notebook-cell:?execution_count=88&line=30)     .on_output()
     [31](vscode-notebook-cell:?execution_count=88&line=31) )

File ~/Desktop/litellm/langfuse-self-host/.venv/lib/python3.9/site-packages/pydantic/main.py:811, in BaseModel.__getattr__(self, item)
    [808](https://file+.vscode-resource.vscode-cdn.net/Users/harshgarg/Desktop/litellm/langfuse-self-host/~/Desktop/litellm/langfuse-self-host/.venv/lib/python3.9/site-packages/pydantic/main.py:808)     return super().__getattribute__(item)  # Raises AttributeError if appropriate
    [809](https://file+.vscode-resource.vscode-cdn.net/Users/harshgarg/Desktop/litellm/langfuse-self-host/~/Desktop/litellm/langfuse-self-host/.venv/lib/python3.9/site-packages/pydantic/main.py:809) else:
    [810](https://file+.vscode-resource.vscode-cdn.net/Users/harshgarg/Desktop/litellm/langfuse-self-host/~/Desktop/litellm/langfuse-self-host/.venv/lib/python3.9/site-packages/pydantic/main.py:810)     # this is the current error
--> [811](https://file+.vscode-resource.vscode-cdn.net/Users/harshgarg/Desktop/litellm/langfuse-self-host/~/Desktop/litellm/langfuse-self-host/.venv/lib/python3.9/site-packages/pydantic/main.py:811)     raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')

AttributeError: 'AzureOpenAI' object has no attribute 'context_relevance_with_cot_reasons'

Environment:

• OS: MacOS Sonoma Version 14.0
• Python Version - 3.9.6
• TruLens version - trulens_eval==0.25.1
• Versions of other relevant installed libraries: llama-index==0.10.17 langchain==0.1.11 chromadb==0.4.24 langchainhub==0.1.20 bs4==0.0.2 langchain-openai==0.0.8 ipytree==0.2.2