Azure Content Filtering on Default QA with sources prompt.

[JonAtDocuWare]

System Info

v0.0.176

Who can help?

@hwchase17 @agola11

Information

• [ ] The official example notebooks/scripts
• [X] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [X] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

The default QA with Sources Chain: langchain/chains/qa_with_sources/stuff_prompt.py

causes issues with Azure Content Filter, which will return the following:

"error": {
    "message": "The response was filtered",
    "type": null,
    "param": "prompt",
    "code": "content_filter",
    "status": 400
}

I believe this is due to the sections that mention issues like covid, which is found in the example source content in lines 22-28.

I modified to the following to resolve the issue:

Content: Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.
Source: 0-pl
Content: And we won’t stop. 
Source: 24-pl
Content: And a proud Ukrainian people, I want you to know that we are going to be okay.
Source: 5-pl
Content: More support for patients and families. Our future is forged. \n\nWell I know this nation.
Source: 34-pl

this is probably not the best version of the source content, but it worked for me.

Expected behavior

The QA prompt should not trigger Azure Content Filtering.