Error when trigger Azure OpenAI’s content management policy

[axiangcoding]

Describe the bug

Azure OpenAI API will return in some cases

{"error":{"message":"The response was filtered due to the prompt triggering Azure OpenAI’s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766","type":null,"param":"prompt","code":"content_filter","status":400}}

it will failed when using like this:

response = openai.Completion.create(
    engine="text-davinci-003",
    prompt="something here",
    temperature=0.3,
    max_tokens=250,
    top_p=1,
    frequency_penalty=0,
    presence_penalty=0,
    best_of=1,
    stop=None)

To Reproduce

1. run the code snippets

Code snippets

response = openai.Completion.create(
    engine="text-davinci-003",
    prompt="SUBREDDIT: r/AskReddit TITLE: Cock blocked by a friend (Who's a girl). POST: So for the past week there's "
         "been this girl in one of my classes I've been talking to, she's pretty cute (dyed red hair, fair skin, "
         "a few freckles, not ginger), she loves star wars and I suspect she's a redditor. I was going to ask her for "
         "her number today, but a girl i met about a year ago came and sat right where the red head had been sitting, "
         "effectively cock-blocking me and driving the girl I was interested in away. Now it seems like the red head "
         "thinks I'm uninterested in her and has since found some other guy to talk to. Has anybody been in a similar "
         "scenario? Advice? \nTL;DR: Got cock blocked by a friend who's a girl.",
    temperature=0.3,
    max_tokens=250,
    top_p=1,
    frequency_penalty=0,
    presence_penalty=0,
    best_of=1,
    stop=None)

OS

Windows

Python version

Python v3.11

Library version

openai v0.27.2

[vincilee2]

I got the same problem. I fetch contents from my documents and ask who are you, then just got the same error as you.

[hallacy]

This seems to be working as intended. Is there some sort of error the python package is throwing?

[axiangcoding]

This seems to be working as intended. Is there some sort of error the python package is throwing?

take a look at https://github.com/openai/openai-python/issues/349, this issue describe it more specific

[StevenTang1998]

The same bug. Is there any solution?

[chuymtz]

I would like to know if there's a work around. For me it is triggered when passing a text which is contains words such as nipple which refers to a part of a particular mechanical tool. It's just called nipple its nothing bad. Is there a way to turn it off?

[solyarisoftware]

Hi, for me that's a TRUE BUG of Azure OpenAI’s content management policy.
When testing a chatbot (made with Azure OpenAI completions, e.g. a deployment using a GPT3.5-Turbo policy) in Italian language, I got an exception

              except openai.error.InvalidRequestError as e:
                  # Handle invalid request error, e.g. validate parameters or log
                  # WARNING: The response was filtered due to the prompt triggering Azure OpenAI’s content management policy.
                  sys.stderr.write(f"OpenAI API request was invalid: {e}")
                  return "Non ho capito bene. Cosa intendi?"

OpenAI API request was invalid: The response was filtered due to the prompt triggering Azure OpenAI’s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766

One naif sentence that trigger the exception is: “vorrei vedere la cosa più bella ad Ercolano. Qual’è?”

BTW, translated to English, means: “I would like to see the most beautiful thing in Herculaneum. What is it?”

So the sentence clearly doesn't violate any content policy categories. See also: https://community.openai.com/t/openai-s-content-management-policy-triggered-wrongly/300869/3

The workaround I implemented is that the chatbot catch the exception replying to user something like: "I do not understand, please rephrase":

:-(

---

Last but not least, paradoxically you can NOT set on Azure a custom policy to disable the content filtering and your are redirected to fullfil a request form to disable the content filtering. That's absurd IMMO.

[rattrayalex]

This is not a bug in the Python library. If you encounter content management policy decisions you find incorrect, please raise it with Microsoft.

[solyarisoftware]

Hi Alex @rattrayalex

I agree with you regarding the Azure filter management component responsibility and I just submitted a formal issue to Azure.
Nevertheless, reading documentation here: https://help.openai.com/en/articles/6897213-openai-library-error-types-guidance, I'm still a bit perplexed about the fact openAI python SDK raises an InvalidRequestError with the e message:

OpenAI API request was invalid: The response was filtered due to the prompt triggering Azure OpenAI’s content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766

I didn't inspected the code, but I think the specific error due to the Azure filter management component could reserve a specific error AzureContentManagaemntPolicy instead of InvalidRequestError.

What do you think?

Thanks Giorgio

[rattrayalex]

That's an interesting suggestion, would you mind opening a separate issue for it? I'm not sure we'll get to it anytime soon (it may be more involved than you'd think), but I certainly do see the merit.

[solyarisoftware]

Thanks Alex, I open a separate issue as you are suggesting.

[supermomo668]

Same issue still, any update as to how to overcome the issue on the user side?

[nadeems2024]

facing same issue - any update or work around ?

[gopikrishnachaganti]

one of the options would be to rephrase your question or content and then retry Azure openai, another would be customizing your Azure content filter which i dont suggest as it might have some some side effects.