AzureOpenAI content filtering gives unclear error

[erst-neaste]

trafficstars

Issue

Using dspy-ai==2.4.5, when the AzureOpenAI API's content filtering is triggered, DSPy returns a very unhelpful error.

AttributeError: 'NoneType' object has no attribute 'strip'

From within/site-packages/dsp/primitives/predict.py:78.

Expected behaviour

The AzureOpenAI LM class should throw a clear error stating that the content has been filtered.

[erst-neaste]

Are you open to pull requests here? Happy to make this change myself.

[arnavsinghvi11]

Thanks @erst-neaste, please do contribute PRs!

Could you also share snippets of related code and the full error stack trace? This seems to be an error in configuring your LM with your DSPy program.

[erst-neaste]

This code will replicate the issue:

gpt3 = dspy.AzureOpenAI(
    deployment_id=gpt3_model_name,
    api_key=gpt3_openai_api_key,
    api_base=gpt3_azure_endpoint,
    model_type="chat",
    api_version=openai_api_version,
    temperature=0.1,
    max_tokens=4096,
)

dspy.settings.configure(lm=gpt3)
signature = dspy.Predict('input -> output')
signature(input='Translate "period is over" to danish')

[erst-neaste]

Stack trace:

File "/usr/local/lib/python3.11/site-packages/dspy/predict/predict.py", line 49, in __call__
    return self.forward(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dspy/predict/predict.py", line 91, in forward
    x, C = dsp.generate(template, **config)(x, stage=self.stage)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp/primitives/predict.py", line 78, in do_generate
    completions: list[Example] = [template.extract(example, p) for p in completions]
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp/primitives/predict.py", line 78, in <listcomp>
    completions: list[Example] = [template.extract(example, p) for p in completions]
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp/templates/template_v2.py", line 152, in extract
    raw_pred = raw_pred.strip()
               ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'strip'