[BUG] Failed to retrieve the discovery service URL when running ViolenceEvaluator

[pamelafox]

Describe the bug

I am trying to run the ViolenceEvaluator sample on an AI project, with this code:

from promptflow.evals.evaluators import ViolenceEvaluator

project_scope = {
    "subscription_id": "<SUBSCRIPTION-ID>",
    "resource_group_name": "MaaSFieldRG",
    "project_name": "aisrinivasan-demo-project-23march",
}

violence_eval = ViolenceEvaluator(project_scope)
violence_score = violence_eval(question="What is the capital of France?", answer="Paris.")

However, it results in this error:

Failed to retrieve the discovery service URL

in promptflow/evals/evaluators/_content_safety/flow/evaluate_with_rai_service.py

How To Reproduce the bug

Run the above code, in a project in the internal azureaimaas tenant.

Running Information(please complete the following information):

• Promptflow Package Version using pf -v: [e.g. 0.0.102309906] { "promptflow": null, "promptflow-azure": "1.14.0", "promptflow-core": "1.14.0", "promptflow-devkit": "1.14.0", "promptflow-evals": "0.3.1", "promptflow-tracing": "1.14.0" }
• Operating System: GitHub Codespaces, python3:12 image
• Python Version using python --version: Python 3.12.5

[pamelafox]

cc @cedricvidal

[rfolkes]

I've just reproduced this running in our private Github repo / workflow.

We are trying to automate Prompt Flow based Evaluations, so very interested in this issue as well

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!

[pragmatical]

I am experiencing something similar to this. Attempting to run a content safety evaluation and seeing this error message.