[BUG]calling pf_client.run inside the callable target function for evaluate got "Error: (AssertionError) daemonic processes are not allowed to have children."

[yanggaome]

Describe the bug A clear and concise description of the bug.

How To Reproduce the bug Steps to reproduce the behavior, how frequent can you experience the bug: I am trying to hook up the evaluate method with my own prompt which used to run from pf_client.run

example code below:

from promptflow.client import PFClient

pf_client = PFClient()

def user_call():
    my_prompt_flow = pf_client.run(my prompt flow configs)
    return my_prompt_flow_output

def run_evaluation():
    results = evaluate(
                 evaluation_name=evaluation_name,
                 data=data_path,
                 target=user_call,
                 evaluators={
                    "violence": violence_evaluator,
                    "sexual": sex_evaluator,
                    "self_harm": self_harm_evaluator,
                    "hate_unfairnes": hate_unfairness_evaluator,
                    "content_safety": content_safety_evaluator
                 },
                azure_ai_project=project_scope
             )

2024-07-06 00:58:41 +0000 1845174 execution.bulk INFO The process [1845174] has received a terminate signal. 2024-07-06 00:58:42 +0000 1844849 execution ERROR 1/1 flow run failed, indexes: [0], exception of index 0: Execution failure in 'user_call': (UnexpectedError) Unexpected error occurred while executing the batch run. Error: (AssertionError) daemonic processes are not allowed to have children.

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Running Information(please complete the following information):

• Promptflow Package Version using pf -v: [e.g. 0.0.102309906]
• Operating System: [e.g. Ubuntu 20.04, Windows 11]
• Python Version using python --version: [e.g. python==3.10.12] { "promptflow": "1.13.0", "promptflow-azure": "1.13.0", "promptflow-core": "1.13.0", "promptflow-devkit": "1.13.0", "promptflow-evals": "0.3.0", "promptflow-tracing": "1.13.0" } Additional context Add any other context about the problem here.

[D-W-]

Hi @yanggaome , evaluate will call pf.run implicitly, which will make your above code call a pf.run inside a pf.run. While pf.run used multi-processing to process each line of data, which might cause above error. We'll use this issue to track root cause of above error. To work around, could you change your user_call to use flow-as-a-function instead of pf.run? Here's sample code

def user_call():
    my_prompt_flow = load_flow("path/to/flow")
    my_prompt_flow(xxx)
    return my_prompt_flow_output

[yanggaome]

thanks @D-W- , that works!

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!