[Feature Request] Allow the target function in evaluate to be a local function

[pamelafox]

I currently get this error when passing a local function to evaluate:

promptflow.exceptions.UserErrorException: Failed to import <function run_evaluation..wrap_target at 0x318bc7ba0> from module <module 'scripts.evaluate' from '/Users/pamelafox/ai-rag-chat-evaluator/scripts/evaluate.py'>, please make sure it's a global function.

That is due to the usage of multiprocessing and its need to serialize the function.

Perhaps you could use asyncio.taskgroup or a different mechanism such that local functions are allowed. That would permit me to write much cleaner and more portable, testable code.

[D-W-]

Hi @pamelafox , thanks for the suggestion. We've already tracking this internally. The reason we did not support this in the first place was to maintain parity between local and cloud (it's hard to run a local function in cloud). However, we will prioritize support for local scenarios first.

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!

[pamelafox]

I don't believe any more input is needed from me. (The stalebot in this repo seems to be tuned to respond really quickly to perceived stalneness, btw).

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!