promptflow icon indicating copy to clipboard operation
promptflow copied to clipboard

[Feature Request] pass addition inputs to target function when using the evaluate method

Open yanggaome opened this issue 1 year ago • 2 comments

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is.

Not sure if this is already supported, but I couldn't find one example

from promptflow.evals.evaluate import evaluate

def user_call(*, question: str, **kwargs):
    # question is a column from data
    # how can I pass in additional inputs to this target user function?

user_call_config = get_config()
evaluate(
data = input_data,
target = user_call # how can i pass in user_call_config variable into user_call?
)

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Currently I am using global variables so the user_call can access, but would like to understand how I can pass in variables that not from data to that user_call

Additional context Add any other context or screenshots about the feature request here.

yanggaome avatar Jul 09 '24 18:07 yanggaome

This is blocked by pf core/devkit.

luigiw avatar Jul 18 '24 20:07 luigiw

@luigiw , i talked to Ankit and was able to have a workaround by creating the target as a class instead of function, i can store the information in the class.

but another thing is, instead of passing available columns where we need to explicitly specify which columns to be used, can we just get the entire row of data? that will help us to generalize the api signature

yanggaome avatar Jul 18 '24 20:07 yanggaome