[Feature request] Can’t pass headers in AzureOpenAI connections

[rishabhsaha]

Describe the bug I can’t seem to be able to pass headers while creating AzureOpenAI connections. Headers are needed in cases of Azure OpenAI instances placed behind Azure API Management.

How To Reproduce the bug Steps to reproduce the behavior, how frequent can you experience the bug:

1. Try passing in an header which you can do while creating the OpenAI client

Expected behavior We should be able to pass in headers in the connection settings as well. OpenAI 1.x library requires you pass it when the client is being initialized. Example as below;

# Setting the auth headers
auth_headers = {
    ‘Ocp-Apim-Subscription-Key’: my_apim_sub_key,
    ‘Authorization’: f’Bearer {my_jwt_token}’
}

# Setting the Azure OpenAI client
azure_openai_client = AzureOpenAI(
   api_key = ‘null’,
   api_version = ‘2023-07-01-preview’,
   azure_endpoint = my_special_url,
   default_headers = auth_headers
)

# Calling Azure OpenAI
openai_response = azure_openai_client.chat.completions.create(
    model = ‘gpt-35-turbo-16k’,
    messages = my_messages
)

Running Information(please complete the following information):

• Promptflow Package Version using pf -v: [e.g. 0.0.102309906]
• Operating System: [e.g. Ubuntu 20.04, Windows 11]
• Python Version using python --version: [e.g. python==3.10.12]

Additional context Add any other context about the problem here.

[cloga]

Hi @rishabhsaha, this is known gaps, AOAI connection to better support OpenAI1.0 is still under design. @luigiw for help

[mindlessroman]

Keeping an eye on this issue since I'm seeing similar struggles on a project I'm working on. I've tried to bypass around by using a custom connection, but I think my configuration setup is wrong. If there's any advice on that end, I'm open ears!

[rishabhsaha]

is your custom connection set up working? would you be able to share what you did on this thread? I have been unable to make it work.

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!

[simon-v-swyftx]

Do you two have custom config to send headers to a flow input, let alone a connection?

[lshade]

Did anyone figure out a workaround?

[yifanniemanu]

Hi Promptflow team, or any promptflow users, did any one figure out a solution on this? Thanks.