[BUG] Unable to make azure openai connection

[IyobedZekarias]

Describe the bug I've installed keyring, keyring.alt, and dbus-python in my conda environment in WSL and I am still getting Exception("Encryption key not found in keyring.").

This is when I'm running this command

> pf connection create -f openai_connection.yaml --set api_key=<api_key>

How To Reproduce the bug Steps to reproduce the behavior, how frequent can you experience the bug:

1. I cloned from this repository https://github.com/microsoft/promptflow.git
2. Installed requirements, and keyrings.alt
3. used my openai_connection.yaml file from a copy of the file in promptflow/examples/connections/azure_openai.yml folder and filled it in
4. ran pf connection create -f

Expected behavior Expected behavior is that it adds this to the promptflow connections

Running Information(please complete the following information):

• Promptflow Package: 1.9.0
• Operating System: WSL
• Python Version: 3.11.8

Additional context traceback:

Traceback (most recent call last):
  File "/home/iyobed/miniconda3/envs/promptflow/bin/pf", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/pf.py", line 14, in main
    _main()
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/_pf/entry.py", line 153, in main
    entry(command_args)
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/_pf/entry.py", line 134, in entry
    cli_exception_and_telemetry_handler(run_command, activity_name)(args)
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/_utils.py", line 384, in wrapper
    raise e
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/_utils.py", line 372, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/_pf/entry.py", line 85, in run_command
    raise ex
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/_pf/entry.py", line 61, in run_command
    dispatch_connection_commands(args)
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/_pf/_connection.py", line 256, in dispatch_connection_commands
    create_connection(args.file, args.params_override, args.name)
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_cli/_pf/_connection.py", line 200, in create_connection
    connection_ops.get(connection.name, raise_error=False)
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_sdk/_telemetry/activity.py", line 265, in wrapper
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_sdk/operations/_connection_operations.py", line 55, in get
    return self._get(name, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_sdk/operations/_connection_operations.py", line 65, in _get
    return _Connection._from_orm_object(orm_connection)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_sdk/entities/_connection.py", line 157, in _from_orm_object
    obj = type_cls._from_orm_object_with_secrets(orm_object)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_sdk/entities/_connection.py", line 231, in _from_orm_object_with_secrets
    obj.secrets = {k: decrypt_secret_value(obj.name, v) for k, v in obj.secrets.items()}
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_sdk/entities/_connection.py", line 231, in <dictcomp>
    obj.secrets = {k: decrypt_secret_value(obj.name, v) for k, v in obj.secrets.items()}
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/iyobed/miniconda3/envs/promptflow/lib/python3.11/site-packages/promptflow/_sdk/_utils.py", line 162, in decrypt_secret_value
    raise Exception("Encryption key not found in keyring.")
Exception: Encryption key not found in keyring.

[brynn-code]

Hi @IyobedZekarias , promptflow will create an encryption key to keyring when first use, the error raised seems because your local sqlite db has existing connections but encryption key not found in keyring, could you please try to delete the local db file (at ~/.promptflow/pf.sqlite) and have a next try?

[IyobedZekarias]

ok this ended up working but it creates a new pf.sqlite file whenever I create a new connection and I have to delete again

[brynn-code]

The pf.sqlite is required to store connections, you don't need to care about it once it works. The db needs to be deleted before is because seems there are already connections inside it, but the required encryption key used to decrypt connection does not exist. Once the db got reset, encryption key will be re-created, and all things should go well. @IyobedZekarias

[github-actions[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!

[Gijs-Koot]

I run into exactly the same issue. After deleting the pf.sqlite file, I can set the correct keys once, but a next action will trigger the following exception

I'm using WSL as well, but not conda.

  File "/home/gijsk/projects/dshub-openai-scripts/.venv/lib/python3.11/site-packages/promptflow/_sdk/entities/_connection.py", line 187, in _from_orm_object_with_secrets
    obj = type_cls._from_orm_object_with_secrets(orm_object)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gijsk/projects/dshub-openai-scripts/.venv/lib/python3.11/site-packages/promptflow/_sdk/entities/_connection.py", line 270, in _from_orm_object_with_secrets
    obj.secrets = {k: decrypt_secret_value(obj.name, v) for k, v in obj.secrets.items()}
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gijsk/projects/dshub-openai-scripts/.venv/lib/python3.11/site-packages/promptflow/_sdk/entities/_connection.py", line 270, in <dictcomp>
    obj.secrets = {k: decrypt_secret_value(obj.name, v) for k, v in obj.secrets.items()}
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gijsk/projects/dshub-openai-scripts/.venv/lib/python3.11/site-packages/promptflow/_sdk/_utilities/general_utils.py", line 164, in decrypt_secret_value
    raise Exception("Encryption key not found in keyring.")
Exception: Encryption key not found in keyring.

[Gijs-Koot]

I figured out a bit more about this issue. The problem is that keyring doesn't just work on WSL. See the section "Using Keyring on headless Linux systems" on https://pypi.org/project/keyring/. I didn't want to go into this solution involving dbus, and instead resorted to using a plaintext keyring implementation.

It would be nice if the package would support using a .env file as is common for other local development.

export PYTHON_KEYRING_BACKEND=keyrings.alt.file.PlaintextKeyring python

[brynn-code]

@Gijs-Koot Thanks for your investigation. Promptflow is calling the keyring package during execution but not aware of which kind of keyring backend was using, I'm afraid we don't have a proper time to load the .env, also we need to consider about the prioritization of .env files in different folders. Thanks again for your time.

[sjsivert]

I believe I am facing the same issue. I tried to create a custom connection in ml azure portal. The connection shows up in promptflow vscode.

When I try to run the flow I get this error

pf.flow.node_test failed with ResolveToolError: Tool load failed in 'search_relevant_examples': (GetConnectionError) Get connection 'qdrant_dev' for node 'search_relevant_examples' error: Connection 'qdrant_dev' not found in dict connection provider. Available keys are ['psopenaiaccountdev']

I have tried to delete the local slite db. It does not work.

Python: 3.9 promptflow: 1.15.0 promptflow-tools: 1.4.0 Mac m2 14.6.1 (23G93)