Self-hosted API Key Issues with both model providers

[benweissbehavehealth]

Hello @everyone I am self-hosting and managed to get everything going except for the API.

I cannot save the Azure OpenAI information even though i set it up correctly following the instructions here: https://github.com/langgenius/dify/issues/109

The save button does not work, it does not register the click to save.

Also the OpenAi api key will not save and is giving me an error. Can someone please help? 🙂

Dify version: Self Host

Steps To Reproduce

1. go to settings
2. try to add your credentials with Azure or OpenAI and neither works

The current behavior

• azure does not save even though green checkmarks
• openai gives an error

The expected behavior

• expect behavior is for it to work

[takatost]

Can you take a look at the logs of the api container? docker logs -n 200 docker-api-1 btw, you can also try to pull the latest image docker-compose pull and redeploy it. We've optimized the model provider's validation prompt, which might help.

[roberteles]

Hi @takatost – we are running the latest image.

The documentation on Azure is not clear on what ENVs we should define https://d.pr/i/Z7oGUM. I found a few (OPENAI_API_TYPE, OPENAI_API_VERSION, OPENAI_API_BASE, OPENAI_API_KEY) in https://github.com/langgenius/dify/blob/7a16c880929a18c0b6401335f5726112d4c0220c/api/core/llm/llm_builder.py, tried setting them up but it's not working.

When setting Azure up from the UI (https://d.pr/i/FaReaY), we get the green checks but the Save button doesn't do anything.

[takatost]

No need to set the env vars in normal use. You can pull the latest image by execute: docker-compose pull, if it still doesn't work, can you post the api service log here by execute docker logs -n 200 docker-api-1.

[roberteles]

Here are the logs:

Running migrations
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
[2023-05-25 11:13:06 +0000] [14] [INFO] Starting gunicorn 20.1.0
[2023-05-25 11:13:06 +0000] [14] [INFO] Listening at: http://0.0.0.0:5001 (14)
[2023-05-25 11:13:06 +0000] [14] [INFO] Using worker: gevent
[2023-05-25 11:13:06 +0000] [21] [INFO] Booting worker with pid: 21
INFO:openai:error_code=404 error_message='Resource not found' error_param=None error_type=None message='OpenAI API error received' stream_error=False
INFO:root:Account b68b9a69-b24f-43ec-b34a-949e472b479c logged in successfully.
INFO:root:Account b68b9a69-b24f-43ec-b34a-949e472b479c logged in successfully.

[roberteles]

Hi @takatost. Here are some updated logs:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/api/services/completion_service.py", line 197, in generate_worker
    Completion.generate(
  File "/app/api/core/completion.py", line 34, in generate
    cls.validate_query_tokens(app.tenant_id, app_model_config, query)
  File "/app/api/core/completion.py", line 289, in validate_query_tokens
    llm = LLMBuilder.to_llm_from_model(
  File "/app/api/core/llm/llm_builder.py", line 76, in to_llm_from_model
    return cls.to_llm(
  File "/app/api/core/llm/llm_builder.py", line 55, in to_llm
    model_credentials = cls.get_model_credentials(tenant_id, provider, model_name)
  File "/app/api/core/llm/llm_builder.py", line 115, in get_model_credentials
    return provider_service.get_credentials(model_name)
  File "/app/api/core/llm/provider/llm_provider_service.py", line 32, in get_credentials
    return self.provider.get_credentials(model_id)
  File "/app/api/core/llm/provider/azure_provider.py", line 43, in get_credentials
    config = self.get_provider_api_key(model_id=model_id)
  File "/app/api/core/llm/provider/base.py", line 39, in get_provider_api_key
    return self.get_decrypted_token(provider.encrypted_config)
  File "/app/api/core/llm/provider/azure_provider.py", line 139, in get_decrypted_token
    config['openai_api_key'] = self.decrypt_token(config['openai_api_key'])
  File "/app/api/core/llm/provider/base.py", line 120, in decrypt_token
    return rsa.decrypt(base64.b64decode(token), self.tenant_id)
  File "/app/api/libs/rsa.py", line 47, in decrypt
    raise PrivkeyNotFoundError("Private key not found, tenant_id: {tenant_id}".format(tenant_id=tenant_id))
libs.rsa.PrivkeyNotFoundError: Private key not found, tenant_id: 92af6604-297d-453e-95c4-96b9c37855d3
ERROR:root:internal server error.
Traceback (most recent call last):
  File "/app/api/controllers/console/app/completion.py", line 158, in generate
    for chunk in response:
  File "/app/api/services/completion_service.py", line 404, in generate
    cls.handle_error(result)
  File "/app/api/services/completion_service.py", line 506, in handle_error
    raise Exception(description)
Exception: Private key not found, tenant_id: 92af6604-297d-453e-95c4-96b9c37855d3
ERROR:app:Exception on /console/api/apps/78f25c0b-018d-4fae-a9e3-7070c6fb6793/chat-messages [POST]
Traceback (most recent call last):
  File "/app/api/controllers/console/app/completion.py", line 158, in generate
    for chunk in response:
  File "/app/api/services/completion_service.py", line 404, in generate
    cls.handle_error(result)
  File "/app/api/services/completion_service.py", line 506, in handle_error
    raise Exception(description)
Exception: Private key not found, tenant_id: 92af6604-297d-453e-95c4-96b9c37855d3
ERROR:root:Unknown Error in completion
Traceback (most recent call last):
  File "/app/api/libs/rsa.py", line 45, in decrypt
    private_key = storage.load(filepath)
  File "/app/api/extensions/ext_storage.py", line 65, in load
    raise FileNotFoundError("File not found")
FileNotFoundError: File not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/api/services/completion_service.py", line 197, in generate_worker
    Completion.generate(
  File "/app/api/core/completion.py", line 34, in generate
    cls.validate_query_tokens(app.tenant_id, app_model_config, query)
  File "/app/api/core/completion.py", line 289, in validate_query_tokens
    llm = LLMBuilder.to_llm_from_model(
  File "/app/api/core/llm/llm_builder.py", line 76, in to_llm_from_model
    return cls.to_llm(
  File "/app/api/core/llm/llm_builder.py", line 55, in to_llm
    model_credentials = cls.get_model_credentials(tenant_id, provider, model_name)
  File "/app/api/core/llm/llm_builder.py", line 115, in get_model_credentials
    return provider_service.get_credentials(model_name)
  File "/app/api/core/llm/provider/llm_provider_service.py", line 32, in get_credentials
    return self.provider.get_credentials(model_id)
  File "/app/api/core/llm/provider/azure_provider.py", line 43, in get_credentials
    config = self.get_provider_api_key(model_id=model_id)
  File "/app/api/core/llm/provider/base.py", line 39, in get_provider_api_key
    return self.get_decrypted_token(provider.encrypted_config)
  File "/app/api/core/llm/provider/azure_provider.py", line 139, in get_decrypted_token
    config['openai_api_key'] = self.decrypt_token(config['openai_api_key'])
  File "/app/api/core/llm/provider/base.py", line 120, in decrypt_token
    return rsa.decrypt(base64.b64decode(token), self.tenant_id)
  File "/app/api/libs/rsa.py", line 47, in decrypt
    raise PrivkeyNotFoundError("Private key not found, tenant_id: {tenant_id}".format(tenant_id=tenant_id))
libs.rsa.PrivkeyNotFoundError: Private key not found, tenant_id: 92af6604-297d-453e-95c4-96b9c37855d3
ERROR:root:internal server error.
Traceback (most recent call last):
  File "/app/api/controllers/console/app/completion.py", line 158, in generate
    for chunk in response:
  File "/app/api/services/completion_service.py", line 404, in generate
    cls.handle_error(result)
  File "/app/api/services/completion_service.py", line 506, in handle_error
    raise Exception(description)
Exception: Private key not found, tenant_id: 92af6604-297d-453e-95c4-96b9c37855d3
ERROR:app:Exception on /console/api/apps/78f25c0b-018d-4fae-a9e3-7070c6fb6793/chat-messages [POST]
Traceback (most recent call last):
  File "/app/api/controllers/console/app/completion.py", line 158, in generate
    for chunk in response:
  File "/app/api/services/completion_service.py", line 404, in generate
    cls.handle_error(result)
  File "/app/api/services/completion_service.py", line 506, in handle_error
    raise Exception(description)
Exception: Private key not found, tenant_id: 92af6604-297d-453e-95c4-96b9c37855d3

[takatost]

Looks like the priv key used to decrypt the api-key is lost. We need to clear out everything in docker/db/data an redeploy, then reinitialize. Make sure your app and worker service volumes exist, otherwise redeploying might result in file loss.

[roberteles]

That worked @takatost – we did lose all the db data by deleting the docker/volumes/db/data folder, but it's all ok (we didn't have much in there yet). Thanks your your help!

[debugger24]

Hi, I am facing the same issue. Not sure why was this key lost.

[jinho-kim-osd]

I am facing the same issue also, I've never changed the environment keys. Why does this happen?

[kevineen]

I am also facing the same issue, but I was able to resolve it with the below command.

docker exec -it docker-api-1 flask reset-encrypt-key-pair

https://docs.dify.ai/getting-started/install-self-hosted/install-faq