microsoft
Python: Introduce Pydantic settings
Motivation and Context
SK Python is tightly coupled to the use of a .env file to read all secrets, keys, endpoints, and more. This doesn't scale well for users who wish to be able to use environment variables with their SK Applications. By introducing Pydantic Settings, it is possible to use both environment variables as well as have a fall-back to a .env file (via a env_file_path parameter), if desired.
By introducing Pydantic Settings, we are removing the requirement to have to create Text/Embedding/Chat completion objects with an api_key or other previously required information (in the case of AzureChatCompletion that means an endpoint, an api_key, a deployment_name, and an api_version). When the AI connector is created, the Pydantic settings are loaded either via env vars or the fall-back .env file, and that means the user can create a chat completion object like:
chat_completion = OpenAIChatCompletion(service_id="test")
or, to optionally override the ai_model_id env var:
chat_completion = OpenAIChatCompletion(service_id="test", ai_model_id="gpt-4-1106")
Note: we have left the ability to specific an api_key/org_id for OpenAIChatCompletion or a deployment_name, endpoint, base_url, and api_version for AzureChatCompletion as before, but if your settings are configured to use env vars/.env file then there is no need to pass this information.
Description
The PR introduces the use of Pydantic settings and removes the use of the python-dotenv library.
- Closes #1779
- Updates notebooks, samples, code and tests to remove the explicit config of api_key or other previous .env files values.
- Adds new unit test config using monkeypatch to simulate env variables for testing
- All unit and integration tests passing
Contribution Checklist
- [X] The code builds clean without any errors or warnings
- [X] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone :smile:
Python 3.10 Test Coverage Report •
File Stmts Miss Cover Missing semantic_kernel/connectors/ai/google_palm/services gp_chat_completion.py 66 16 76% 10, 135–140, 151–152, 201, 204, 207–208, 216, 220–221, 229 gp_text_completion.py 38 6 84% 10, 57–58, 64–65, 94 gp_text_embedding.py 27 5 81% 10, 47–48, 57–58 semantic_kernel/connectors/ai/open_ai/services azure_chat_completion.py 76 34 55% 248, 254–255, 264–265, 270–287, 291–298, 310–321 open_ai_text_completion.py 16 1 94% 128 open_ai_text_embedding.py 13 1 92% 83 semantic_kernel/connectors/search_engine bing_connector.py 35 35 0% 3–71 semantic_kernel/core_plugins/sessions_python_tool sessions_python_plugin.py 95 9 91% 76–82, 122, 168, 172 TOTAL 5867 1019 83%
Python 3.10 Unit Test Overview
| Tests | Skipped | Failures | Errors | Time |
|---|---|---|---|---|
| 1278 | 1 :zzz: | 0 :x: | 0 :fire: | 13.545s :stopwatch: |
