Upgrade to text-embedding-3-large model as default, with vector storage optimizations

[pamelafox]

trafficstars

Purpose

This pull request changes the default embedding model to text-embedding-3-large, with 3072 dimensions, along with these AI Search vector storage optimizations:

• Truncate dimensions to 1024
• Binary quantization
• Preserve originals in the search index for rescoring
• Don't store the vectors in the search index itself

See this notebook for a demonstration of the effects of those optimizations. Due to the rescoring, the search quality remains high.

This PR introduces a new environment variable AZURE_SEARCH_FIELD_NAME_EMBEDDING so that developers can theoretically have multiple fields in their index, for different embedding sizes/models.

This PR also changes the SKU for all models to GlobalStandard. It's becoming really tricky to find a region for the Standard SKU that works for all the models. Some developers may not be comfortable with GlobalStandard, depending on their regulations, so they can still change the SKU manually as desired.

Fixes #2383

Does this introduce a breaking change?

When developers merge from main and run the server, azd up, or azd deploy, will this produce an error? If you're not sure, try it out on an old environment.

[X] Yes - I am trying to make it backwards compatible, but it's hard! I suspect that developers that recently deployed gpt-4o-mini with Standard sku will get an error, and need to run `azd env set` to change the deployment name or sku name.
[ ] No

Does this require changes to learn.microsoft.com docs?

This repository is referenced by this tutorial which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial, check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.

[ ] Yes
[X] No

Type of change

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

Code quality checklist

See CONTRIBUTING.md for more details.

• [ ] The current tests all pass (python -m pytest).
• [ ] I added tests that prove my fix is effective or that my feature works
• [ ] I ran python -m pytest --cov to verify 100% coverage of added lines
• [ ] I ran python -m mypy to check for type errors
• [ ] I either used the pre-commit hooks or ran ruff and black manually on my code.

[github-actions[bot]]

Check Country Locale in URLs

We have automatically detected added country locale to URLs in your files. Review and remove country-specific locale from URLs to resolve this issue.

Check the file paths and associated URLs inside them. For more details, check our Contributing Guide.

File Full Path	Issues
docs/deploy_features.md	# Link Line Number 1 https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#models-by-deployment-type 197

[github-actions[bot]]

Check Broken URLs

We have automatically detected the following broken URLs in your files. Review and fix the paths to resolve this issue.

Check the file paths and associated broken URLs inside them. For more details, check our Contributing Guide.

File Full Path	Issues
docs/deploy_troubleshooting.md	# Link Line Number 1 https://stackoverflow.com/questions/35569042/ssl-certificate-verify-failed-with-python3/43855394#43855394 11
docs/customization.md	# Link Line Number 1 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 41 2 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 43 3 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 45 4 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 45 5 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question_vision.prompty 55 6 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py 59 7 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 62 8 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 64 9 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question_vision.prompty 73 10 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 173 11 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 174 12 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/textsplitter.py 176

[github-actions[bot]]

Check Broken URLs

We have automatically detected the following broken URLs in your files. Review and fix the paths to resolve this issue.

Check the file paths and associated broken URLs inside them. For more details, check our Contributing Guide.

File Full Path	Issues
docs/deploy_troubleshooting.md	# Link Line Number 1 https://stackoverflow.com/questions/35569042/ssl-certificate-verify-failed-with-python3/43855394#43855394 11
docs/customization.md	# Link Line Number 1 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 41 2 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 43 3 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 45 4 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 45 5 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question_vision.prompty 55 6 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py 59 7 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 62 8 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 64 9 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question_vision.prompty 73 10 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 173 11 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 174 12 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/textsplitter.py 176

[github-actions[bot]]

Check Broken URLs

We have automatically detected the following broken URLs in your files. Review and fix the paths to resolve this issue.

Check the file paths and associated broken URLs inside them. For more details, check our Contributing Guide.

File Full Path	Issues
data/Contoso_Electronics_Company_Overview.md	# Link Line Number 1 http://www.contoso.com 48
docs/deploy_troubleshooting.md	# Link Line Number 1 https://stackoverflow.com/questions/35569042/ssl-certificate-verify-failed-with-python3/43855394#43855394 11
docs/customization.md	# Link Line Number 1 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/chatreadretrieveread.py 39 2 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 41 3 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 43 4 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_query_rewrite.prompty 45 5 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 45 6 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question_vision.prompty 55 7 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py 59 8 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 62 9 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 64 10 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question_vision.prompty 73 11 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 173 12 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 174 13 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/textsplitter.py 176

[github-actions[bot]]

Check Broken URLs

We have automatically detected the following broken URLs in your files. Review and fix the paths to resolve this issue.

Check the file paths and associated broken URLs inside them. For more details, check our Contributing Guide.

File Full Path	Issues
README.md	# Link Line Number 1 https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#role-based-access-control-administrator-preview 79 2 https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator 79 3 https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#owner 79 4 https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#role-based-access-control-administrator-preview 79 5 https://learn.microsoft.com/azure/cognitive-services/openai/concepts/models#model-summary-table-and-region-availability 178 6 https://learn.microsoft.com/azure/developer/python/get-started-app-chat-template?toc=%2Fazure%2Fdeveloper%2Fai%2Ftoc.json&bc=%2Fazure%2Fdeveloper%2Fai%2Fbreadcrumb%2Ftoc.json&tabs=github-codespaces 273 7 https://learn.microsoft.com/azure/search/search-what-is-azure-search 275 8 https://learn.microsoft.com/azure/cognitive-services/openai/overview 276
docs/customization.md	# Link Line Number 1 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 43 2 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question.prompty 45 3 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/chat_answer_question_vision.prompty 55 4 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py 59 5 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 62 6 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question.prompty 64 7 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/prompts/ask_answer_question_vision.prompty 73 8 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 173 9 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/searchmanager.py 174 10 https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/prepdocslib/textsplitter.py 176

[pamelafox]

@mattgotteiner I made changes to how vector fields are handed for gpt-vision as it was previously using the vector field names in the settings, and also, I think it was actually buggy the way it was implemented. Now it's done very similar to retrieval mode, with either text, image, or both, as the options.