Python: Introducing Keyword Hybrid Search and Lambda Filters for Azure AI Search

[eavanvalkenburg]

Motivation and Context

Hybrid Adds a new abstraction for keyword hybrid search Implements this for Azure AI Search Adds search tests for Azure AI Search Filters Adds support for a Callable (lambda func) as a filter in VectorSearchOptions Implements a ast walker to go from a lambda to a string for the filter Adds a set of test parameter that can be extended

This further does some work on the vector abstractions:

• rename is_filterable to is_indexed and is_full_text_searchable to is_full_text_indexed: Closes #11598
• remove local_embedding from embedding field, adds has_local_embedding to data field, this allows one to create a data model without local embedding, in the old model that required a vector field anyway, now that can be set on the data field that needs to be embedded by the service.
• dimensions is now a required field: Closes #11517
• Add support for getting records without a key, with order_by options and top and skip: Closes #9911
• Add does_collection_exists and delete_collection methods to VectorStore: Closes #11597

Relates to #10561

Description

Contribution Checklist

• [x] The code builds clean without any errors or warnings
• [x] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• [x] All unit tests pass, and I have added new tests where possible
• [ ] I didn't break anyone :smile:

[markwallace-microsoft]

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
3488	6 :zzz:	0 :x:	0 :fire:	1m 42s :stopwatch:

[roji]

If I understand correctly, there aren't any integration tests for this, right? I'm seeing what looks like unit tests only for now?

[eavanvalkenburg]

@roji indeed int tests still to be done, made a note

[eavanvalkenburg]

Replace by multiple smaller PRS