semantic-kernel
semantic-kernel copied to clipboard
.Net: dotnet Pinecone connector improvements: move to Pinecone.NET and new VectorRecordStore API
Motivation and Context
SK memory connectors APIs are being re-designed and the individual connector implementations need to adjust accordingly. This work handles the transition for Pinecone database. To take advantage of large breaking changes, also delegating low-level operations against the database to Pinecone.NET package.
Addresses https://github.com/microsoft/semantic-kernel/issues/6678
Description
Pinecone recently went through a big overhaul of their APIs (https://docs.pinecone.io/guides/operations/migrate-to-the-new-api). Old API was deprecated and will be removed in the future. Additionally, Pinecone added support for serverless indexes which are supposed to be much cheaper in terms of operating costs (up to 50x according to their blog post - https://www.pinecone.io/blog/serverless/). SK's low-level driver was lacking those improvements, while Pinecone.NET is up to date so users can take advantage of the new features right away. Also this move avoids future effort duplication within the .net ecosystem and lowers maintainability cost on the SK side. Testing: Pinecone does not have a local development/testing story so the integration tests must run against live service. Added xuint infrastructure which allows for conditionally executing (or skipping) the tests. In order to run them, Pinecone API key must be provided via user secrets, like so: dotnet user-secrets set "PineconeApiKey" "your_Pinecone_API_key"
Contribution Checklist
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations (not in the code I added)
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone :smile: