pinecone-datasets
pinecone-datasets copied to clipboard
to_index() should always use gRPC for bulk upserts
@miararoy my bad, I missed this in #22. That this code should never have been merged - it breaks one of the key principles behind pinecone-datasets
Problem
One of the design principles of pinecone-dataset from day one was providing fast bulk upserts via gRPC. which isn't optional. The only change from version 0.5 to 0.6 should have been the underlying client - from still beta Client 3.0 to Client 2.2[grpc].
pinecone-datasets is not meant to support REST based upserts, which can be achieved through the client directly.
Solution
Made pinecone-client[grpc] a mandatory requirement, and use GRPCIndex as the only supported index type
Type of Change
- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change requires a documentation update
- [ ] Infrastructure change (CI configs, etc)
- [ ] Non-code change (docs, etc)
- [ ] None of the above: (explain here)
Test Plan
Full coverage in current unit tests