how to setup colbertv2 model on my own data?

[manoj-kore]

Hi. In the notebooks, we can use a pre-set server for colbert model that works on wikipedia data. But I want to know how to use the same for my own set of documents. Can anybody please help?

[okhat]

Yes! We’re almost done releasing that cc: @VThejas

[okhat]

See here: https://github.com/stanford-futuredata/ColBERT#running-a-lightweight-colbertv2-server

Before you launch the server, use the ColBERT intro notebook (or the Overview in the ColBERT README) to index your collection

[theholymath]

Is there a way to pipe a FAISS index into this pipeline? Is there an example of how to ingest other vector DBs?

[drawal1]

A RAG example using Pinecone for retrieval would be helpful. Or... are there reasons not to use Pinecone here?

[okhat]

You can use Pinecone for sure, @drawal1. We don't have it built-in though. Would you like to add it?

We just need something as minimal as this wrapper for it:

import dspy

class Pinecone(dspy.Retrieve):
    def __init__(self, k=3):
        super().__init__(k=k)
        # TODO: initialize pinecone here with any kwargs you need

    def forward(self, query):
        # TODO: passages = search with pinecone for self.k top passages for `query`
        return dspy.Prediction(passages=passages)

For more info (probably not necessary), see https://github.com/stanfordnlp/dspy/blob/main/dspy/retrieve/retrieve.py

[okhat]

then you can use dspy.Pinecone instead of dspy.Retrieve (there's a cleaner way to do this, but we can start like that)

[drawal1]

@okhat, ty! I tested below code and it works. I will submit a pull request if this looks reasonable

"""
Retriever model for Pinecone
"""

import pinecone  # type: ignore
import openai   # type: ignore
import dspy     # type: ignore

OPENAI_API_KEY = 'YOUR OPENAPI KEY'
PINECONE_API_KEY = 'YOUR_PINECONE_API_KEY'
PINECONE_ENVIRONMENT = 'YOUR PINCONE ENVIRONMENT' # for example 'us-east4-gcp'
INDEX_NAME = "YOUR PINECONE INDEX NAME" # You should have an index build already. See Pinecone docs
EMBED_MODEL = "YOUR EMBEDDING MODEL" # For example 'text-embedding-ada-002' for OpenAI gpt-3.5-turbo

def init_pinecone(pinecone_api_key, pinecone_env, index_name):
    """Initialize pinecone and load the index"""
    pinecone.init(
        api_key=pinecone_api_key,  # find at app.pinecone.io
        environment=pinecone_env,  # next to api key in console
    )

    return pinecone.Index(index_name)

PINECONE_INDEX = init_pinecone(PINECONE_API_KEY, PINECONE_ENVIRONMENT, INDEX_NAME)

class PineconeRM(dspy.Retrieve):
    """
        Retrieve module for Pinecone
        Example usage:
            self.retrieve = PineconeRM(k=num_passages)
    """
    def __init__(self, k=3):
        super().__init__(k=k)

    def forward(self, query_or_queries):
        """ search with pinecone for self.k top passages for query"""
        # convert query_or_queries to a python list if it is not
        queries = [query_or_queries] if isinstance(query_or_queries, str) else query_or_queries

        embedding = openai.Embedding.create(input=queries, engine=EMBED_MODEL, openai_api_key=OPENAI_API_KEY)
        query_vec = embedding['data'][0]['embedding']

        # retrieve relevant contexts from Pinecone (including the questions)
        results_dict = PINECONE_INDEX.query(query_vec, top_k=self.k, include_metadata=True)

        passages = [result['metadata']['text'] for result in results_dict['matches']]
        return dspy.Prediction(passages=passages)

[drawal1]

@okhat I have submitted the pull request, fyi