examples icon indicating copy to clipboard operation
examples copied to clipboard

Query vector dimension 76800 does not match the dimension of the index 768

Open SaraAmd opened this issue 1 year ago • 3 comments

I get the following error ApiException: (400) Reason: Bad Request HTTP response headers: HTTPHeaderDict({'content-type': 'application/json', 'date': 'Mon, 06 Mar 2023 17:34:00 GMT', 'x-envoy-upstream-service-time': '2', 'content-length': '110', 'server': 'envoy', 'connection': 'close'}) HTTP response body: {"code":3,"message":"Query vector dimension 76800 does not match the dimension of the index 768","details":[]}

For the code snippet:

import random

batch_size = 100 triplets = []

for i in tqdm(range(0, len(pairs), batch_size)): # embed queries and query pinecone in batches to minimize network latency i_end = min(i+batch_size, len(pairs)) queries = [pair[0] for pair in pairs[i:i_end]] print(len(queries)) pos_passages = [pair[1] for pair in pairs[i:i_end]] #print((pos_passages)) # create query embeddings query_embs = model.encode(queries, convert_to_tensor=True, show_progress_bar=False) # search for top_k most similar passages res = index.query(query_embs.tolist(), top_k=1) # iterate through queries and find negatives for query, pos_passage, query_res in zip(queries, pos_passages, res['results']): top_results = query_res['matches'] # shuffle results so they are in random order random.shuffle(top_results) for hit in top_results: neg_passage = pairs[int(hit['id'])][1] # check that we're not just returning the positive passage if neg_passage != pos_passage: # if not we can add this to our (Q, P+, P-) triplets triplets.append(query+'\t'+pos_passage+'\t'+neg_passage) break

SaraAmd avatar Mar 06 '23 18:03 SaraAmd