ELQ failing at faiss index search while training on WebQSP dataset

[sairamgv]

When trying to train the ELQ model on the WebQSP dataset, encountered the following error.

Traceback (most recent call last):
  File "elq/biencoder/train_biencoder.py", line 602, in <module>
    main(params)
  File "elq/biencoder/train_biencoder.py", line 293, in main
    logger=logger, faiss_index=cand_encs_index,
  File "elq/biencoder/train_biencoder.py", line 101, in evaluate
    top_cand_logits_shape, top_cand_indices_shape = faiss_index.search_knn(embedding_ctxt, 10)
  File "/data/entity-linking/ELQ/elq/index/faiss_indexer.py", line 123, in search_knn
    scores, indexes = self.index.search(query_vectors, top_k)
  File "/data/anaconda3/envs/el4qa2/lib/python3.7/site-packages/faiss/__init__.py", line 132, in replacement_search
    n, d = x.shape
ValueError: too many values to unpack (expected 2)

[kingsaint]

Looks like the issue is with the embedding_ctxt . I assume it should be a 2D tensor/numpy array, but it is a 4D tensor when it is passed to search_knn.

[mohannahoveyda]

Hi @sairamgv,

I was wondering if you managed to handle this?