ColBERT icon indicating copy to clipboard operation
ColBERT copied to clipboard

Error when no paragraph was matched: torch.cat(): expected a non-empty list of Tensors

Open AnywhereSoftware opened this issue 1 year ago • 0 comments

To better understand the behavior of pids, I've added and removed (IndexUpdater.remove) short paragraphs. It started crashing with one specific query. It looks like it fails when no pid was matched. Code:

with Run().context(RunConfig(experiment='notebook')): searcher = Searcher(index=index_name) results = searcher.search("7", k=4, filter_fn=None)

Error: File "/home/erel/PycharmProjects/colbert_test/test.py", line 78, in search results = searcher.search("7", k=4, filter_fn=None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/erel/python_env/lib/python3.12/site-packages/colbert/searcher.py", line 67, in search return self.dense_search(Q, k, filter_fn=filter_fn, pids=pids) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/erel/python_env/lib/python3.12/site-packages/colbert/searcher.py", line 129, in dense_search pids, scores = self.ranker.rank(self.config, Q, filter_fn=filter_fn, pids=pids) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/erel/python_env/lib/python3.12/site-packages/colbert/search/index_storage.py", line 104, in rank scores, pids = self.score_pids(config, Q, pids, centroid_scores) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/erel/python_env/lib/python3.12/site-packages/colbert/search/index_storage.py", line 148, in score_pids approx_scores = torch.cat(approx_scores, dim=0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: torch.cat(): expected a non-empty list of Tensors

AnywhereSoftware avatar Sep 29 '24 11:09 AnywhereSoftware