Error when no paragraph was matched: torch.cat(): expected a non-empty list of Tensors
To better understand the behavior of pids, I've added and removed (IndexUpdater.remove) short paragraphs. It started crashing with one specific query. It looks like it fails when no pid was matched. Code:
with Run().context(RunConfig(experiment='notebook')): searcher = Searcher(index=index_name) results = searcher.search("7", k=4, filter_fn=None)
Error: File "/home/erel/PycharmProjects/colbert_test/test.py", line 78, in search results = searcher.search("7", k=4, filter_fn=None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/erel/python_env/lib/python3.12/site-packages/colbert/searcher.py", line 67, in search return self.dense_search(Q, k, filter_fn=filter_fn, pids=pids) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/erel/python_env/lib/python3.12/site-packages/colbert/searcher.py", line 129, in dense_search pids, scores = self.ranker.rank(self.config, Q, filter_fn=filter_fn, pids=pids) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/erel/python_env/lib/python3.12/site-packages/colbert/search/index_storage.py", line 104, in rank scores, pids = self.score_pids(config, Q, pids, centroid_scores) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/erel/python_env/lib/python3.12/site-packages/colbert/search/index_storage.py", line 148, in score_pids approx_scores = torch.cat(approx_scores, dim=0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: torch.cat(): expected a non-empty list of Tensors