ColBERT icon indicating copy to clipboard operation
ColBERT copied to clipboard

How to set chunk_size

Open kevinningthu opened this issue 11 months ago • 0 comments

def get_chunksize(self): return min(25_000, 1 + len(self) // Run().nranks)

Seems like it is set to 25_000 when the dataset is very large.

How to set the parameter chunk_size?

Thanks!

kevinningthu avatar Mar 06 '24 14:03 kevinningthu