BubbleCal

Results 46 comments of BubbleCal

IVF_HNSW_PQ index causes OOM，any way to fix this?

@brucewkz increasing `num_partitions` should help, we got to keep vectors of at least one partition in memory during indexing. OOM means that the memory can't serve some large partition. And...

IVF_HNSW_PQ index causes OOM，any way to fix this?

We've implemented a new index builder that can use much less memory than before, it has been applied for building IVF_HNSW_SQ. This [PR](https://github.com/lancedb/lance/pull/2552) applies it for IVF_HNSW_PQ

create_fts_index doc missing

~~As we are deprecating this method, do we still need to add this?~~ ~~after [this](https://github.com/lancedb/lancedb/pull/1483), the FTS index can be created via `create_scalar_index`~~ Will add it

Incremental data indexing mode

We missed to support `optimize` when adding `IVF_HNSW_PQ` [This](https://github.com/lancedb/lance/pull/2552) can fix this but it requires to re-create the index

Incremental data indexing mode

> version: 0.17.0 language: typescript > > Optimize is too expensive. I just want to make the new index text to be searched. I tried recreate the fts index, but...

chore: don't print warning if fp16 kernel not build

how can we know if fp16 kernel is supported after this?