John Trengrove

HNSW Multi-value vectors (colbert)

1

### Describe your feature request With the introduction of https://github.com/weaviate/weaviate/issues/2465 via https://github.com/weaviate/weaviate/pull/4180, Weaviate can now create multiple vector indices per class (i.e a separate vector for `description` and `title`). As...

Slice out of bounds recovering objects

14

Reference https://weaviate.slack.com/archives/C017EG2SL3H/p1683579170564039 ``` > kubectl logs weaviate-659459fc5d-fpwzt -f {"action":"config_load","config_file_path":"./weaviate.conf.json","level":"info","msg":"no config file specified, using default or environment based","time":"2023-05-09T03:20:13Z"} {"action":"startup","default_vectorizer_module":"text2vec-openai","level":"info","msg":"the default vectorizer modules is set to \"text2vec-openai\", as a result all new...

Vector index rebuild api

1

A new api to rebuild the vector index - Ability to rebuild HNSW / PQ with different parameters (`maxConnections` or `efConstruction`) - Shard will go read only while being rebuilt...

Fix flaky dynamic restore test

2

### What's being changed: ### Review checklist - [ ] Documentation has been updated, if necessary. Link to changed documentation: - [ ] Chaos pipeline run or not necessary. Link...

- When a dynamic index is upgraded, all data is reindexed into a new HNSW index in the background. - In the case where hundreds of these operations are happening...

[HNSW] Sparse visited list implementation

1

- Visited lists are used to track which nodes in the HNSW graph have already been visited in the graph - The current implementaiton uses [list sets](https://github.com/weaviate/weaviate/blob/main/adapters/repos/db/vector/hnsw/visited/list_set.go) and isn't memory...

[HNSW] In-place normalization of vectors

2

### What's being changed: - With the default `cosine` distance metrics all vectors are stored on disk in original un-normalized format and both query and object vectors are normalized as...

Sample reassign neighbor logs

1

### What's being changed: - These logs can generate millions of events in large clusters due to non critical events (node already being deleted) - This PR reduces logging by...

Improve PQ distance calculation

1

### What's being changed: - This change does minor changes to the product quantization distance between compressed vectors function to improve performance. - Gives around a 15% improvement with 512...