lantern

lantern copied to clipboard

Narek/hnswbuild single wal pass

19

Ngalstyan4

#147

broccoliSpicy

Use proper palloc-based memory allocator in usearch

1

Currently usearch code uses the default allocator (malloc-based) We should use palloc to make sure all memory is manged through postgres We can do that by adding a field to...

Ngalstyan4

It would be good to have a GUC-gated flag that enables statistics collection on index usage patterns in a systems table (e.g. table with schema `lantern.index_usage_statistics(indexoid OID, tableoid OID, indexdef...

Ngalstyan4

https://github.com/lanterndata/lantern/blob/33e6bcfbe4336b61d11344ef5b55b88a6fd688a2/src/hnsw/utils.c#L81-L82

broccoliSpicy

Reduce critical section on inserts

8

Currently we hold a lock on the header page of the index for much much longer than necessary on insertions. - [ ] Benchmark this and see if concurrent inserts...

Ngalstyan4

- Added new option to index `bool external` - If external is true we will call `_create_external_index` function from lantern_extras - The multicore index creation will be done in Rust...

var77

- [x] Add `drop_quantization` API that deletes all PQ-quantization resources associated with a table - [x] Add relevant hooks to make sure all quantization resources and table hooks are deleted...

Ngalstyan4

Currently, when the binary format of the index changes we reindex only the indexes created via the postgres's `CREATE INDEX` API we should make sure to reindex externally created indexes...

Ngalstyan4

Lantern’s Performance vs. pgvector - Authenticity and Future Improvements

3

Hi Lantern Team, I noticed your commitment to performance excellence in the README. However, a comparison on tembo.io (https://tembo.io/blog/postgres-vector-search-pgvector-and-lantern) suggests Lantern falls behind pgvector in certain metrics.  Could you...

Haskely