Warning message when calling load_index

[pbbhat]

Hello,

I'm using v 0.5.1 of hnswlib

If I call load_index after calling init_index, I get the following warning:

Warning: Calling load_index for an already inited index. Old index is being deallocated.

To avoid the warning, I removed the call to init_index. Although this doesn't show the warning, the index doesn't seem to have successfully loaded. After load_index, the element_count is 0.

I wonder if this is an issue, or if I'm doing something wrong. Please help. Thanks.

[yurymalkov]

Hm. Initing index is not needed if you are going to load it. I suggest referring to https://github.com/nmslib/hnswlib/blob/master/examples/example.py for the example of index loading. I suspect there is something wrong with the arguments (maybe sharing the code will help).

[pbbhat]

Thank you for the prompt response. Yes, I tried the load without initing the index. However, in this scenario, the element_count (after the load) shows 0. Attached is the test code, which is a modified version of the sample you pointed to.

Thanks, Prashanth testhnsw.py.gz

[yurymalkov]

Hm. The code is actually working fine when I remove the p.init_index(max_elements=num_elements, ef_construction=100, M=16) line, but something is wrong when there is an init (while it should be still fine). Will debug it.

[ShuAiii]

@yurymalkov

I'm observing the same issue as @pbbhat. Although. element_count seems to be a smaller problem as the method get_current_count() will return you the correct count:

>>> knn = hnswlib.Index("l2", 10)
>>> knn.init_index(max_elements=1000, ef_construction = 200, M = 16, random_seed = 100)
>>> knn.add_items(data, num_threads=1)
>>> knn.save_index("knn.bin")
>>> new_knn = hnswlib.Index(space='l2', dim=10)
>>> new_knn.load_index("knn.bin", max_elements = 1000)
>>> knn.get_items(knn.get_ids_list())
[[1.0, 23.0, 4.0, 5.0, 6.0, 432.0, 3.0, 4.0, 5.0, 6.0]]
>>> new_knn.get_items(new_knn.get_ids_list())
[[1.0, 23.0, 4.0, 5.0, 6.0, 432.0, 3.0, 4.0, 5.0, 6.0]]
>>> knn.element_count
1
>>> new_knn.element_count
0
>>> new_knn.get_current_count()
1

As you see from the code snippet from above, the index has been successfully loaded, only the property element_count isn't being set correctly.

The bigger problems are the parameters M and ef_construction. Since they don't have the equivalent get_current_count() methods, there is no way for us to "value check" them upon reloading the index.

>>> knn.M
16
>>> new_knn.M
0
>>> knn.ef_construction
200
>>> new_knn.ef_construction
0

Please see full issue here: https://github.com/nmslib/hnswlib/issues/316

Thanks!