sparse_dot_topn icon indicating copy to clipboard operation
sparse_dot_topn copied to clipboard

but what is abut other many similarities

Open Sandy4321 opened this issue 3 years ago • 1 comments

great code thanks 1 but what is about other many similarities like https://docs.scipy.org/doc/scipy/reference/spatial.distance.html are you planning to implement some of them ? 2 why you use especially cosine similarity ? 3 Windows calculation results for https://github.com/ing-bank/sparse_dot_topn/blob/master/example/comparison.py

density = 1e-06 nr_vocab = 33554432 n_samples = 1000000 n_duplicates = 1000000 nnz_a = 33554432 nnz_b = 33554432

Original sparse_dot_topn function 24.145782099999998 Threaded function with 1 thread 25.3003591 Threaded function with 2 threads 12.1798754 Threaded function with 3 threads 9.594898599999993 Threaded function with 4 threads 8.203243100000009 Threaded function with 5 threads 6.813879 Threaded function with 6 threads 6.1746841000000074 Threaded function with 7 threads 6.414162599999997 Scipy+numpy original function 773.6003263

Sandy4321 avatar Jul 01 '22 16:07 Sandy4321

  1. no plan for it, which one do you have in mind, and for which use case?
  2. because that typically makes sense in NLP

stephanecollot avatar Jul 03 '22 18:07 stephanecollot