sparse_dot_topn
sparse_dot_topn copied to clipboard
but what is abut other many similarities
great code thanks 1 but what is about other many similarities like https://docs.scipy.org/doc/scipy/reference/spatial.distance.html are you planning to implement some of them ? 2 why you use especially cosine similarity ? 3 Windows calculation results for https://github.com/ing-bank/sparse_dot_topn/blob/master/example/comparison.py
density = 1e-06 nr_vocab = 33554432 n_samples = 1000000 n_duplicates = 1000000 nnz_a = 33554432 nnz_b = 33554432
Original sparse_dot_topn function 24.145782099999998 Threaded function with 1 thread 25.3003591 Threaded function with 2 threads 12.1798754 Threaded function with 3 threads 9.594898599999993 Threaded function with 4 threads 8.203243100000009 Threaded function with 5 threads 6.813879 Threaded function with 6 threads 6.1746841000000074 Threaded function with 7 threads 6.414162599999997 Scipy+numpy original function 773.6003263
- no plan for it, which one do you have in mind, and for which use case?
- because that typically makes sense in NLP