[ENH] Replace argsort by argpartition in similarity search
Describe the feature or idea you want to propose
After review with Patrick, to do after #2473 is merged.
Replace the use of argsort by argpartition in files such as series/_commons.py (e.g. _extract_top_k_from_dist_profile) to speed up the functions using those.
Describe your proposed solution
Replace argsort by argpartition and adjust the algorithms, notably for the successive call to argpartition in cases where trivial matches should be avoided.
Describe alternatives you've considered, if relevant
No response
Additional context
No response
Hi @baraline, I would like to work on this issue. Can I start working from that branch and rebase it once the PR gets merged?
Hi, if by that branch you mean the branch from 2473 yes you can.
Yes, from 2473. I will start working.
@aeon-actions-bot assign @Kaustbh
Hi, I have made the changes just waiting for the PR #2473 to get merged.
Hey @Kaustbh, PR is in, you can raise yours, thanks !
I will make the PR very soon.
Hi @baraline, I noticed that when allow_trivial_matches=False, the algorithm sometimes returns fewer than k matches. Is this expected behavior due to the exclusion zone filtering out trivial matches? Or should the algorithm always try to return exactly k matches?
Hi, great question. This is should happen when the number of admissibles matches is less than k. Which can happen with small series and use of trival matches to false. Did you have a specific example where the behaviour was suspicious?
Thank you for the clarification! Yes, that makes sense. It can happen because of small series, can it also happen if we gave large value for length parameter?
@baraline, I have made the PR Please review it and let me know if any changes or improvements are needed.