Support Multi-threaded Benchmarks for Production-Realistic Performance Metrics

[mesibo]

Currently, ANN-Benchmarks enforces single-CPU execution during experimentation, disabling multi-threading capabilities at the hardware level (AWS single-cpu mode) so that even libraries supporting multi-core CPU/threading can't use it. While batch mode exists, it doesn't address the potential performance benefits that multi-threading could offer.

There are ANN libraries supporting multi-core/multi-threading processing to improve performance. However, the single-threaded benchmark is unlikely to project their true potential in the production environment. This is especially true since most end users are likely to run ANN on multi-core CPUs, and there is no reason they should choose the wrong implementation based on single CPU benchmarks. Running and providing both single- and multi-threaded benchmarks will allow users to view more realistic benchmarks, matching production setup.

We request to consider adding multi-threading benchmarks alongside existing single-threaded tests.

• Optionally, Add a new flag (e.g., --threading) to enable multi-threading evaluation
• Create separate performance plots for single-threaded vs multi-threaded execution

We're happy to contribute to the implementation if needed and have already made some changes for local benchmarking to test ANN in multi-threaded mode.

Looking forward to your thoughts on this.