Implement Parallel/Asynchronous API Calls for Faster Topic Generation

[estevanmendes]

Problem Description:

The current process for generating topics using the OpenAI API is experiencing a significant bottleneck due to sequential processing. Each text requires an individual API call, which takes approximately 1 second.

While this performance might be acceptable for small datasets, it becomes a major limitation for larger corpora. For instance, processing my dataset is estimated to take around 10 hours with the current approach. This sequential execution prevents users from leveraging higher OpenAI API tiers (like increased Requests Per Minute - RPM and Tokens Per Minute - TPM limits) to speed up processing.

Proposed Solution:

To drastically improve the processing speed, I propose implementing parallel or asynchronous processing for the OpenAI API calls involved in topic generation.

This would involve making multiple API requests concurrently, potentially using techniques like threading, multiprocessing, or asyncio. A crucial aspect of this implementation would be the careful management of OpenAI's rate limits (RPM and TPM) to ensure stable and efficient processing without hitting API errors.

Benefits:

• Significant Speedup: Reduce processing time from hours to potentially minutes for large datasets.
• Scalability: Allow users with higher API tiers to process data much faster.
• Improved User Experience: Make the library more practical and accessible for real-world applications involving large text volumes.

Current Status & Effort:

I understand the technical requirements involve managing concurrency and respecting API rate limits. I have successfully implemented similar parallel API processing solutions in other libraries (e.g., using Langchain), so I know it's feasible.

While I could potentially implement this feature myself, it would require a significant time investment (likely 4-8 hours) to do it within this library's structure.

Call for Contribution:

I'm opening this issue to:

1. Highlight this performance bottleneck.
2. Propose a concrete solution (parallel/async processing).
3. Gauge interest from the community.
4. See if anyone has suggestions on the best approach within this library's architecture.
5. Find collaborators who might be interested in contributing to the design or implementation of this feature.

Any input, guidance, or willingness to contribute to this effort would be greatly appreciated!