markitdown
markitdown copied to clipboard
microsoft
Support for GitHub issue/prs to markdown
Enables the following
markitdown https://github.com/andrewyng/aisuite/issues/61
The output would be:
# Request for Python Asyncio Support
I would like to request support for Python’s asyncio in this library. This feature would be particularly beneficial for Python services, which often rely on asynchronous programming for efficient and scalable operations.
Some providers, such as OpenAI, already offer native async support (e.g., `from openai import AsyncOpenAI`), making it straightforward to wrap these APIs. Others, like AWS, have community-supported async wrappers, such as `aioboto3`. For providers without async support, an interim solution using a synchronous wrapper could be implemented while awaiting a proper asyncio implementation.
Asyncio support would greatly enhance the usability of this library. Thank you for considering this enhancement.
**State:** open
**Created at:** 2024-11-26 02:16:05+00:00
**Updated at:** 2024-11-30 01:28:19+00:00
**Comments:**
- sarthakforwet (2024-11-26 03:47:34+00:00): Can you please assign this issue to me?
- soulcarus (2024-11-26 03:59:20+00:00): I refactored the code to use a thread pool instead of asyncio.
Initially, I attempted an asyncio-based solution. However, implementing a feature that solely uses asyncio would have required modifying several lines of code, which would have been time-consuming and inefficient for this specific task.
With just over 30 additional lines of code, I implemented a method that handles the heavy lifting by assigning each model inference to a separate thread. This change results in a performance improvement, reducing execution time by
PR has drifted too far from main. Merging is tough.
Also, I'm not 100% sold on calling an API here (though I suppose we do this for YouTube transcripts)