exo-explore
[BOUNTY - $200] make it so process_prompt is cancellable outside of a download (follow up to #305)
See #305
Currently the fix is to shield process_prompt however this also makes it so that if a chatgpt api request times out, it never gets cancelled even when a download is not in progress.
This can lead to behaviour where exo keeps generating even after a chatgpt api request has timed out.
Gotchas: make sure however exo works right is actually cancellable end-to-end. If an inference is in progress, that needs to be cancellable - InferenceEngines need to support this behaviour.
Started working on this. But since holidays are near, it could take a few days.
Hi @AlexCheema,
I'd like to work on this issue. I understand the main goal is to make process_prompt cancellable when not downloading while maintaining download protection from #305.
Would you like me to proceed with the implementation, so can you assign this issue to me?
