llama-stack icon indicating copy to clipboard operation
llama-stack copied to clipboard

chore: convert blocking calls to async calls in some providers

Open jaideepr97 opened this issue 7 months ago • 2 comments

What does this PR do?

Converts blocking calls to async calls within the following providers/components:

  • runpod (inference)
  • sentence_transformers (inference)
  • litellm (inference)

Partially addresses #1489

Test Plan

[Describe the tests you ran to verify your changes with result summaries. Provide clear instructions so the plan can be easily re-executed.]

jaideepr97 avatar Mar 14 '25 17:03 jaideepr97