run non-stream requests as async calls to avoid blocking requests

[eliran89c]

*Issue #22

Description of changes: This PR demonstrates that running boto3 in async mode resolves the issue described.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[daixba]

Thanks for the PR, I will need to validate it first. Will try to update as soon as I can.

[daixba]

For urgent usage, please do a customization to the code and deploy and host your own ECR images.

[eliran89c]

For urgent usage, please do a customization to the code and deploy and host your own ECR images.

Thanks, we are using custom deployment for EKS but I believe this is something others will benefit from

[Wh1isper]

I mostly agree with this pr and prefer from fastapi.concurrency import run_in_threadpool which using anyio for both asyncio and trio backend.

[reckless-huang]

I think this is very important, can it be merged?

[Wh1isper]

@reckless-huang I'm currently using litellm (proxy only).