[Feature]: Support dual stack (IPv4/IPv6)

[ptaranat]

The Feature

Currently LiteLLM supports either IPv4 support by setting HOST=0.0.0.0 or IPv6 support by setting HOST=::, but not at the same time. Uvicorn supports this by setting HOST to an empty string "" or None. https://github.com/encode/uvicorn/discussions/1529#discussioncomment-3061823

It appears when HOST is unset, LiteLLM defaults to 0.0.0.0 when this is different from Uvicorn's default behavior.

Motivation, pitch

We connect to LiteLLM on our internal network over IPv6, but we need access to to the application admin UI over IPv4.

Are you a ML Ops Team?

No

Twitter / LinkedIn details

No response

[buildgreatthings]

We have a similar issue to this as well. Could we get a prioritized fix?

[jmackie]

Same here ✋ I'm deploying LiteLLM as an internal service on fly.io, which means it has to listen on :: in order to be visible to other services, but fly's health-checks use IPv4 so I'm currently unable to add health-checks 😬

Hopefully this is an easy one to fix, happy to throw up a PR if that's helpful?

[ishaan-jaff]

picking this up today

[ishaan-jaff]

You can already solve this by setting host to ""

litellm --config proxy_config.yaml --detailed_debug --host ""

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.