[Bug]: Custom Proxy Base Url not working

[DanielOtter]

What happened?

I was trying to deploy litellm with custom proxy base url settings but get requests of /sso and /ui do not append the base url and call the wrong urls. Since I am using a url map in my setup to redirect requests with different base urls to their respective services, these requests go to the void.

I followed the docs and built my image with a slightly edited Dockerfile:

# Use the provided base image
FROM ghcr.io/berriai/litellm:litellm_fwd_server_root_path-dev

# Set the working directory to /app
WORKDIR /app

# Install Node.js and npm (adjust version as needed)
RUN apt-get update && apt-get install -y nodejs npm

# Copy the UI source into the container
COPY ./ui/litellm-dashboard /app/ui/litellm-dashboard

# Set an environment variable for UI_BASE_PATH
# This can be overridden at build time
# set UI_BASE_PATH to "<your server root path>/ui"
ENV UI_BASE_PATH="/litellm/ui"

# Build the UI with the specified UI_BASE_PATH
WORKDIR /app/ui/litellm-dashboard
RUN npm install
RUN UI_BASE_PATH=$UI_BASE_PATH npm run build

# Create the destination directory
RUN mkdir -p /app/litellm/proxy/_experimental/out

# Move the built files to the appropriate location
# Assuming the build output is in ./out directory
RUN rm -rf /app/litellm/proxy/_experimental/out/* && \
    mv ./out/* /app/litellm/proxy/_experimental/out/

# Switch back to the main app directory
WORKDIR /app
RUN pip install prisma 
RUN prisma generate

RUN chmod +x entrypoint.sh

COPY config.yaml config.yaml

EXPOSE 8080/tcp

ENTRYPOINT ["litellm"]
CMD ["--port", "8080", "--config", "config.yaml"]

The environment variable of the running container is set to: SERVER_ROOT_PATH="/litellm"

Relevant log output

[01/Oct/2024:13:18:29 +0000] "GET /ui/favicon.ico HTTP/1.1" 200 944 "https://placeholderdomain.com/litellm/ui/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36"

Twitter / LinkedIn details

No response

[krrishdholakia]

cc: @ishaan-jaff - i think you have more context on custom base url's

[max0uuuu]

I think I have the same issue

[ma-armenta]

I noticed this too. By reading source code, python code has multiple locations where /ui is hardcoded as the dashboard ui path.

@ishaan-jaff, I can look into it.

[ishaan-jaff]

that would be great- can you add testing + tag me on the PR @ma-armenta

[DanielOtter]

Hi, has there been progress with this issue. Is there any way I can assist?

[Jflick58]

@DanielOtter can you provide more info for your build flow? I'm running into issues with this as reported in #7318

[DanielOtter]

Well, it has been a few months, but basically, like I have said, I followed the guide in the documentation and set the UI_BASE_PATH in my dockerfile. Then I created a docker image with it and ran it with the environment variable SERVER_ROOT_PATH. What specifically do you want to know about the build flow?

In one of my projects several generic paths are already in use like for example /ui, that is why I wanted to change the path. And that is how I noticed quickly that something was wrong, when traffic was not going through the right endpoints.

[DanielOtter]

@Jflick58 If you look at my Dockerfile in my description of the issue the documentation then mentioned a specific base image: ghcr.io/berriai/litellm:litellm_fwd_server_root_path-dev. I have seen your issue, maybe the silent change to the main image introduced this bug.

[Jflick58]

Do you know what version you built off of?

[DanielOtter]

ghcr.io/berriai/litellm:litellm_fwd_server_root_path-dev refers to version 1.40.3 I think, while the actual version I was trying to build was 1.48.x.

[krrishdholakia]

i think i'm able to repro this

[krrishdholakia]

https://github.com/user-attachments/assets/ecbc9bb7-5c7b-4f7a-b349-08d0605c88f8

Fixed a small bug in the redirect - this now works for me. Will try and get this out by EOD. If someone (@Jflick58 / @DanielOtter / @ma-armenta ) can help confirm this works, that would be great!

[DanielOtter]

Hi, I finally have time to try this out. It is not working for me but this may be a mistake on my part with the dockerfile.

Here my errorlog when trying to access /litellm/ui wiht custom base path set to /litellm:

2025-03-14 04:11:11.479 CET
GET30494 B3 msChrome 134 https://redacteddomain.com/litellm/ui/
2025-03-14 04:11:11.491 CET
INFO: 169.254.169.126:48468 - "GET /litellm/ui/ HTTP/1.1" 304 Not Modified

Here my dockerfile, it has two stages to clone the repository and build the basepath:

FROM ubuntu:latest AS ui-builder

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends nodejs npm git

RUN git clone https://github.com/BerriAI/litellm.git .

ENV UI_BASE_PATH="/litellm/ui"

WORKDIR /app/ui/litellm-dashboard
RUN npm install
RUN UI_BASE_PATH=$UI_BASE_PATH npm run build

RUN mkdir -p /app/out
RUN rm -rf /app/out/* && mv ./out/* /app/out/

FROM ghcr.io/berriai/litellm:main-latest

WORKDIR /app

RUN mkdir -p /app/litellm/proxy/_experimental/out

COPY --from=ui-builder /app/out /app/litellm/proxy/_experimental/out

RUN chmod +x ./docker/entrypoint.sh

EXPOSE 8080/tcp

CMD ["--port", "8080", "--config", "/mnt/config/config.yaml"]

I deployed this in Google Cloud Run with the environment variable SERVER_ROOT_PATH set to /litellm. Let me know if I did anything wrong or what I can try out.

[DanielOtter]

@krrishdholakia Can this be reopened or should a formulate a new issue?

[DanielOtter]

Btw. I noticed in your video that the /sso does not seem to use the base path aswell. I further investigated this and /login does not go to the new base path aswell. This is a problem for my setup aswell since only /litellm traffic will reach the container in my setup. I tried rerouting all traffic to the litellm container but that does not work aswell. I am out of ideas.

[Theoreticallyhugo]

Bumping this because I'm running into the same issue. the custom base path mostly works, but sso is expected at root.

We're trying to run litellm behind a proxy, and its supposed to only use www.someurl.com/litellm/... which i don't really see how to realise with the sso.

[krrishdholakia]

I was able to get a good repro of this by running this script (run litellm on a custom route, and run something else on the base route), and have a PR out to fix this - https://github.com/BerriAI/litellm/pull/11337

import os
import sys

import uvicorn
from dotenv import load_dotenv
from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse

load_dotenv()
sys.path.insert(
    0, os.path.abspath("../../..")
)  # Adds the parent directory to the system path

from litellm.proxy.proxy_server import app as litellm_app
from litellm.proxy.proxy_server import proxy_startup_event

# Create main FastAPI app
app = FastAPI(title="Custom LiteLLM Server", lifespan=proxy_startup_event)

# Add CORS middleware
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Mount LiteLLM app at /litellm
app.mount("/litellm", litellm_app)


# Default route at /
@app.get("/")
async def root():
    return {"message": "Welcome to the API Gateway", "litellm_endpoint": "/litellm"}


# Health check endpoint
@app.get("/health")
async def health_check():
    return {"status": "healthy"}


if __name__ == "__main__":
    # Run the server on port 8000
    uvicorn.run(app, host="0.0.0.0", port=4000, log_level="info")

[DanielOtter]

Hi @krrishdholakia, I was finally able to test this and it nearly everything works now. There is just one tiny problem the ui login mask after login success routes to /litellm/litellm/ui/?login=success' instead of /litellm/ui/?login=success for the server root path of /litellm.

This means I can only see an error page after logging in. This is no big issue since the login itself works and calling /login/ui routes me to the ui but it would be nice to have this fixed.

[HuiJun]

I had that issue also and I was able to resolve by setting PROXY_BASE_URL without the SERVER_ROOT_PATH. https://docs.litellm.ai/docs/proxy/admin_ui_sso#step-3-set-proxy_base_url-in-your-env