[BUG] Issues with Cloudflare proxy - Support for RunPod

[kodxana]

🐛 Bug

Hi there! I'm working on getting native support h2o-llmstudio on RunPod containers. I was able to get it fully made into docker container that includes training downloads etc. Issue is that web UI is not being able to connect to UI when we use our cloudflare proxy

On the other hand if I setup direct connection using IP:PORT I can access web UI without issues. Any ideas what could be an issue here. Mayby its issue with some headers?

[kodxana]

# Use the specified base image
FROM nvidia/cuda:11.6.2-cudnn8-devel-ubuntu20.04
ENV DEBIAN_FRONTEND=noninteractive

# Install Python 3.10 and set it as the default
RUN apt-get update && \
    apt-get install -y software-properties-common && \
    add-apt-repository -y ppa:deadsnakes/ppa && \
    apt-get update && \
    apt-get install -y python3.10 python3.10-distutils python3.10-dev && \
    update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1 && \
    update-alternatives --install /usr/bin/python python /usr/bin/python3.10 1

# Install pip
RUN apt-get install -y wget git && \
    wget https://bootstrap.pypa.io/get-pip.py && \
    python3.10 get-pip.py && \
    rm get-pip.py

# Set the working directory
WORKDIR /app

# Clone the repository
RUN git clone https://github.com/h2oai/h2o-llmstudio.git /app

# Install required packages and dependencies
RUN pip install --upgrade pip \
    && pip install pipenv==2022.10.4 \
    && pipenv install --python 3.10 \
    && pipenv run pip install deps/h2o_wave-nightly-py3-none-manylinux1_x86_64.whl --force-reinstall

# Set environment variables for Wave and HF_HOME
ENV H2O_WAVE_MAX_REQUEST_SIZE=25MB \
    H2O_WAVE_NO_LOG=True \
    H2O_WAVE_PRIVATE_DIR="/download/@/app/output/download" \
    HF_HOME="/app/HF_CACHE"

# Create the HF_CACHE directory
RUN mkdir -p /app/HF_CACHE

# Expose the Wave server's port
EXPOSE 10101

# Start the Wave server
CMD ["pipenv", "run", "wave", "run", "app"]

Sharing also dockerfile I made :)

[psinger]

Hi @kodxana

If it is running fine by accessing it via IP:PORT, I assume there is an issue with your cloudfare proxy.

The proxy needs to allow websockets, given the error message, it appears this is not the case.

I would suggest to check the configuration for that.

[psinger]

Closing for now, please re-open if needed.