server
server copied to clipboard
triton-inference-server
--no-container-build not work when build with --backend=onnxruntime option
Description i build tritonserver with option --no-container-build and --backend=onnxruntime
python build.py --no-container-build --build-dir=`pwd`/build --enable-logging --enable-stats --enable-tracing --enable-gpu --endpoint=http --endpoint=grpc --backend=onnxruntime --backend=tensorrt --backend=ensemble
it will run with build/onnxruntime/build/Dockerfile.ort
-- Build files have been written to: /app/server/build/onnxruntime/build
[ 2%] Building CXX object _deps/repo-common-build/src/CMakeFiles/triton-common-async-work-queue.dir/async_work_queue.cc.o
[ 7%] Building CXX object _deps/repo-core-build/CMakeFiles/triton-core-serverstub.dir/src/tritonserver_stub.cc.o
[ 9%] Building CXX object _deps/repo-common-build/src/CMakeFiles/triton-common-async-work-queue.dir/thread_pool.cc.o
[ 9%] Building CXX object _deps/repo-common-build/src/CMakeFiles/triton-common-async-work-queue.dir/error.cc.o
[ 14%] Building CXX object _deps/repo-common-build/src/CMakeFiles/triton-common-error.dir/error.cc.o
[ 17%] Building CXX object _deps/repo-common-build/src/CMakeFiles/triton-common-logging.dir/logging.cc.o
[ 17%] Building CXX object _deps/repo-common-build/src/CMakeFiles/triton-common-thread-pool.dir/thread_pool.cc.o
[ 21%] Building CXX object _deps/repo-common-build/src/CMakeFiles/triton-common-table-printer.dir/table_printer.cc.o
[ 21%] Building CXX object _deps/googletest-build/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[ 24%] Building CUDA object _deps/repo-backend-build/CMakeFiles/kernel_library_new.dir/src/kernel.cu.o
[ 26%] Building ONNX Runtime
error: linux build requires --cudnn-home and --cuda-home
[ 29%] Linking CXX shared library libtritonserver.so
[+] Building 0.0s (0/1) docker:default
[ 31%] Linking CXX static library libtritoncommonerror.a
[ 31%] Built target triton-core-serverstub
[+] Building 0.3s (4/24) docker:default
=> [internal] load build definition from Dockerfile.ort 0.0s
=> => transferring dockerfile: 4.07kB 0.0s
=> [internal] load metadata for nvcr.io/nvidia/tritonserver:25.02-py3-min 0.0s
[+] Building 0.6s (4/24) docker:default
=> [internal] load build definition from Dockerfile.ort 0.0s
[+] Building 1.1s (4/24) docker:default
=> [internal] load build definition from Dockerfile.ort 0.0s
[+] Building 3.3s (4/24) docker:default
[+] Building 3.5s (4/24) docker:default
=> [internal] load build definition from Dockerfile.ort 0.0s
=> => transferring dockerfile: 4.07kB 0.0s
=> [internal] load metadata for nvcr.io/nvidia/tritonserver:25.02-py3-min 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> importing cache manifest from tritonserver_onnxruntime_cache0 3.5s
=> importing cache manifest from tritonserver_onnxruntime 3.5s
=> importing cache manifest from tritonserver_onnxruntime_cache1 3.5s
[+] Building 4.5s (9/24) docker:default
=> [internal] load build definition from Dockerfile.ort 0.0s
=> => transferring dockerfile: 4.07kB 0.0s
=> [internal] load metadata for nvcr.io/nvidia/tritonserver:25.02-py3-min 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> ERROR importing cache manifest from tritonserver_onnxruntime_cache0 4.0s
=> ERROR importing cache manifest from tritonserver_onnxruntime 4.0s
=> ERROR importing cache manifest from tritonserver_onnxruntime_cache1 3.8s
=> [ 1/18] FROM nvcr.io/nvidia/tritonserver:25.02-py3-min 0.0s
=> CACHED [ 2/18] WORKDIR /workspace 0.0s
=> ERROR [ 3/18] RUN ln -sf /opt/python/cp312* /opt/python/v 0.4s
------
> importing cache manifest from tritonserver_onnxruntime_cache0:
------
------
> importing cache manifest from tritonserver_onnxruntime:
------
------
> importing cache manifest from tritonserver_onnxruntime_cache1:
------
------
> [ 3/18] RUN ln -sf /opt/python/cp312* /opt/python/v:
0.234 ln: failed to create symbolic link '/opt/python/v': No such file or directory
Triton Information What version of Triton are you using?
r25.02
Are you using the Triton container or did you build it yourself?
i build it myself
To Reproduce Steps to reproduce the behavior.
build with attachment dockerfile
# docker build -t triton-server:r25.02 .
ARG _FROM=rockylinux/rockylinux:9.5
FROM ${_FROM} AS builder
RUN date -R \
&& yum install -y epel-release \
&& rm -rf /etc/yum.repos.d/epel-cisco-openh264.repo \
&& yum makecache \
&& date -R
RUN date \
&& yum install -y binutils bison make gcc gcc-c++ diffutils cmake \
&& yum -y install autoconf automake bzip2 libtool pkgconfig gettext \
&& yum -y install freetype-devel zlib-devel openssl-devel bzip2-devel ncurses-devel \
&& date -R
RUN date \
&& yum install -y wget \
&& date
ENV PATH=/usr/local/cuda/bin:$PATH
RUN date \
&& wget https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo \
-O /etc/yum.repos.d/cuda-rhel9.repo \
&& yum makecache \
&& yum install -y cuda-toolkit-12-8 cudnn9-cuda-12-8 libcudnn9-cuda-12 libnccl \
&& date
RUN date \
&& yum install -y tensorrt \
&& date
RUN date \
&& yum install -y python3-devel \
&& yum install -y rapidjson-devel \
# && yum install -y boost1.78-devel \
&& yum install -y systemd-devel \
&& yum install -y numactl-devel \
&& yum install -y re2-devel \
&& date
RUN date \
&& yum install -y which \
&& wget https://archives.boost.io/release/1.87.0/source/boost_1_87_0.tar.gz \
&& tar -xzf boost_1_87_0.tar.gz \
&& rm -rf boost_1_87_0.tar.gz \
&& cd boost_1_87_0 \
&& ./bootstrap.sh \
&& ./b2 install --with-filesystem --with-thread --with-interprocess \
&& cd .. \
&& date -R
RUN date \
&& wget -O /etc/yum.repos.d/docker-ce.repo \
https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo \
&& yum makecache \
&& yum install -y docker-ce-cli \
&& date -R
ARG PY_URL=https://mirrors.aliyun.com/pypi/simple
RUN date \
&& ln -sf /usr/bin/python3 /usr/bin/python \
&& yum install -y python3-pip \
&& pip config set global.index-url ${PY_URL} \
&& python3 -m pip install --upgrade pip setuptools wheel \
&& pip install distro requests build \
&& date
RUN yum clean packages
RUN date \
&& yum install -y git \
&& date
ARG TAG=r25.02
RUN date \
&& git clone -b $TAG --single-branch --recursive https://github.com/triton-inference-server/server.git \
&& date
RUN date \
&& yum install -y libb64-devel libb64-tools \
&& date
ARG _OPTS="\
--no-container-build \
--build-dir=`pwd`/build \
--enable-logging \
--enable-stats \
--enable-tracing \
--enable-gpu \
--endpoint=http \
--endpoint=grpc \
--backend=onnxruntime \
--backend=tensorrt \
--backend=ensemble \
"
RUN date \
&& cd server \
&& python build.py ${_OPTS} \
&& date
Expected behavior A clear and concise description of what you expected to happen.
i hope to build tritonserver without docker
I also found this problem when building with option --no-container-build. It seems that docker must be used when compiling some backends?
including onnxruntime、tensorflow、pytorch、tensorrtllm ...
