server
server copied to clipboard
Published
20 hours ago •
triton-inference-server
triton-inference-server
Failed when building for Windows 10
Description
Hi there,
I got an error while building triton server for Windows 10, according to the instructions. I have no idea how to fix this. Help Please!
-- Build files have been written to: C:/tmp/tritonbuild/onnxruntime/build
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
12>C:\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(238,5): error MSB8066: Custom build for 'C:\tmp\tritonbuil
d\onnxruntime\build\CMakeFiles\1391fbda87be57075fb5bba7a38c2954\onnxruntime.rule;C:\tmp\tritonbuild\onnxruntime\build\CMakeFil
es\c0b7ec8ce4dc22ca22ac8622f7a49e15\ort_target.rule' exited with code 1. [C:\tmp\tritonbuild\onnxruntime\build\ort_target.vcxp
roj]
exited with status code 1
exited with status code 1
error: build failed
The detailed message and reporduce steps are shown below
Triton Information triton version: r22.05 repo branch: r22.05 cuda: 11.7 cudnn: 8.4.1.50 tensorrt: 8.4.1.5
To Reproduce
- Git clone this repo
- Git checkout r22.05
- Modify Dockerfile.win10.min as following
# Copyright 2021-2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# Windows min container for Triton build
ARG BASE_IMAGE=mcr.microsoft.com/dotnet/framework/sdk:4.8
FROM ${BASE_IMAGE}
SHELL ["cmd", "/S", "/C"]
RUN mkdir c:\tmp
WORKDIR /tmp
RUN powershell.exe [Net.ServicePointManager]::Expect100Continue = $true; [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls, [Net.SecurityProtocolType]::Tls11, [Net.SecurityProtocolType]::Tls12, [Net.SecurityProtocolType]::Ssl3; Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1')
RUN @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
RUN choco install git docker unzip -y
# Be aware that pip can interact badly with VS cmd shell so need to pip install before
# vsdevcmd.bat (see https://bugs.python.org/issue38989)
ADD https://www.python.org/ftp/python/3.8.10/python-3.8.10-amd64.exe python-3.8.10-amd64.exe
RUN python-3.8.10-amd64.exe /quiet InstallAllUsers=1 PrependPath=1 Include_doc=0
RUN mklink "c:\Program Files\Python38\python3.exe" "c:\Program Files\Python38\python.exe"
RUN pip install --upgrade wheel setuptools docker
RUN pip install grpcio-tools
# Download and install Build Tools for Visual Studio. The use of
# powershell for the install seems to be required to make the command
# wait for the install to complete before continuing. To avoid failures
# caused by VS regressions we want to stick with a working
# compiler. Currently this is 16.8.5. This page contains download
# links for buildtools.
# https://docs.microsoft.com/en-us/visualstudio/releases/2019/history#release-dates-and-build-numbers
ADD https://download.visualstudio.microsoft.com/download/pr/20130c62-1bc8-43d6-b4f0-c20bb7c79113/145a319d79a83376915d8f855605e152ef5f6fa2b2f1d2dca411fb03722eea72/vs_BuildTools.exe vs_buildtools.exe
ARG VS_INSTALL_PATH_WP="C:\BuildTools"
RUN powershell.exe Start-Process -FilePath vs_buildtools.exe -ArgumentList "--wait","--quiet","--norestart","--nocache","--installPath","%VS_INSTALL_PATH_WP%","--channelUri","C:\tmp\doesnotexist.chman","--add","Microsoft.VisualStudio.Workload.VCTools", "--includeRecommended","--add","Microsoft.Component.MSBuild" -Wait -PassThru
WORKDIR /
RUN git clone --single-branch --depth=1 -b 2022.04.12 https://github.com/microsoft/vcpkg.git
WORKDIR /vcpkg
RUN bootstrap-vcpkg.bat
RUN vcpkg.exe update
RUN vcpkg.exe install openssl:x64-windows openssl-windows:x64-windows rapidjson:x64-windows re2:x64-windows boost-interprocess:x64-windows boost-stacktrace:x64-windows zlib:x64-windows pthread:x64-windows b64:x64-windows
RUN vcpkg.exe integrate install
# Install a recent version of cmake. The version installed with VS is
# older and specifically we need more recent version to avoid
# https://gitlab.kitware.com/cmake/cmake/-/issues/21492.
ARG CMAKE_VERSION=3.21.2
ARG CMAKE_FILE=cmake-${CMAKE_VERSION}-windows-x86_64
WORKDIR /
ADD https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/${CMAKE_FILE}.zip ${CMAKE_FILE}.zip
RUN unzip %CMAKE_FILE%.zip
RUN move %CMAKE_FILE% CMake
RUN setx PATH "c:\CMake\bin;%PATH%"
ARG CUDA_MAJOR=11
ARG CUDA_MINOR=7
ARG CUDA_PATCH=0
ARG CUDA_VERSION=${CUDA_MAJOR}.${CUDA_MINOR}.${CUDA_PATCH}
ARG CUDA_PACKAGES="nvcc_${CUDA_MAJOR}.${CUDA_MINOR} \
cudart_${CUDA_MAJOR}.${CUDA_MINOR} \
nvml_dev_${CUDA_MAJOR}.${CUDA_MINOR} \
cublas_${CUDA_MAJOR}.${CUDA_MINOR} cublas_dev_${CUDA_MAJOR}.${CUDA_MINOR} \
cufft_${CUDA_MAJOR}.${CUDA_MINOR} cufft_dev_${CUDA_MAJOR}.${CUDA_MINOR} \
curand_${CUDA_MAJOR}.${CUDA_MINOR} curand_dev_${CUDA_MAJOR}.${CUDA_MINOR} \
cusolver_${CUDA_MAJOR}.${CUDA_MINOR} cusolver_dev_${CUDA_MAJOR}.${CUDA_MINOR} \
cusparse_${CUDA_MAJOR}.${CUDA_MINOR} cusparse_dev_${CUDA_MAJOR}.${CUDA_MINOR} \
thrust_${CUDA_MAJOR}.${CUDA_MINOR} \
visual_studio_integration_${CUDA_MAJOR}.${CUDA_MINOR}"
ARG CUDA_INSTALL_ROOT_WP="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v${CUDA_MAJOR}.${CUDA_MINOR}"
ARG TENSORRT_VERSION=8.4.1.5
ARG TENSORRT_ZIP=TensorRT-${TENSORRT_VERSION}.Windows10.x86_64.cuda-11.6.cudnn8.4.zip
ARG CUDNN_VERSION=8.4.1.50
ARG CUDNN_DIR=cudnn-windows-x86_64-${CUDNN_VERSION}_cuda11.6-archive
ARG CUDNN_ZIP=${CUDNN_DIR}.zip
WORKDIR /tmp
ADD https://developer.download.nvidia.com/compute/cuda/${CUDA_VERSION}/network_installers/cuda_${CUDA_VERSION}_windows_network.exe cuda_${CUDA_VERSION}_windows_network.exe
COPY ${CUDNN_ZIP} .
COPY ${TENSORRT_ZIP} .
WORKDIR /
RUN unzip /tmp/%TENSORRT_ZIP%
RUN move TensorRT-%TENSORRT_VERSION% TensorRT
ENV TRT_VERSION ${TENSORRT_VERSION}
WORKDIR /tmp
#RUN cuda_%CUDA_VERSION%_windows_network.exe -s %CUDA_PACKAGES%
RUN echo "'-s %CUDA_PACKAGES%'"
RUN powershell.exe Start-Process -FilePath ".\cuda_%CUDA_VERSION%_windows_network.exe" "'-s %CUDA_PACKAGES%'" -Wait -NoNewWindow
WORKDIR /
RUN unzip /tmp/%CUDNN_ZIP%
RUN move %CUDNN_DIR% cudnn
RUN copy cudnn\bin\cudnn*.dll "%CUDA_INSTALL_ROOT_WP%\bin\."
RUN copy cudnn\lib\cudnn*.lib "%CUDA_INSTALL_ROOT_WP%\lib\x64\."
RUN copy cudnn\include\cudnn*.h "%CUDA_INSTALL_ROOT_WP%\include\."
# Copy the CUDA visualstudio integration from where it was installed
# into the appropriate place in BuildTools
RUN copy "%CUDA_INSTALL_ROOT_WP%\extras\visual_studio_integration\MSBuildExtensions\*" "%VS_INSTALL_PATH_WP%\MSBuild\Microsoft\VC\v160\BuildCustomizations"
RUN setx PATH "%PATH%;c:\TensorRT\lib;%CUDA_INSTALL_ROOT_WP%\bin"
# It is important that the entrypoint initialize VisualStudio
# environment otherwise the build will fail. Also set
# CMAKE_TOOLCHAIN_FILE and VCPKG_TARGET_TRIPLET so
# that cmake can find the packages installed by vcpkg.
ENV CMAKE_TOOLCHAIN_FILE /vcpkg/scripts/buildsystems/vcpkg.cmake
ENV VCPKG_TARGET_TRIPLET x64-windows
ENTRYPOINT C:\BuildTools\VC\Auxiliary\Build\vcvars64.bat &&
- Build Windows 10 "Min" Image according to (this link)
- Done building win10-py3-in
- Build Triton Server
python build.py --no-container-pull --image=base,win10-py3-min --enable-logging --enable-stats --enable-tracing --enable-gpu --endpoint=grpc --endpoint=http --repo-tag=common:r22.05 --repo-tag=core:r22.05 --repo-tag=backend:r22.05 --repo-tag=thirdparty:r22.05 --backend=ensemble --backend=tensorrt:r22.05 --backend=onnxruntime:r22.05 --backend=openvino:r22.05
- Build failed and the error messages:
Building Triton Inference Server
platform windows
machine amd64
version 2.22.0
build dir D:\Code\server\build
install dir None
cmake dir None
default repo-tag: r22.05
container version 22.05
upstream container version 22.05
endpoint "grpc"
endpoint "http"
backend "ensemble" at tag/branch "r22.05"
backend "tensorrt" at tag/branch "r22.05"
backend "onnxruntime" at tag/branch "r22.05"
backend "openvino" at tag/branch "r22.05"
image "base": "win10-py3-min"
component "common" at tag/branch "r22.05"
component "core" at tag/branch "r22.05"
component "backend" at tag/branch "r22.05"
component "thirdparty" at tag/branch "r22.05"
Windows PowerShell
Sending build context to Docker daemon 1.526GB
Step 1/12 : ARG TRITON_VERSION=2.22.0
Step 2/12 : ARG TRITON_CONTAINER_VERSION=22.05
Step 3/12 : ARG BASE_IMAGE=win10-py3-min
Step 4/12 : FROM ${BASE_IMAGE}
---> 897ef96685ec
Step 5/12 : ARG TRITON_VERSION
---> Using cache
---> 5c2bb1117e0c
Step 6/12 : ARG TRITON_CONTAINER_VERSION
---> Using cache
---> ce905a22b4d2
Step 7/12 : SHELL ["cmd", "/S", "/C"]
---> Using cache
---> 3f36892da77c
Step 8/12 : ENV TRITON_SERVER_VERSION ${TRITON_VERSION}
---> Running in 38ee5c11799c
Removing intermediate container 38ee5c11799c
---> fa6ee4932f3c
Step 9/12 : ENV NVIDIA_TRITON_SERVER_VERSION ${TRITON_CONTAINER_VERSION}
---> Running in 97e49b27578d
Removing intermediate container 97e49b27578d
---> f0ac93b9ea32
Step 10/12 : WORKDIR /workspace
---> Running in e2f3dcf86056
Removing intermediate container e2f3dcf86056
---> 3f6f445f0be8
Step 11/12 : RUN rmdir /S/Q * || exit 0
---> Running in 24d38b930e82
The filename, directory name, or volume label syntax is incorrect.
Removing intermediate container 24d38b930e82
---> 64298e990e31
Step 12/12 : COPY . .
---> 8bb724fcd115
Successfully built 8bb724fcd115
Successfully tagged tritonserver_buildbase:latest
Error: No such container: tritonserver_builder
**********************************************************************
** Visual Studio 2019 Developer Command Prompt v16.8.5
** Copyright (c) 2020 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
Try the new cross-platform PowerShell https://aka.ms/pscore6
Directory: C:\tmp\tritonbuild\tritonserver
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/28/2022 3:15 PM build
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19042.
-- The C compiler identification is MSVC 19.28.29337.0
-- The CXX compiler identification is MSVC 19.28.29337.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Using MSVC as compiler, default target on Windows 10. If the target system is not Windows 10, please update _WIN32_WINNT to corresponding value.
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
DALI_VERSION
TRT_VERSION
-- Build files have been written to: C:/tmp/tritonbuild/tritonserver/build
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
Directory: C:\tmp\tritonbuild\install
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/28/2022 4:19 PM bin
Directory: C:\tmp\tritonbuild\install\include
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/28/2022 4:19 PM triton
Directory: C:\tmp
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/28/2022 4:19 PM tritonbuild
Cloning into 'tensorrt'...
remote: Enumerating objects: 23, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (22/22), done.
Receiving objects: 100% (23/23), 52.45 KiB | 596.00 KiB/s, done.ceiving objects: 82% (19/23)
Resolving deltas: 100% (13/13), done.
Directory: C:\tmp\tritonbuild\tensorrt
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/28/2022 4:19 PM build
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19042.
-- The C compiler identification is MSVC 19.28.29337.0
-- The CXX compiler identification is MSVC 19.28.29337.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- RapidJSON found. Headers: /vcpkg/installed/x64-windows/share/rapidjson/../../include
-- RapidJSON found. Headers: /vcpkg/installed/x64-windows/share/rapidjson/../../include
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
Using MSVC as compiler, default target on Windows 10. If the target system is not Windows 10, please update _WIN32_WINNT to corresponding value.
Using MSVC as compiler, default target on Windows 10. If the target system is not Windows 10, please update _WIN32_WINNT to corresponding value.
-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/include (found version "11.7.64")
-- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7 (found version "11.7")
-- Using CUDA 11.7
Using MSVC as compiler, default target on Windows 10. If the target system is not Windows 10, please update _WIN32_WINNT to corresponding value.
-- Using CUDA 11.7
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
DALI_VERSION
TRITON_ENABLE_METRICS
TRT_VERSION
-- Build files have been written to: C:/tmp/tritonbuild/tensorrt/build
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
Directory: C:\tmp\tritonbuild\install
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/28/2022 4:23 PM backends
Directory: C:\tmp
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/28/2022 4:19 PM tritonbuild
Cloning into 'onnxruntime'...
remote: Enumerating objects: 25, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (23/23), done.
Receiving objects: 100% (25/25), 53.79 KiB | 1.19 MiB/s, done.
Resolving deltas: 100% (5/5), done.16 (delta 0), pack-reused 0Resolving deltas: 100% (5/5)
Directory: C:\tmp\tritonbuild\onnxruntime
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 6/28/2022 4:23 PM build
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19042.
-- The C compiler identification is MSVC 19.28.29337.0
-- The CXX compiler identification is MSVC 19.28.29337.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using Onnxruntime docker: ON
-- RapidJSON found. Headers: /vcpkg/installed/x64-windows/share/rapidjson/../../include
-- RapidJSON found. Headers: /vcpkg/installed/x64-windows/share/rapidjson/../../include
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
Using MSVC as compiler, default target on Windows 10. If the target system is not Windows 10, please update _WIN32_WINNT to corresponding value.
Using MSVC as compiler, default target on Windows 10. If the target system is not Windows 10, please update _WIN32_WINNT to corresponding value.
-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/include (found version "11.7.64")
-- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7 (found version "11.7")
-- Using CUDA 11.7
Using MSVC as compiler, default target on Windows 10. If the target system is not Windows 10, please update _WIN32_WINNT to corresponding value.
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
DALI_VERSION
TRITON_ENABLE_METRICS
-- Build files have been written to: C:/tmp/tritonbuild/onnxruntime/build
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
12>C:\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(238,5): error MSB8066: Custom build for 'C:\tmp\tritonbuil
d\onnxruntime\build\CMakeFiles\1391fbda87be57075fb5bba7a38c2954\onnxruntime.rule;C:\tmp\tritonbuild\onnxruntime\build\CMakeFil
es\c0b7ec8ce4dc22ca22ac8622f7a49e15\ort_target.rule' exited with code 1. [C:\tmp\tritonbuild\onnxruntime\build\ort_target.vcxp
roj]
exited with status code 1
exited with status code 1
error: build failed
Expected behavior Triton Server is built successfully
Unfortunately there is some limitations int terms of CUDA + ONNXRUNTIME compatibility. Best way to describe it is simply look into ONNXRUNTIME documentation: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html
We can build base container for Windows using version CUDNN_VERSION=8.4.1.50, TENSORRT_VERSION=8.4.1.5 and CUDA_VERSION=11.7.0 but ONNXRUNTIME has not added support for it yet, that's why Triton Server windows container creation is not confirmed.
Closing issue due to lack of activity. Please re-open the issue if you would like to follow up with this.
