conda: add _GLIBCXX_USE_CXX11_ABI=1 support for linux cpu wheel

[zhuhong61]

The target of this commit is to add the support for a new linux cpu conda wheel file built with _GLIBCXX_USE_CXX11_ABI=1.

we add a Dockerfile (conda/Dockerfile_cxx11-abi) with CentOS8 as base image to support CXX11_ABI=1.

To build the new docker image with CXX11_ABI support, run: CUDA_VERSION=cpu-cxx11-abi conda/build_docker.sh or conda/build_all_docker.sh

To build a linux cpu conda wheel with CXX11_ABI within this image, run: // the below settings are special for this image export DESIRED_CUDA=cpu-cxx11-abi # change from cpu for wheel name export GPU_ARCH_TYPE=cpu-cxx11-abi # change from cpu for build.sh export DOCKER_IMAGE=pytorch/conda-builder:cpu-cxx11-abi export DESIRED_DEVTOOLSET=cxx11-abi

// the below settings are as usual export BINARY_ENV_FILE=/tmp/env export BUILDER_ROOT=/builder export DESIRED_PYTHON=3.7 # or 3.8, 3.9, etc. export IS_GHA=1 export PACKAGE_TYPE=conda export PYTORCH_FINAL_PACKAGE_DIR=/artifacts export PYTORCH_ROOT=/pytorch export GITHUB_WORKSPACE=/your_path_to_workspace

// the '-e DESIRED_DEVTOOLSET' below is newly added for this container, // others are as usual set -x mkdir -p artifacts/ container_name=$(docker run -e BINARY_ENV_FILE -e BUILDER_ROOT -e DESIRED_CUDA -e DESIRED_PYTHON -e GPU_ARCH_TYPE -e IS_GHA -e PACKAGE_TYPE -e PYTORCH_FINAL_PACKAGE_DIR -e PYTORCH_ROOT -e DOCKER_IMAGE -e DESIRED_DEVTOOLSET --tty --detach -v "${GITHUB_WORKSPACE}/pytorch:/pytorch" -v "${GITHUB_WORKSPACE}/builder:/builder" -v "${RUNNER_TEMP}/artifacts:/artifacts" -w / "${DOCKER_IMAGE}" )

// build conda wheel as usual, // and the built wheel file name looks like: pytorch-1.13.0.dev20220616-py3.7_cpu_0.tar.bz2 docker exec -t -w "${PYTORCH_ROOT}" "${container_name}" bash -c "bash .circleci/scripts/binary_populate_env.sh" docker exec -t "${container_name}" bash -c "source ${BINARY_ENV_FILE} && bash /builder/conda/build.sh"

// to verify the built wheel file, we'll see 'True' $ conda install pytorch-1.13.0.dev20220616-py3.7_cpu_0.tar.bz2 $ python -c 'import torch; print(torch._C._GLIBCXX_USE_CXX11_ABI)' True

Co-authored-by: Zhu Hong [email protected] Co-authored-by: Guo Yejun [email protected]

[gujinghui]

@atalman @malfet This is new copy of https://github.com/pytorch/builder/pull/1062 We cannot reproduce this error. Could you help give more info? Thanks.

[malfet]

The right way of doing it is to create a PR against pytorch/pytorch that uses the package and validate that it finishes correctly, here are the few example: https://github.com/pytorch/builder/pull/1012 (see test PR https://github.com/pytorch/pytorch/pull/74141 )

[zhuhong61]

The right way of doing it is to create a PR against pytorch/pytorch that uses the package and validate that it finishes correctly, here are the few example: #1012 (see test PR pytorch/pytorch#74141 )

Hi @malfet We submitted this PR https://github.com/pytorch/pytorch/pull/83253 to validate the conda wheel. Could you please help check the ci issues? https://github.com/pytorch/pytorch/pull/83253#discussion_r953366262 Thanks!

[zhuhong61]

Hi @malfet Could you help check ci failure issues in https://github.com/pytorch/pytorch/pull/83253, thanks!