incubator-gluten
incubator-gluten copied to clipboard
Use environment variables to specify the local package path in CI
Description
Due to network issues, packages like Arrow and Thrift often fail to download. Since https://github.com/facebookincubator/velox/pull/4325 already supports specifying the local package path through environment variables, the corresponding packages can be downloaded into the Docker image and used locally through environment variables. This approach can help avoid CI failures caused by unsuccessful package downloads.
For example.
export ARROW_THRIFT_URL=/mnt/disk1/thirdparty/thrift-0.16.0.tar.gz
export ARROW_GTEST_URL=/mnt/disk1/thirdparty/googletest-release-1.11.0.tar.gz
export ARROW_RAPIDJSON_URL=/mnt/disk1/thirdparty/rapidjson-232389d4f1012dddec4ef84861face2d2ba85709.tar.gz
export VELOX_CPR_URL=/mnt/disk1/thirdparty/cpr-1.10.5.tar.gz
export VELOX_ARROW_URL=/mnt/disk1/thirdparty/apache-arrow-15.0.0.tar.gz
cc @PHILO-HE @zhouyuan @Yohahaha.
@liujiayi771, thanks so much for your advice! We will do this improvement.