TensorFlow 2.7.0 No AVX, No GPU, Python 3.7, 3.8, 3.9, Ubuntu 18.04, multiple Archs

[Novaal]

Built on Ubuntu 18.04. Builds are not tested and provided as is.

Example configuration for Python 3.7 and westmere:

PYTHON_VERSION=python3.7
PYTHON_BIN_PATH=$(which $PYTHON_VERSION) \
PYTHON_LIB_PATH=$($PYTHON_VERSION -c "import pip; print(pip.__path__[0].rstrip('/pip'))") \
TF_NEED_CUDA=0 \
TF_NEED_ROCM=0 \
TF_DOWNLOAD_CLANG=0 \
CC_OPT_FLAGS="-march=westmere -Wno-sign-compare" \
TF_SET_ANDROID_WORKSPACE=0 \
TF_ENABLE_XLA=1 \
TF_NEED_OPENCL_SYCL=0 \
TF_NEED_MPI=0 \
./configure

This produces the following .tf_configure.bazelrc for Python 3.7.

build --action_env PYTHON_BIN_PATH="/usr/bin/python3.7"
build --action_env PYTHON_LIB_PATH="/usr/local/lib/python3.7/dist-packages"
build --python_path="/usr/bin/python3.7"
build --define=with_xla_support=true
build:opt --copt=-march=westmere
build:opt --host_copt=-march=westmere
build:opt --copt=-Wno-sign-compare
build:opt --host_copt=-Wno-sign-compare
test --flaky_test_attempts=3
test --test_size_filters=small,medium
test:v1 --test_tag_filters=-benchmark-test,-no_oss,-gpu,-oss_serial
test:v1 --build_tag_filters=-benchmark-test,-no_oss,-gpu
test:v2 --test_tag_filters=-benchmark-test,-no_oss,-gpu,-oss_serial,-v1only
test:v2 --build_tag_filters=-benchmark-test,-no_oss,-gpu,-v1only

GCC Compiler Option (-march)	Python 3.7	Python 3.8	Python 3.9	All Builds
barcelona	Download	Download	Download	Link
btver1	Download	Download	Download	Link
core2	Download	Download	Download	Link
nehalem	Download	Download	Download	Link
westmere	Download	Download	Download	Link

Install with:

pip install --ignore-installed --upgrade tensorflow-2.7.0-cp37-cp37m-linux_x86_64.whl

Please let me know here if it works for you.

[isaackogan]

New to Tensorflow, low level of Linux knowledge. Is there documentation I can read to help get this set up on my machine? I'm not sure which GCC Compiler Option I require.

Thanks!

[isaackogan]

Figured it out, nevermind!

[alexcoder04]

I'm on Devuan 4 (Debian 11), Python 3.9.2, AMD Athlon(tm) II X2 215 Processor; the architecture I would need is amdfam10, however barcelona seems to work fine. Thank you very much!

[kshurov]

Successfully ran build /core2/tensorflow-2.6.2-cp39-cp39-linux_x86_64.whl on my Synology 920+ NAS (Intel Celeron J4125 CPU) inside docker. This is the only noavx wheel that fits my needs I could find in last couple of days. @Novaal, you saved my month man!

[DrChrisLevy]

@isaackogan So how do you determine which GCC Compiler Option to use?

Im trying to install Tensorflow within a Docker image FROM --platform=linux/amd64 python:3.8-slim on a mac with mac m1 chip Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:41 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T6000 arm64.

[isaackogan]

@isaackogan So how do you determine which GCC Compiler Option to use?

Im trying to install Tensorflow within a Docker image FROM --platform=linux/amd64 python:3.8-slim on a mac with mac m1 chip Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:41 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T6000 arm64.

Trial and error, I did not (do not) have the foreknowledge to do it any other way. There were only 5 builds, so it was just a matter of wasting a bit of time on my end. Sorry I can't help further.

[Novaal]

@DrChrisLevy @isaackogan The GCC Compiler Option depends on the CPU you use. For example if you want to use Tensorflow with an Intel Xeon E5520 you have to choose nehalem.

The Intel Xeon E5520 is based on the Intel microarchitecture Nehalem (more infos on Wikipedia). For most of the microarchitectures GCC has its own compiler options to optimize the code for. In case of Tensorflow it is most likely necessary to build your own packages when the CPUs doesn't support AVX. You can find more about which GCC Compile Options are available here.

@DrChrisLevy Unfortunatly my builds are optimized for some x86 Processors and the new Apple M1 is an ARM-based CPU. As a comment here and here suggests it might be possible with barcelona. But it might be very slow.

[DrChrisLevy]

Thanks @Novaal ! Yes indeed slow. I have switched back to getting things running on aarch64 instead of going the emulation route. Thanks for your time and input.

[jdburgosr]

Im new using linux and this litterally saved me. Thanks <3

[Asarioglo]

@Novaal Do you think you can make similar builds for 2.9? I'm interested in Barcelona.

We have a 2.9 dependency and I'm trying to run the application in a container on a m1 machine but its not working. This solution worked with a sample application, but fails on the main app. I tried to build it locally with the parameters that you've specified, but running into errors. While I'm trying to get it to work, maybe you can get it faster?

[Novaal]

@Asarioglo Thanks for your interest in my builds. Unfortunatly i will not build any new wheels for new Tensorflow versions. But i hope i can give you some hints.

1. I'm using a docker images customized for the python version as "build-env". Here are the ones for v2.8.0: cp37.Dockerfile.txt, cp38.Dockerfile.txt, cp39.Dockerfile.txt, cp310.Dockerfile.txt
2. Starting the container and running the following script with the correct env vars (fitting to the "build-env") to build the wheel. I used this script in a CI-Pipeline, so it uses env vars. You can also easily customize it according to your needs. In your case $CI_ENVIRONMENT_NAME would be barcelona.

#!/bin/bash

# check if necessary envs are available
for i in PYTHON_VERSION PYTHON_VERSION_SHORT CI_ENVIRONMENT_NAME TF_VERSION
do
    if [ -z ${!i} ];
        then
            echo "variable '$i' is not set";
            exit 1;
    fi
done

cd /tensorflow_src

PYTHON_BIN_PATH=$(which $PYTHON_VERSION) \
PYTHON_LIB_PATH=$($PYTHON_VERSION -c "import pip; print(pip.__path__[0].rstrip('/pip'))") \
TF_NEED_CUDA=0 \
TF_NEED_ROCM=0 \
TF_DOWNLOAD_CLANG=0 \
CC_OPT_FLAGS="-march=$CI_ENVIRONMENT_NAME -Wno-sign-compare" \
TF_SET_ANDROID_WORKSPACE=0 \
TF_ENABLE_XLA=1 \
TF_NEED_OPENCL_SYCL=0 \
TF_NEED_MPI=0 \
./configure

cat .tf_configure.bazelrc

# build files, can take a while
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package &> log.txt

# create package
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /mnt

ls -la /mnt

Take in consideration, that one build can take some while. In my case each build took up to 10 hours. After that you should find the wheel in the docker container under /mnt. I don't know exactly what is necessary to build 2.9 onwards. It might be necessary to update bazel and to install some other dependencies.

[LobanovB]

Hi. I am trying to build the package to install tf for e500v2 (ppc853) -avx. But the problem is that I cant find the bazel for this arch. Could anyone help me?

[HarryCaveMan]

Can y'all please stop callilng this a QEMU/Rosetta bug? Their hands are sort of tied due to AVX being patented by intel. People still need to be able to test your software via docker emulation because they might be working on an M1 mac but running on intel chips in the cloud, and without the need for maintaining two sets of CI scripts.

[Abdansb]

@Novaal Thank you so much for your exceptional work Tested on: Intel Celeron N4000 Ubuntu 22.04 Python3.10 Tensorflow 2.8 Westmere

also with different version: Ubuntu 18.04 Python3.7 Tensorflow 2.6 Westmere