Building on linux with ppc64le arch

Open allegro0132 opened this issue 1 year ago • 0 comments

Thanks for your great job! Now JAX has become a platform for running machine learning and scientific computation in IBM ppc64le system(Power8, Power9). The pre-built jax and tensorflow wheels have been provided by Open-CE environment https://github.com/open-ce/open-ce/issues/549 . However, as the key of JAX ecosystem, tensorstore does not have ppc64le support yet.

I am working on building tensorstore in a Power9 system with Ubuntu 20.04, after patching some of .bazel files. The compiling successfully finished, and the built wheel seems working as expected.

My commit: https://github.com/allegro0132/tensorstore/commit/0cc3017d44bf2a411df2afdc5545b6bf144988c0

Stats

lscpu

Architecture:                    ppc64le
Byte Order:                      Little Endian
CPU(s):                          128
On-line CPU(s) list:             0-127
Thread(s) per core:              4
Core(s) per socket:              16
Socket(s):                       2
NUMA node(s):                    6
Model:                           2.3 (pvr 004e 1203)
Model name:                      POWER9, altivec supported
Frequency boost:                 enabled
CPU max MHz:                     3800.0000
CPU min MHz:                     2300.0000
L1d cache:                       1 MiB
L1i cache:                       1 MiB
L2 cache:                        8 MiB
L3 cache:                        160 MiB
NUMA node0 CPU(s):               0-63
NUMA node8 CPU(s):               64-127

gcc -v

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/powerpc64le-linux-gnu/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: powerpc64le-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 10.5.0-1ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=powerpc64le-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --with-libphobos-druntime-only=yes --enable-objc-gc=auto --enable-secureplt --with-cpu=power8 --enable-targets=powerpcle-linux --disable-multilib --enable-multiarch --disable-werror --with-long-double-128 --enable-offload-targets=nvptx-none=/build/gcc-10-7uyHQ3/gcc-10-10.5.0/debian/tmp-nvptx/usr --without-cuda-driver --enable-checking=release --build=powerpc64le-linux-gnu --host=powerpc64le-linux-gnu --target=powerpc64le-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-mutex
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.5.0 (Ubuntu 10.5.0-1ubuntu1~20.04)

python3

Python 3.9.5 (default, Nov 23 2021, 15:27:38) 
[GCC 9.3.0] on linux

Compile routine.

python3 -m venv ts-venv
source ts-venv/bin/activate
export BAZELISK_BASE_URL="https://ftp2.osuosl.org/pub/ppc64el/bazel"
python3 -m pip install -vv .

The selected output shows as following...

Building wheels for collected packages: tensorstore
  Created temporary directory: /tmp/pip-wheel-8htksuxq
  Destination directory: /tmp/pip-wheel-8htksuxq
  Running command Building wheel for tensorstore (pyproject.toml)
  WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
  running bdist_wheel
  running build
  running build_py
  creating /tmp/tmpj2o9y_8b/lib.linux-ppc64le-cpython-39
  creating /tmp/tmpj2o9y_8b/lib.linux-ppc64le-cpython-39/tensorstore
  copying python/tensorstore/__init__.py -> /tmp/tmpj2o9y_8b/lib.linux-ppc64le-cpython-39/tensorstore
  running build_ext
  /home/allegro/ts-venv/bin/python3 -u bazelisk.py build -c opt //python/tensorstore:_tensorstore__shared_objects --verbose_failures --action_env=PATH=/home/allegro/ts-venv/bin:/home/allegro/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin --copt=-fvisibility=hidden
  Downloading https://ftp2.osuosl.org/pub/ppc64el/bazel/ubuntu_20.04/bazel-6.4.0...
  Downloading https://ftp2.osuosl.org/pub/ppc64el/bazel/ubuntu_20.04/bazel-6.4.0.sha256...
  The Bazel mirror does not have a checksum file; skipping checksum verification.Extracting Bazel installation...
  Starting local Bazel server and connecting to it...
====================================
tons of gcc log
====================================
[4,430 / 4,435] Compiling tensorstore/kvstore/ocdbt/python/bindings.cc; 16s linux-sandbox ... (4 actions running)
  [4,433 / 4,435] Compiling tensorstore/kvstore/gcs_grpc/gcs_grpc.cc; 16s linux-sandbox
  [4,434 / 4,435] [Prepa] Linking python/tensorstore/_tensorstore.so
  [4,434 / 4,435] Linking python/tensorstore/_tensorstore.so; 1s linux-sandbox
  Target //python/tensorstore:_tensorstore__shared_objects up-to-date (nothing to build)
  INFO: Elapsed time: 161.770s, Critical Path: 109.84s
  INFO: 4435 processes: 763 internal, 3668 linux-sandbox, 4 local.
  INFO: Build completed successfully, 4435 total actions
  Copying extension bazel-bin/python/tensorstore/_tensorstore.so -> /tmp/tmpj2o9y_8b/lib.linux-ppc64le-cpython-39/tensorstore/_tensorstore.cpython-39-powerpc64le-linux-gnu.so
  installing to /tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel
  running install
  running install_lib
  creating /tmp/tmpj2o9y_8b/bdist.linux-ppc64le
  creating /tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel
  creating /tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel/tensorstore
  copying /tmp/tmpj2o9y_8b/lib.linux-ppc64le-cpython-39/tensorstore/__init__.py -> /tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel/tensorstore
  copying /tmp/tmpj2o9y_8b/lib.linux-ppc64le-cpython-39/tensorstore/_tensorstore.cpython-39-powerpc64le-linux-gnu.so -> /tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel/tensorstore
  running install_egg_info
  running egg_info
  writing python/tensorstore.egg-info/PKG-INFO
  writing dependency_links to python/tensorstore.egg-info/dependency_links.txt
  writing requirements to python/tensorstore.egg-info/requires.txt
  writing top-level names to python/tensorstore.egg-info/top_level.txt
  adding license file 'LICENSE'
  adding license file 'AUTHORS'
  writing manifest file 'python/tensorstore.egg-info/SOURCES.txt'
  Copying python/tensorstore.egg-info to /tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel/tensorstore-0.1.56.post1.dev3-py3.9.egg-info
  running install_scripts
  creating /tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel/tensorstore-0.1.56.post1.dev3.dist-info/WHEEL
  creating '/tmp/pip-wheel-8htksuxq/.tmp-kj4xkvpk/tensorstore-0.1.56.post1.dev3-cp39-cp39-linux_ppc64le.whl' and adding '/tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel' to it
  adding 'tensorstore/__init__.py'
  adding 'tensorstore/_tensorstore.cpython-39-powerpc64le-linux-gnu.so'
  adding 'tensorstore-0.1.56.post1.dev3.dist-info/AUTHORS'
  adding 'tensorstore-0.1.56.post1.dev3.dist-info/LICENSE'
  adding 'tensorstore-0.1.56.post1.dev3.dist-info/METADATA'
  adding 'tensorstore-0.1.56.post1.dev3.dist-info/WHEEL'
  adding 'tensorstore-0.1.56.post1.dev3.dist-info/top_level.txt'
  adding 'tensorstore-0.1.56.post1.dev3.dist-info/RECORD'
  removing /tmp/tmpj2o9y_8b/bdist.linux-ppc64le/wheel
  Building wheel for tensorstore (pyproject.toml) ... done
  Created wheel for tensorstore: filename=tensorstore-0.1.56.post1.dev3-cp39-cp39-linux_ppc64le.whl size=14660400 sha256=f9b1ff1246381a97848d5be7589037767c8b2c95ef232bea3960d1373f47e07e
  Stored in directory: /tmp/pip-ephem-wheel-cache-u69hmua8/wheels/f9/39/93/2990bae44095900a48e22fead957edb0728334c76e37fe588d
Successfully built tensorstore
Installing collected packages: ml-dtypes, tensorstore


Successfully installed ml-dtypes-0.3.2 tensorstore-0.1.56.post1.dev3
Remote version of pip: 24.0
Local version of pip:  24.0
Was pip installed by pip? True
Removed build tracker: '/tmp/pip-build-tracker-upzgoi2r'

Validation

I tested tensorstore by the examples in the tutorialhttps://google.github.io/tensorstore/python/tutorial.html

$ cat test_ts.py 
import tensorstore as ts
import numpy as np
dataset = ts.open({
    'driver': 'n5',
    'kvstore': {
        'driver': 'file',
        'path': 'tmp/dataset/',
    },
    'metadata': {
        'compression': {
            'type': 'gzip'
        },
        'dataType': 'uint32',
        'dimensions': [1000, 20000],
        'blockSize': [100, 100],
    },
    'create': True,
    'delete_existing': True,
}).result()
write_future = dataset[80:82, 99:102].write([[1, 2, 3], [4, 5, 6]])
write_future.result()
dataset[80:82, 99:102] = [[1, 2, 3], [4, 5, 6]]
print(dataset[80:83, 99:102].read().result())

$ python test_ts.py 
[[1 2 3]
 [4 5 6]
 [0 0 0]]

But when I ran python3 bazelisk.py test //..., Bazel raised errors to tell me @pypa_grpcio failed to be compiled.

I am not sure the modification of bazel is corrected or not, and did the tensorstore working as expected or not...

Mar 20 '24 09:03 allegro0132