tensorflow io on Mac not working with s3 paths

[ankitvgupta]

I recently installed tensorflow io on my M1 Mac in an arm64 conda environment. the install worked fine, and I'm able to import tensorflow and tensorflow io, but for some reason it can't seem to find the s3 filesystem scheme. I'm getting the error:

tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 's3' not implemented (file: 's3://***')

(with the *** replaced by my S3 path). Also, when I import tensorflow_io, I get the following warning:

UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so

Any ideas for how to resolve this?

Here are the steps I used to install it in my conda environment - the install worked without error.

python setup.py -q bdist_wheel --project tensorflow_io_gcs_filesystem
pip install dist/tensorflow_io_gcs_filesystem-0.26.0-cp39-cp39-macosx_11_0_arm64.whl
python setup.py -q bdist_wheel
pip install --no-deps dist/tensorflow_io-0.26.0-cp39-cp39-macosx_11_0_arm64.whl

[dvaldivia]

You probably need to import tensorflow_io like

import tensorflow_io

[dvaldivia]

I take that back, upon import there's some errors reported regarding libraries not being loaded

/Users/dvaldivia/miniforge3/envs/mlpipeline/lib/python3.9/site-packages/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/Users/dvaldivia/miniforge3/envs/mlpipeline/lib/python3.9/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']
caused by: ["[Errno 2] The file to load file system plugin from does not exist.: '/Users/dvaldivia/miniforge3/envs/mlpipeline/lib/python3.9/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so'"]
  warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}")
/Users/dvaldivia/miniforge3/envs/mlpipeline/lib/python3.9/site-packages/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/Users/dvaldivia/miniforge3/envs/mlpipeline/lib/python3.9/site-packages/tensorflow_io/python/ops/libtensorflow_io.so']
caused by: ["dlopen(/Users/dvaldivia/miniforge3/envs/mlpipeline/lib/python3.9/site-packages/tensorflow_io/python/ops/libtensorflow_io.so, 0x0006): tried: '/Users/dvaldivia/miniforge3/envs/mlpipeline/lib/python3.9/site-packages/tensorflow_io/python/ops/libtensorflow_io.so' (no such file)"]
  warnings.warn(f"file system plugins are not loaded: {e}")

[sdharar]

I recently installed tensorflow io on my M1 Mac in an arm64 conda environment. the install worked fine, and I'm able to import tensorflow and tensorflow io, but for some reason it can't seem to find the s3 filesystem scheme. I'm getting the error:

tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 's3' not implemented (file: 's3://***')

(with the *** replaced by my S3 path). Also, when I import tensorflow_io, I get the following warning:

UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so

Any ideas for how to resolve this?

Here are the steps I used to install it in my conda environment - the install worked without error.

python setup.py -q bdist_wheel --project tensorflow_io_gcs_filesystem
pip install dist/tensorflow_io_gcs_filesystem-0.26.0-cp39-cp39-macosx_11_0_arm64.whl
python setup.py -q bdist_wheel
pip install --no-deps dist/tensorflow_io-0.26.0-cp39-cp39-macosx_11_0_arm64.whl

I'm facing the same problem. Were you able to find a solution to this? Thanks.

[sdharar]

@ankitvgupta, @dvaldivia - I'm not sure if this will help, but I don't see the warning about unable to load libtensorflow_io*.so if I just installtensorflow_io_gcs_filesystem (tensorflow_io_gcs_filesystem-0.26.0-cp39-cp39-macosx_11_0_arm64.whl) and skip installation of tensorflow-io (tensorflow_io-0.26.0-cp39-cp39-macosx_11_0_arm64.whl) as the initial installation includes *.so libraries needed for tensorflow-io.

(tensorflow) ---@--- ~ % sysctl -n machdep.cpu.brand_string
Apple M1 Max
(tensorflow) ---@--- ~ % python
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:33) 
[Clang 13.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>> import tensorflow as tf
>> import tensorflow_io as tfio
>> exit()
(tensorflow) ---@--- ~ % pip list | grep tensor
tensorboard                   2.10.1
tensorboard-data-server       0.6.1
tensorboard-plugin-wit        1.8.1
tensorflow-addons             0.18.0
tensorflow-estimator          2.10.0
tensorflow-io-gcs-filesystem  0.27.0
tensorflow-macos              2.10.0
tensorflow-metal              0.6.0
(tensorflow) ---@SEGReceLT063 ~ %