custom_federated_algorithms_2.ipynb - use of possibly outdated version of tff.framework.local_executor_factory function in the tutorial

[MKZuziak]

Describe the bug In the Custom Federated Algorithms, Part 2: Implementing Federated Averaging tutorial that is implemented in the Colab notebook colab we execute at the beginning the following code:

executor_factory = tff.framework.local_executor_factory( support_sequence_ops=True)

However, according to the latest documentation of TensorFlowFederated tff.framework.local_executor_factory does not have support_sequence_ops argument. If we try to run this function without passing support_sequence_ops=True, this amount to NotImplementedError: The intrinsic 'sequence_reduce' is not implemented. in the further part of the tutorial.

Environment (please complete the following information):

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 x64 (v.10.0)
• Python package versions (e.g., TensorFlow Federated, TensorFlow): tensorflow-federated 0.17.0
• Jupyter-client version: 7.3.0
• Ipykernel version: 6.13.0
• Python version: 3.8.13
• CUDA/cuDNN version: V.11.5.119

Expected behaviour According to the custom_federated_algorithms_2.ipynb invoking tff.framework.local_executor_factory function with support_sequence_ops=True should return an instance of executor_factory.ExecutorFactory encapsulating the executor construction logic specified above. However, in the current version of tensorflow-federated it returns TypeError: local_executor_factory() got an unexpected keyword argument 'support_sequence_ops'

Additional context According to the tutorial, invocation of executor_factory = tff.framework.local_executor_factory(support_sequence_ops=True) is necessary because we must use the Python context as it supports tff.sequence_* intrinsic. However, currently the function does not accept it as a valid argument. It leads to to NotImplementedError: intrinsic 'sequence_reduce' is not implemented. in the further part of the tutorial.

[zcharles8]

Hi @Scolpe. Thanks for the detailed information. As for your question, the tutorials are generally updated to reflect the latest version of TFF (in your case, I believe 0.23.0, though 0.24.0 was just released this week). As such, I wouldn't expect them to work in older versions of TFF. Notably, the latest version of TFF also requires python 3.9.

I haven't gone through this tutorial in a while though, so the tutorial may still be broken at the latest version. I'll try to go through it when I get a chance, but please let me know if you have any other findings on this matter.

[zcharles8]

Quick follow-up: The tutorial appears to be working at the latest github commit (ab932a8633f0ea4c9c95f46d7e06a58924560bae for posterity). If you run into issues of running it at the latest release, please let me know.

[dashupat30]

Hello @zcharles8,

I had a similar issue as described in the original post when I had an older Python version (3.7). After upgrading to later Python releases, I am getting problems installing TensorFlow-federated using a PIP installer. The errors depend on the Python versions. For example, for python3.9.13, the h5py setup fails for some reason I cannot understand.

I am using Windows 10 OS machine. I am a bit new to Python and ML world, so perhaps I am doing something wrong fundamentally.

LOG: INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C. Collecting tensorflow-federated Using cached tensorflow_federated-0.6.0-py2.py3-none-any.whl (256 kB) Using cached tensorflow_federated-0.5.0-py2.py3-none-any.whl (228 kB) Requirement already satisfied: grpcio<2.0,>=1.24.3 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (1.47.0) Requirement already satisfied: packaging in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (21.3) Requirement already satisfied: absl-py>=1.0.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (1.2.0) Requirement already satisfied: google-pasta>=0.1.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (0.2.0) Requirement already satisfied: libclang>=13.0.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (14.0.6) Requirement already satisfied: gast<=0.4.0,>=0.2.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (0.4.0) Requirement already satisfied: keras-nightly~=2.11.0.dev in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (2.11.0.dev2022080207) Requirement already satisfied: setuptools in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (58.1.0) Requirement already satisfied: keras-preprocessing>=1.1.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (1.1.2) Requirement already satisfied: typing-extensions>=3.6.6 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (4.3.0) Requirement already satisfied: termcolor>=1.1.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (1.1.0) Requirement already satisfied: tb-nightly~=2.10.0.a in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (2.10.0a20220803) Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (0.26.0) Requirement already satisfied: wrapt>=1.11.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (1.14.1) Requirement already satisfied: astunparse>=1.6.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tf-nightly->tensorflow-federated) (1.6.3) Requirement already satisfied: wheel<1.0,>=0.23.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from astunparse>=1.6.0->tf-nightly->tensorflow-federated) (0.37.1) Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (1.8.1) Requirement already satisfied: requests<3,>=2.21.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (2.28.1) Requirement already satisfied: google-auth<3,>=1.6.3 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (2.9.1) Requirement already satisfied: werkzeug>=1.0.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (2.2.1) Requirement already satisfied: markdown>=2.6.8 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (3.4.1) Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (0.4.6) Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (0.6.1) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from packaging->tf-nightly->tensorflow-federated) (3.0.9) Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from google-auth<3,>=1.6.3->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (0.2.8) Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from google-auth<3,>=1.6.3->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (5.2.0) Requirement already satisfied: rsa<5,>=3.1.4 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from google-auth<3,>=1.6.3->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (4.9) Requirement already satisfied: requests-oauthlib>=0.7.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (1.3.1) Requirement already satisfied: importlib-metadata>=4.4 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from markdown>=2.6.8->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (4.12.0) Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from requests<3,>=2.21.0->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (1.26.11) Requirement already satisfied: idna<4,>=2.5 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from requests<3,>=2.21.0->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (3.3) Requirement already satisfied: certifi>=2017.4.17 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from requests<3,>=2.21.0->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (2022.6.15) Requirement already satisfied: charset-normalizer<3,>=2 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from requests<3,>=2.21.0->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (2.1.0) Requirement already satisfied: MarkupSafe>=2.1.1 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from werkzeug>=1.0.1->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (2.1.1) Requirement already satisfied: zipp>=0.5 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (3.8.1) Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (0.4.8) Requirement already satisfied: oauthlib>=3.0.0 in c:\users\x\appdata\local\programs\python\python39\lib\site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tb-nightly~=2.10.0.a->tf-nightly->tensorflow-federated) (3.2.0) Building wheels for collected packages: h5py Building wheel for h5py (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [64 lines of output] running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.9 creating build\lib.win-amd64-3.9\h5py copying h5py\h5py_warnings.py -> build\lib.win-amd64-3.9\h5py copying h5py\highlevel.py -> build\lib.win-amd64-3.9\h5py copying h5py\ipy_completer.py -> build\lib.win-amd64-3.9\h5py copying h5py\version.py -> build\lib.win-amd64-3.9\h5py copying h5py_init_.py -> build\lib.win-amd64-3.9\h5py creating build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\attrs.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\base.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\compat.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\dataset.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\datatype.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\dims.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\files.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\filters.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\group.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\selections.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\selections2.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\vds.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl_init_.py -> build\lib.win-amd64-3.9\h5py_hl creating build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\common.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_attribute_create.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_attrs.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_attrs_data.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_base.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_completions.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dataset.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dataset_getitem.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dataset_swmr.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_datatype.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_deprecation.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dimension_scales.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dims_dimensionproxy.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dtype.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_file.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_file2.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_file_image.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_filters.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_group.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5d_direct_chunk.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5f.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5p.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5pl.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5t.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_objects.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_selections.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_slicing.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_threads.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests_init_.py -> build\lib.win-amd64-3.9\h5py\tests creating build\lib.win-amd64-3.9\h5py\tests\test_vds copying h5py\tests\test_vds\test_highlevel_vds.py -> build\lib.win-amd64-3.9\h5py\tests\test_vds copying h5py\tests\test_vds\test_lowlevel_vds.py -> build\lib.win-amd64-3.9\h5py\tests\test_vds copying h5py\tests\test_vds\test_virtual_source.py -> build\lib.win-amd64-3.9\h5py\tests\test_vds copying h5py\tests\test_vds_init_.py -> build\lib.win-amd64-3.9\h5py\tests\test_vds running build_ext error: Could not find module 'hdf5.dll' (or one of its dependencies). Try using the full path with constructor syntax. Loading library to get version: hdf5.dll [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for h5py Running setup.py clean for h5py Failed to build h5py Installing collected packages: tf-estimator-nightly, h5py, tf-nightly, tensorflow-federated Attempting uninstall: tf-estimator-nightly Found existing installation: tf-estimator-nightly 2.10.0.dev2022072808 Uninstalling tf-estimator-nightly-2.10.0.dev2022072808: Successfully uninstalled tf-estimator-nightly-2.10.0.dev2022072808 Attempting uninstall: h5py Found existing installation: h5py 3.7.0 Uninstalling h5py-3.7.0: Successfully uninstalled h5py-3.7.0 Running setup.py install for h5py ... error error: subprocess-exited-with-error

× Running setup.py install for h5py did not run successfully. │ exit code: 1 ╰─> [64 lines of output] running install running build running build_py creating build creating build\lib.win-amd64-3.9 creating build\lib.win-amd64-3.9\h5py copying h5py\h5py_warnings.py -> build\lib.win-amd64-3.9\h5py copying h5py\highlevel.py -> build\lib.win-amd64-3.9\h5py copying h5py\ipy_completer.py -> build\lib.win-amd64-3.9\h5py copying h5py\version.py -> build\lib.win-amd64-3.9\h5py copying h5py_init_.py -> build\lib.win-amd64-3.9\h5py creating build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\attrs.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\base.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\compat.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\dataset.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\datatype.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\dims.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\files.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\filters.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\group.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\selections.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\selections2.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl\vds.py -> build\lib.win-amd64-3.9\h5py_hl copying h5py_hl_init_.py -> build\lib.win-amd64-3.9\h5py_hl creating build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\common.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_attribute_create.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_attrs.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_attrs_data.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_base.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_completions.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dataset.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dataset_getitem.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dataset_swmr.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_datatype.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_deprecation.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dimension_scales.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dims_dimensionproxy.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_dtype.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_file.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_file2.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_file_image.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_filters.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_group.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5d_direct_chunk.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5f.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5p.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5pl.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_h5t.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_objects.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_selections.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_slicing.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests\test_threads.py -> build\lib.win-amd64-3.9\h5py\tests copying h5py\tests_init_.py -> build\lib.win-amd64-3.9\h5py\tests creating build\lib.win-amd64-3.9\h5py\tests\test_vds copying h5py\tests\test_vds\test_highlevel_vds.py -> build\lib.win-amd64-3.9\h5py\tests\test_vds copying h5py\tests\test_vds\test_lowlevel_vds.py -> build\lib.win-amd64-3.9\h5py\tests\test_vds copying h5py\tests\test_vds\test_virtual_source.py -> build\lib.win-amd64-3.9\h5py\tests\test_vds copying h5py\tests\test_vds_init_.py -> build\lib.win-amd64-3.9\h5py\tests\test_vds running build_ext error: Could not find module 'hdf5.dll' (or one of its dependencies). Try using the full path with constructor syntax. Loading library to get version: hdf5.dll [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. Rolling back uninstall of h5py Moving to c:\users\x\appdata\local\programs\python\python39\lib\site-packages\h5py-3.7.0.dist-info
from C:\Users\x\AppData\Local\Programs\Python\Python39\Lib\site-packages~5py-3.7.0.dist-info Moving to c:\users\x\appdata\local\programs\python\python39\lib\site-packages\h5py
from C:\Users\x\AppData\Local\Programs\Python\Python39\Lib\site-packages~5py error: legacy-install-failure

× Encountered error while trying to install package. ╰─> h5py

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure.

Regards,