Bump Jax from 0.4.20 to 0.4.28, Torch from 2.2.1 to 2.3.0, Transformers from 4.39.3 to 4.40.2, Pandas from 2.2.1 to 2.2.2, Tqdm from 4.66.2 to 4.66.4, Matplotlib from 3.8.3 to 3.9.0, Uvicorn from 0.27.1 to 0.29.0, Fastapi from 0.110.0 to 0.111.0, Networkx from 3.2.1 to 3.3, and Typing_extensions from 4.10.0 to 4.11.0

[Smartappli]

Description

Please include a summary of the change, the motivation, and any additional context that will help others understand your PR. If it closes one or more open issues, please tag them as described here.

Affected Dependencies

List any dependencies that are required for this change.

How has this been tested?

• Describe the tests that you ran to verify your changes.
• Provide instructions so we can reproduce.
• List any relevant details for your test configuration.

Checklist

• [ ] I have followed the Contribution Guidelines and Code of Conduct
• [ ] I have commented my code following the OpenMined Styleguide
• [ ] I have labeled this PR with the relevant Type labels
• [ ] My changes are covered by tests

[madhavajay]

Hi @Smartappli thanks for the PRs. We actually have some manual testing in Google Colab, we need to do before we can bump some packages.

Sometimes these small lib changes cause incompatibility so I am hesitant to merge these without additional testing.

You can do this by running this in Google Colab and then importing the library and making sure things work:

!pip install 'git+https://github.com/YOURFORK/PySyft@YOUR_BRANCH#egg=syft&subdirectory=packages/syft'

Let me know how you go and report back. Same would apply to the other PRs. You could combine these into one and test it together.

https://github.com/OpenMined/PySyft/pull/8684 https://github.com/OpenMined/PySyft/pull/8688

[Smartappli]

Hi @Smartappli thanks for the PRs. We actually have some manual testing in Google Colab, we need to do before we can bump some packages.

Sometimes these small lib changes cause incompatibility so I am hesitant to merge these without additional testing.

You can do this by running this in Google Colab and then importing the library and making sure things work:

!pip install 'git+https://github.com/YOURFORK/PySyft@YOUR_BRANCH#egg=syft&subdirectory=packages/syft'

Let me know how you go and report back. Same would apply to the other PRs. You could combine these into one and test it together.

#8684 #8688

Hi @madhavajay

Thank you for your advising.

I combined all the PRs into one and tested the installation on google colab as requested. It installs without error :)

[madhavajay]

@Smartappli looks like the versions we install for [matplotlib,mpl_toolkits,numpy,psutil] are causing incompatibility requiring a session restart.

The best way to handle this is to figure out what versions Colab has before we install and then add them to a lower range in setup.cfg so that Colab is satisfied while people who install syft in a seperate fresh virtualenv elsewhere will get the latest versions.

Can you see if we can fix this?

[Smartappli]

@madhavajay done

[madhavajay]

@Smartappli great work. 🙌

I think the only problem now is that the new version of numpy has broken some tests:

E           syft.service.response.SyftAttributeError: 'APIModule' api.numpy object has no submodule or method 'zeros_like', you may not have permission to access the module you are trying to access.If you think this is an error, try calling `client.refresh()` to update the API.

If you want to revert numpy to an older version that doesn't have the problem we could address the numpy upgrade in a different PR.

[Smartappli]

@madhavajay work done :)

[Smartappli]

@madhavajay I have separated numpy into another PR as requested.

[Smartappli]

@madhavajay Why not use hypercorn instead of uvicorn to benefit from the double stack of ipv4 and ipv6?

[Smartappli]

Merged with https://github.com/OpenMined/PySyft/pull/8761

[yashgorana]

E AssertionError: Can not evaluate vstack(([0, 1, 1, 2, 2, 3], np.array([0, 1, 1, 5, 5, 3]))) with 'APIModule' apinumpy object has no submodule or method 'vstack', you may not have permission to access the module you are trying to access.If you think this is an error, try calling client.refresh() to update the API. 2024-05-14T05:12:41.8653971Z E assert SyftAttributeError("'APIModule' apinumpy object has no submodule or method 'vstack', you may not have permission to access the module you are trying to access.If you think this is an error, try calling client.refresh() to update the API.") == SyftAttributeError

Tests still failing. Probably numpy version compat issue that needs to be looked into.

Meanwhile, torch will be updated to 2.3.0 with #8605.

[Smartappli]

@madhavajay @yashgorana I propose to migrate numpy to version 2.0.0 which will be released in two weeks in a separate PR in order to close this PR. What do you think?

Related PR: https://github.com/OpenMined/PySyft/pull/8875

[madhavajay]

@Smartappli yes I saw the release announcement. My main concern would be if it does break compatibility, I wonder how long others will wait to upgrade, for example google colab. It wouldn't hurt for us to get to a newer numpy for now just to keep the deprecation window moving.