Ubuntu workflow is unstable

[ishaileshpant]

Need to fix the ubuntu workflow that has been unstable for quite some time against most of the PRs https://github.com/securefederatedai/openfl/actions/workflows/ubuntu.yml