Once this PR is merged, we will be able to loosen the dependency requirements to allow sklearn>=0.24.0 and scipy>=1.6.0.
This issue serves as a reminder to do that and update tox.ini accordingly
tox.ini