evaluate
evaluate copied to clipboard
add versioning the `HubEvaluationModuleFactory`
This PR couples the module repositories on the hub with the main repository in evaluate
, based on tags that are propagated from evaluate
to each space when they are pushed to the hub via actions (see evaluate/.github/hub/push_evaluations_to_hub.py
).
The logic is:
- if user provides
revision
, this is used. - else: there is no revision is provided then,
- if evaluate has a dev tag
main
is used - else version tag of evaluate is used (e.g.
v0.2.0
)
- if evaluate has a dev tag
This addresses the massive issues when merging #188. This will not solve the issues for existing versions but at least for newer versions this should not be an issue anymore.
The documentation is not available anymore as the PR was closed or merged.
Alternatively it can use main
by default, but if it contains a requirements.txt with an old version of evaluate
it can check the tags for release-*
?
(tbh I was thinking of something like that for datasets
)
This way we stay consistent with datasets and models: use main
by default + no need to make a release to update something
The issue with that is that we'll never be able to have a breaking change between the hub modules and the github repository without breaking every older version of the library.
this is what's happening with #188: since changed the module structure a bit all the older version of evaluate will try to load them.
If we use release tags release-*
we can resolve to the latest tag with a requirements.txt
that is compatible with the user's local version of evaluate
no ?