execution-spec-tests
execution-spec-tests copied to clipboard
feat(consume): refine the `--input` flag for fixture releases
Implement the comments and suggests from #745.
From https://github.com/ethereum/execution-spec-tests/pull/745#discussion_r1723225970, add the following:
I think we should use the Github API to determine which release is the latest and then define input_source to the URL that points to a specific release (i.e., a URL containing v[0-9]+.[0-9]+.[0-9]+). This has a few benefits:
- The pytest header will display the correct release version (this is a must imho, so that you can quickly see which release was actually being executed).
- We can display the exact release in the consume commands displayed in hiveview to help reproduce tests.
- The extracted fixtures will be extracted to the correct cache directory.
Another approach would be to grab the version from the .meta/fixtures.ini file, but I thinkI prefer resolving latest. If we're being very careful, we could check that the resolved version matches that of .meta/fixtures.ini, but it's a bit paranoid.
From https://github.com/ethereum/execution-spec-tests/pull/745#issuecomment-2288383673, consider adding the following:
- I was hoping a while ago to be able to add the release tag and fixture "tag" as input options.
So we could have --input=<release_tag|latest>-<fixture_tag>-release, for example:
--input=latest-stable-release
equivalent to--input=v3.0.0-stable-release
.[email protected]
for the EOF v1.0.7 fixtures. At this point we are may be going overkill! But at least if we include the former it covers all cases regarding releases.I do still like URLs especially if pointing to releases outside of the main repo but then... 2) we could add git user names to the input!