berry ci: benchmark latest npm

What's the problem this PR addresses?

We've pinned the major npm version benchmarked to npm@^7 back when the latest tag still pointed to npm@6. Today the latest tag points to npm 8, but we still benchmark npm 7.

How did you fix it?

I've unpinned the npm version used in the benchmark. I've also added some logging to be able to tell which version is used by the benchmark.

Checklist

[X] I have read the Contributing Guide.

[X] I have set the packages that need to be released for my changes to be effective.

[X] I will check that all automated PR checks pass before the PR gets reviewed.

Aug 14 '22 01:08 paul-soporan

Perhaps we should just use *, which will always pick the highest?

Aug 18 '22 09:08 arcanis

Just tested the behavior of * on all package managers and it seems that Yarn Modern is the only one that installs the latest version when * is used. All others, including classic, install the version pointed by the latest tag instead :thinking:.

(You can use @paul-soporan/test-package to test it. latest is set to 0.2.0 but the latest version is 1.0.0)

Aug 18 '22 14:08 paul-soporan

Yes, that's a know (and intended) behavior. Semver ranges aren't constrained by latest, as it would mean that matching semver ranges would depend on external factors. Given that the point of semver is to define whether a given version matches a given request, it's not something we implement.

Note that we make one exception for excluding deprecated packages - mostly because I didn't see a way out of this one. Still, I don't like it for the same reason even though, at least, a deprecated package will be unlikely to become undeprecated, so we can assume every user will get the same candidate versions or a subset (unlike this filtering on latest, which would allow a superset of the resolution).

Somewhat related ref: https://github.com/yarnpkg/berry/issues/2206#issuecomment-738072778

Aug 19 '22 08:08 arcanis