pub icon indicating copy to clipboard operation
pub copied to clipboard

"dart pub libyear"

Open Hixie opened this issue 2 years ago • 10 comments

It would be interesting if pub could compute and present the libyear metric for a project.

A libyear is the sum of the durations between the release date of each dependency's selected version and latest version.

For example, if a project depends on foo 1.0 which was released in January 2021, while foo 2.0 was released in January 2022, then it has a dependency freshness of one libyear. If it also depends on bar 1.0 which was released in January 2021, while bar 2.0 was released in January 2023, then it has a dependency freshness of 3 libyears (1 for foo, plus 2 for bar).

By default this metric should only apply to direct dependencies.

Proposed additional flags:

  • --transitive: compute the sum over the transitive closure of all dependencies, not just direct dependencies.
  • --dev: include direct dev dependencies as well.
  • --releases: compute the metric just by counting the number of releases between the selected version and the latest version, rather than the release date delta.
  • --versions: compute the metric using the semver delta rather than release date delta (see Cox et al, 2015 for the algorithm).

Hixie avatar Jan 18 '23 22:01 Hixie

Not sure it really fits in the pub client - perhaps better in a separate package - but should be easy to compute from a pubspec.lock and api calls to pub.dev. It includes the "published" date for each version in the version listing.

That is not part of the version listing spec though - so it cannot be used for any pub server.

sigurdm avatar Jan 19 '23 09:01 sigurdm

Hmm - tried to cook something up: https://github.com/sigurdm/libyear

sigurdm avatar Jan 19 '23 14:01 sigurdm

haha, that's great!

I was thinking it'd be a pub command on the basis that it's a direct way to figure out how out-of-date your dependencies are. It seems similar to dart pub outdated except instead of outputting what can be upgraded, it outputs the metric of how far they are outdated, basically. Indeed now that I look at dart pub outdated I could see the libyear metric (or maybe all three metrics) being included in that output by default.

Hixie avatar Jan 19 '23 21:01 Hixie

it's a direct way to figure out how out-of-date your dependencies are.

Yeah - however these metrics

  • are somewhat arbitrary / non-canonical.
  • are non actionable - what should I do if I'm 100 years outdated?

I'm not sure pub wants to take this opinion. @jonasfj what do you think?

sigurdm avatar Jan 20 '23 14:01 sigurdm

We could potentially make dart pub outdated report the metric.

But it's an obvious candidate for use in pana. If nothing else the dependencies section of the score tab on pub.dev could have the number.

Highlighting it on pub.dev in the sidebar is not a bad idea either. Assuming we think this is a useful metric.

jonasfj avatar Jan 20 '23 19:01 jonasfj

I think it's a useless metric in the abstract. But I think it becomes very interesting as a relative metric. As @jonasfj says, it might be an interesting metric when comparing two packages, to see which one is most likely to be up to date. It might be a useful metric for app developers to keep an eye on, as we get more experience with the mapping of this metric to the actual effort that will be required to migrate.

The actionability of the metric ties to this relative measure. If I'm 0.2 libyears out of date, or 0.0.2 semvers out of date, then there's probably little to worry about. If I'm 100 libyears out of date, then maybe I need to schedule some time to migrate my dependencies. I don't know what the thresholds are yet, I don't think anybody does.

It's also possible this is ultimately useless even as a relative measure, I think it's too early to say.

Hixie avatar Jan 20 '23 22:01 Hixie

I do want to mention that we shouldn't be penalizing packages for having very broad version constraints. Basically if we do this only the upper bound should be considered (not the lower).

jakemac53 avatar Jan 23 '23 15:01 jakemac53

If I'm 0.2 libyears out of date

You can easily have this with a major revision bump needing lots of migration. Then I prefer the semver metric.

sigurdm avatar Jan 26 '23 13:01 sigurdm

It's probably too early to tell which of the three variants of this metric are most useful, but I agree that it might be the semver one in practice. If we do expose these metrics I think there's value in showing all three but I don't have any opinion about which would be the default or anything.

Hixie avatar Jan 26 '23 20:01 Hixie

It's also possible this is ultimately useless even as a relative measure, I think it's too early to say.

But is it worth the risk of adding this feature to the client without being able to quantify the value up front? It will be a breaking change to remove it again.

I'm tempted to close this issue as I don't think we're going here in the current future.

sigurdm avatar Aug 30 '24 12:08 sigurdm