dart-lang
Publish queue with ecosystem compatibility testing
(Filing for tracking based on offline discussions.)
Something that has come up indirectly via Flutter issue reports (when breaking changes happen in transitive dependencies) and directly in discussions with package maintainers is that it can be tricky to reliably determine what is a breaking change and version accordingly. Sometimes it's just that people aren't paying enough attention to a change and break things accidentally, but even when people are being very vigilant there are grey areas: e.g., is adding an optional parameter to a method breaking? Almost nobody treats it as such unless it's a class deliberately designed to be subclassed/implemented, but just because it's not intended doesn't mean that people don't do it (example: https://pub.dev/packages/file).
Something that could substantially cut down on ecosystem-wide breakage would be a publishing queue that did cross-package testing before publishing. As a rough high-level outline:
- A package maintainer puts a new version into the publish queue (e.g., maybe it's a new command, or a flag like
publish --with-queue) - pub.dev takes that version, finds the top N (ideally a large value of N) packages that depend on it, and for each one analyzes it with the new version of the package (subject to normal version constraints).
- If everything passes, the package is published.
- If not, the maintainer gets an email linking to a page of results with the failing packages and the errors.
There would have to be a way of bypassing this system and publishing directly, but if this were the standard way of publishing it would be much, much harder for a package to accidentally break other packages in a way that causes confusing errors for everyone in the transitive dependency graph.
Before doing it as part of the pub.dev (staged publishing), I think first we should build a local tool for such checks. Once we are happy how it works locally, we may figure it out to integrate into the publishing workflow...
