dart-lang
Dart workspaces clarification and documentation improvements
I would love some clarification and improvements to the existing documentation around Dart workspaces. There are many ways to organize a mono-repo with Dart workspaces and it seems there's no clear guidance or guard-rails steering developers one way or another. For context, we're building a Flutter/Dart specific CI product at Shorebird and we would love to better understand how workspaces were intended to be used so that we can reinforce best-practices.
Open Questions:
- Even though the Dart workspaces documentation implies that workspace roots should be empty, there's technically nothing preventing developers from having a workspace root that also contains a library. For example, see https://github.com/LukasMirbt/hacker_news/tree/6631d4b49325bd1d5452a70cd3cc7397c62fba7d. Is there a valid case for allowing developers to include library implementations directly within a workspace root?
- How should a mono-repo with multiple packages be structured with workspaces (e.g. https://github.com/felangel/bloc)?
- How should a mono-repo with a single Flutter application and several local packages be structure with workspaces (e.g. https://github.com/LukasMirbt/hacker_news/tree/6631d4b49325bd1d5452a70cd3cc7397c62fba7d)
- How should a mono-repo with a full-stack application (multiple Flutter apps, server-side components, and packages) be structured with workspaces (e.g. https://github.com/flutter/news_toolkit/tree/main/flutter_news_example)
cc @sigurdm (please transfer this issue to dart-lang/pub if we're tracking workspace items there)
I don't see any problems with allowing this. And I also don't think we can change it now without being breaking.
The root package could contain tools that work across all the workspace, and library code to support those tools.
Do you have a special reason you want this enforced?
I don't see any problems with allowing this. And I also don't think we can change it now without being breaking.
The root package could contain tools that work across all the workspace, and library code to support those tools.
Do you have a special reason you want this enforced?
I was just asking because I'm guessing most developers that have a single Flutter app with several packages will adopt this structure of keeping the Flutter app in the workspace root rather than moving the Flutter app out of the workspace root.
Possible issues I see with this are:
- CI checks that are optimized to only perform checks on modified parts of the workspace will perform potentially unnecessary checks on the workspace root.
- This also means that in CI, running checks on the workspace root by default will run on all packages. e.g.
dart format .anddart analyze .will format and analyze the entire workspace not just the app at the root of the workspace which may be undesirable from a performance/time-savings standpoint. - My understanding was that the workspace root's
pubspec.yamlshould only contain shared dependencies and in cases like the one described above this is no longer true. The workspace root will declare dependencies that are specific to a single application.
This also means that in CI, running checks on the workspace root by default will run on all packages. e.g. dart format . and dart analyze . will format and analyze the entire workspace not just the app at the root of the workspace which may be undesirable from a performance/time-savings standpoint.
I don't think this is a very serious issue. If indeed the root package is empty all these checks should be trivial and take very little time.
My understanding was that the workspace root's pubspec.yaml should only contain shared dependencies and in cases like the one described above this is no longer true. The workspace root will declare dependencies that are specific to a single application.
Where is this understanding from? This is one possible setup, but not mandated by the pub workspace design.
It was indeed a thought that the example/ project of a flutter plugin could be part of the workspace of the plugin itself - in that case the root would not be empty. I have never got to make that change in the flutter templates, but hope to get there one day.
Closing this, as we are very unlikely to change this design.
I don't think this is a very serious issue. If indeed the root package is empty all these checks should be trivial and take very little time.
My point is the root package in the case I'm describing (see example linked above) is not empty -- it's a full Flutter application.
Where is this understanding from? This is one possible setup, but not mandated by the pub workspace design.
The documentation and your prior comment: "The root package could contain tools that work across all the workspace, and library code to support those tools.". The workspace root in the case I'm describing contains a Flutter application, not tools that work across the workspace.
It was indeed a thought that the example/ project of a flutter plugin could be part of the workspace of the plugin itself - in that case the root would not be empty. I have never got to make that change in the flutter templates, but hope to get there one day.
So then what is the guidance for when something should be defined directly in the workspace root versus being a package within the workspace?
So then what is the guidance for when something should be defined directly in the workspace root versus being a package within the workspace?
I don't think we have a specific guidance. Pub workspaces is a tool for resolving multiple packages together. It is designed with some amount of flexibility, and several different use cases in mind. It is designed to be a generalization of the single root package pub used to consider.
One obvious use case is that of a mono-repo with multiple packages living together. In that case it doesn't make a lot of sense to have a Flutter app in the root.
Another potential use-case is that of an application that has a few supporting packages nested inside. Or a plugin with an example package.
But again, pub itself does not dictate or even recommend anything here.
@sigurdm the goal of this issue was to get clarification on how the Dart team intends for workspaces to be used (e.g. what are the best practices for structuring a mono-repo with workspaces). I think the current documentation doesn't really cover this and I think the broader community would benefit from more concrete guidance around the various use cases developers face and how workspaces can be used to help in those scenarios. Some possible common use cases I can think of are:
- Package mono-repo (a bunch of packages that are being published to pub.dev)
- Flutter app mono-repo (a single Flutter app and a bunch of packages used by that Flutter app)
- Full stack mono-repo (a single Flutter app, Dart backend, and various packages used by the Flutter app and/or backend)
It's not immediately clear in each of these scenarios what the best way to structure the project is with Dart workspaces and imo this issue should be reopened until the documentation is improved around workspaces.
Again, I'm not sure the dart team wants to specify an opinion. Or even that we at this point know what the best practice would be.
Perhaps it is better for a tool like melos that wants to impose a certain structure gives a guideline.
Are there specific questions around the organization of a workspace you would like answered?
@mit-mit wdyt? Should we try to come up with guidelines? Don't we risk over-guiding here?
Again, I'm not sure the dart team wants to specify an opinion. Or even that we at this point know what the best practice would be.
It seems strange for a new feature to be rolled out without any guidance around how to use it imo. Especially given there are many ways to structure a mono-repo with workspaces.
Are there specific questions around the organization of a workspace you would like answered?
- Should any libraries be included directly in the workspace root? If so, what are the guidelines for when something should be in the root (
_) vs being declared in the workspace and having it's own root (with resolution: workspace)? - How should a mono-repo with multiple packages be structured with workspaces (e.g. https://github.com/felangel/bloc)?
- How should a mono-repo with a single Flutter application and several local packages be structure with workspaces (e.g. https://github.com/LukasMirbt/hacker_news/tree/6631d4b49325bd1d5452a70cd3cc7397c62fba7d)
- How should a mono-repo with a full-stack application (multiple Flutter apps, server-side components, and packages) be structured with workspaces (e.g. https://github.com/flutter/news_toolkit/tree/main/flutter_news_example)
It seems to me that you wouldn't want to declare any libraries (unless they are shared) in the workspace root for the reasons I mentioned above but based on the earlier replies it seems that's not necessarily the case?
Right - I'll try to write some thoughts here, perhaps we can find improvements we should add to the docs.
I think the current documentation shows an example of how to structure a mono-repo of somewhat related packages. I think that is an OK structure for migrating an existing mono-repo, though not the only way workspaces can be used.
It doesn't mention if the root package should be empty or not. I'd argue that there's no really problem in using the root as a place for tools for manipulating the entire workspace. But you could also place that in a separate package if you prefer - there's no real restrictions from our point of view.
We have migrated flutter engine, the sdk and flutter itself to this structure, giving us some confidence that this can work out quite well.
I think it makes sense for plugins/packages to have their example/ folder included in a workspace, in which case the main plugin would be the root, and thus non-empty, we haven't documented this, and we haven't migrated the current flutter create/dart create templates to this structure, so there might be things I haven't really thought about.
And these two layouts can be combined, given that mono-repos can nest. So we could imagine migrating eg. flutter/plugins mono-repo to this structure.
For the flutter_news_example I personally think I would move the main app into the packages/ folder, and have a more or less empty root workspace. But I think the current layout can work also. Perhaps the most important guideline is that you should not nest packages that you don't want to publish together with the parent inside the parent. (Because publication will publish more or less everything inside the folder).