sdk icon indicating copy to clipboard operation
sdk copied to clipboard

[cli] Introduce `dart install` (replacing `dart pub global activate` to use)

Open dcharkes opened this issue 1 year ago • 11 comments

2025-06-11: Updated to introduce dart install, see https://github.com/dart-lang/sdk/issues/60889#issuecomment-2961956693 for a detailed design.

We should change dart pub global to use AOT instead of JIT.

Also, we should not let pub need to worry about how dart is compiled and run, dart should be responsible for that.

@jonasfj suggested that dart pub global activate should use dart build exe.

Subsequently, $PUB_CACHE/bin/ could contain symlinks to the executables in $PUB_CACHE/global_packages/<package_name>/outputdir/<package_name>.exe.

This would

  1. remove logic from pub-global about how Dart is compiled,
  2. speedup startup of pub-global activated packages, and
  3. make native assets work (https://github.com/dart-lang/sdk/issues/56044).

Since the executables would include a dart precompiled runtime. The Dart version which is running would be the one that was used to run dart pub global activate. If we ever want to re-activate a package with a different Dart SDK version, we might need some way to detect what version of the Dart SDK was used to compile the executable.

@jonasfj and @sigurdm probably know much more constraints and considerations. I'm filing an issue so that we have something to point to.

dcharkes avatar Jun 20 '24 11:06 dcharkes

One stumbling block is the lack of dart:mirrors support in AOT compilation

sigurdm avatar Jun 20 '24 11:06 sigurdm

+1. One step towards this could be to use existing binary (if it was compiled before manually). It is strange, that Dart uses .snapshot file from .dart-tool instead of executable from bin/, especially since manual edit of .bat script in cache and pointing there to .exe works well. @sigurdm is support for this feature dart:mirror going to be implemented in some subsequent release?

gryznar avatar Sep 19 '24 06:09 gryznar

I don't think dart:mirrors will be supported in AOT-mode (See https://github.com/dart-lang/sdk/issues/44489).

I think that we should consider overhauling everything to do with global activation. Ideally, it shouldn't have to be a two-step process, but something more akin to npx. But that's just like my opinion (at this point in time; not fully conceived).

But I do think we should consider changing from JIT to AOT along with a minor overhaul of global activation in general. Both things will affect PUB_CACHE and cause pain for people switching between SDK versions.

jonasfj avatar Sep 24 '24 14:09 jonasfj

@jonasfj suggested that dart pub global activate should use dart build exe.

We must do this, otherwise any package using build hooks with code assets will stop working.

I don't think dart:mirrors will be supported in AOT-mode (See dart-lang/sdk#44489).

I believe we should consider marking mirrors as no longer supported. @jonasfj Any chance we have data on which packages are pub global activated by users, is that instrumented? If so, we could check if any of the top ones uses mirrors.

@mkustermann suggested we should introduce dart install (which would be very similar to package manager CLI commands). If we introduce a new command, we also don't have to say not supporting mirrors is a breaking change. 🙃

@mit-mit Any thoughts on dart install as a command and not supporting mirrors?

dcharkes avatar May 22 '25 16:05 dcharkes

@mkustermann suggested we should introduce dart install

I can also give some background thoughts on this.

The current state of dart pub global activate is

  • It lazily compiles the entrypoint to a kernel file (for faster time-to-main in future runs)
  • If the SDK changes it may need to rebuild the kernel file => This causes unpredictable time-to-main
  • Therefore it needs to have the source code of the app => It's not self contained but requires pub cache
  • The app can (not sure) probably not find it's own package or it's package dependencies => It's probably not able to find any resources in package dirs
  • It doesn't work with the new dart hooks

Now IMHO we should have a way to build an entrypoint of a package and get a self-contained deployment bundle that can be run locally (or even distributed to others). The dart build command will produce such bundles.

For the use case where we want to get such bundles for packages from pub for local use, I think a dart install that puts it into an installation path would be great. It wouldn't depend on the ~/.pub-cache or .dart_tool folders. It could install to /usr/local/* or user-defined directories.

This would also potentially also allow package authors to pre-built not just hook artifacts (e.g. shared libraries) but also the apps themselves - bypassing the AOT compilation at dart install time.

The installed apps may have been built with older Dart SDK versions, so it may make sense to have some tooling to manage those installed packages:

  • list all installed packages
  • see if any of them are out-dated (i.e. package has newer version or newer Dart SDK to compile them with)
  • remove packages

mkustermann avatar May 27 '25 08:05 mkustermann

Maybe one addition: One could allow a dart install --mode=jit where the deployable bundle that is locally installed uses JIT and dart:mirrors - it could compile the app to kernel (as pub global activate does now) - run build hooks and bundle all assets with the kernel file - but compared to pub global activate, it could include the JIT runtime - making the bundle self contained (just like in AOT) and not suffer from the downsides mentioned above (unpredictable time-to-main, need pub cache, Dart SDK upgrade influences the installed app, ...)

mkustermann avatar May 28 '25 08:05 mkustermann

Installation structure

Design

I believe our best model is Homebrew and Nix:

  1. Managed Installation Location: Dart CLI tools would be installed into a dedicated, versioned, and managed directory structure: ~/.dart-tools/installed/<package-name>/hosted/pub.dev/<version>/ or ~/.dart-tools/installed/<package-name>/git/<hash>/. This allows for:

    • Isolation: Each tool and its dependencies are contained within its own managed space.
    • Clean Uninstallation: Easy and complete removal of tools.
    • Version Management: Facilitates the installation of multiple versions of the same tool.
  2. Centralized bin Directory with symlinks: , ~/.dart-tools/bin/. Having this directory on the PATH makes all Dart CLI apps available on the terminal. This allows for:

    • Atomic updates: Installing a new bin is atomically updating a symlink.
    • Responsibility in Dart: Not in pub.
  3. PATH Check and Warning:

    • Since Dart and Flutter do not currently use installers to modify system PATH, the dart install command itself would take on this responsibility.
    • On the first (or subsequent) run of dart install, it would check if ~/.dart-tools/bin/ directory is present in the user's PATH environment variable.
    • If not, dart install would print a clear, actionable warning instructing the user on how to add the directory to their PATH, with specific commands for common shells (e.g., Bash, Zsh, PowerShell, Command Prompt). This provides a friendly nudge without forcing changes or requiring an external installer.

Open questions

  • ~/.dart-tools/? ~/.dart-cli/? ~/.dart-global/? ~/.dart/tools/? ~/.dart/cli/?~/.dart/global/?
    • ~/.dart-installed/
  • If multiple packages define the same bin/xxx.dart, their executable names would conflict.
    • Probably we should caution package authors to prefer only using their package name.
    • We could possibly track this and warn the user on installation. (Alternatively, we could simply overwrite for now.)
  • Should we automatically remove older versions, or should we have a dart activate that enables one to activate an older version again?
    • The older versions won't be used, but if the newer version doesn't work properly it's nice to be able to revert.
  • If you have multiple versions of Dart floating on your system (with fvm), we want to ensure that using dart install from these doesn't break.
    • Using symlinks in the bin dir is mostly atomic so that should be fine.
    • Should we consider adding the dart-version to the managed install dir: ~/.dart-tools/installed/<dart-version>/<package-name>/hosted/pub.dev/<version>/? Users should likely not do dart install in two different Dart SDKs concurrently.
  • Symlinks require privileges on Windows.

Alternatives Considered

  • Go's go install / Cargo's cargo install Approach (Direct write to a single bin on PATH):

    • This approach works well for languages like Go and Rust because they default to producing self-contained, statically linked executables.
    • Dart programs, are not self-contained binaries, they can have code assets and data assets in the future. Hence a model with symlinks into isolated directories is a better approach
  • Filesystem Hierarchy Standard (FHS) Approach (Distributing files across system directories):

    • This is common for system-level package managers (like apt), where executables go into /usr/bin, libraries into /usr/lib, configuration files into /etc, etc. These standard directories are already on the system PATH.
    • This is not self-contained. Our aim is not to share dynamic libraries, we want a self-contained bundle.

CLI

Design

  • dart install
  • dart uninstall uninstall a package
  • dart installed list installed packages and versions

Alternatives considered

  1. Go's go install: No related commands to install.

    • Go has go install but no go uninstall or go list. This is impractical.
  2. Rust's cargo install --list: Flags for the less important subcommands.

    • This is a bit of an anti-pattern. We don't want different subcommands under flags. (Then all other options/flags apply to it.)
  3. Python's pip install: Package manager as separate executable.

    • We explicitly moved away from pub as a separate top-level executable.
  4. dart pub install

    • We deprecated dart pub run in favor of dart run, so dart install belongs at the top level too.

I probably forgot to consider a bunch of aspects. Any suggestions @goderbauer @mkustermann @bkonyi @jonasfj @sigurdm?

dcharkes avatar Jun 11 '25 09:06 dcharkes

I'm generally supportive of this. Using pub for installing CLI tooling has always felt a bit strange and dart pub global activate is very verbose and doesn't really do a great job of communicating what the command does.

I think it's worth pulling together a public(?) design doc that we can share here and with the team (at least the Dart CLI mailing list) if we want to move forward with this.

bkonyi avatar Jun 11 '25 14:06 bkonyi

We also need to figure out what the migration path would be, people already have the pub cache bin dir on their PATH for globally activated packages, and using dart install will create similarly named binaries, so probably which ever is on the PATH last will just win, which will be confusing.

A future version of the SDK could just aggressively delete the pub cache bin dir, although that seems a bit nuclear.

jakemac53 avatar Jun 12 '25 22:06 jakemac53

We also need to figure out what the migration path would be, people already have the pub cache bin dir on their PATH for globally activated packages, and using dart install will create similarly named binaries, so probably which ever is on the PATH last will just win, which will be confusing.

A future version of the SDK could just aggressively delete the pub cache bin dir, although that seems a bit nuclear.

Good point. 👍 We could also try to dart pub global deactivate the package in question first in dart install.

dcharkes avatar Jun 13 '25 07:06 dcharkes

I think dart install should leave pub global activate alone.

When it installs an executable it can check if there is another executable by the same name on the path, and explain paths of action.

It could also suggest doing dart pub global deactivate if the existing executable is in the pub cache. But I would not like it to do anything automatically.

sigurdm avatar Jun 13 '25 07:06 sigurdm