Add `dub install` command

[ximion]

As per issue #811 it would be nice to have a dub install command to install the built binary/shared|static library and accompanying data files into a standard directory hierarchy used in Linux systems. Ideally, the install command would - for shared/static libraries - also install the needed sources or .di files into a directory hierarchy which can be used as a local repository, as outlined in #838. This would be of great help for packaging dub-using D code in Linux distributions.

Since there were concerns about users blindly dub-installing files into their filesystem, I propose to make setting a --target= parameter mandatory. Distributors could run dub install --target=debian/tmp without issues, and to protect users dub could ask for a second confirmation when someone passes / as target directory (something along the line of "This will install this software permanently on your system, and might be incompatible with distribution packageing. Do you want to proceed?"). That being said, theoretically a dub uninstall command would also be possible to remove installed stuff from the file system again ^^

But this report is about dub install --target=DIR - would be awesome if we could have this feature.

[schancel]

Maybe take a look at go install which will look for a GOBIN environment variable and attempt to place the binaries in the path given by that variable. It makes it rather convenient to just add to your shell environment if you work with go quite a bit.

[mogud]

IMO. dub install is important for developers who write tools in D. For example: D Profile Viewer

[MartinNowak]

Where to install the files? Having them (at least the source) somewhat isolated seems worthwhile. How to deal with different versions? Leave it to the user (or pkg-config like tools) to declare correct import paths?

/usr/include/d/<package>-<version>/package to be used with -I/usr/include/d/<package>-<version> for packages that have source files in their root folder (see https://github.com/dlang/dub/pull/735). /usr/lib64/lib<package>-<maj.min>.a /usr/lib64/lib<package>.so.maj.min.patch (+symlinks)

Library path depends on distribution, e.g. /usr/lib/x86_64-linux-gnu on debian, /usr/lib64/ on redhat...

What about integration with .rpm and .deb building instead of directly installing files?

[mogud]

Why not install them in a hidden folder of $HOME directory? So many others do like this.

[MartinNowak]

Indeed dub should offer an installation mechanism similar to pip install <pkg> --user, and maybe also some system-wide installation pip install <pkg>, npm install -g <pkg>. The binaries would just be symlinks, so that packages can access their normal resources (using thisExePath). Shared libraries would also need to be symlinked.

Maybe we could simply start this as experiment, installing to ~/.local/bin and ~/.local/lib, to figure out where this goes. Should be simple enough to be implemented by anyone interested.

[wilzbach]

@ximion: @s-ludwig and I briefly discussed this during the DConf hackathon. The summary:

@s-ludwig is still a bit wary about complicating dub with the logic of figuring out where to install system binaries as this heavily depends on the distro. As far was we could tell for system installations a wrapper around dub or tool that uses dub's API should work fine and would avoid the predictable issue flood if we implement this only for some distros. Once https://github.com/dlang/dub/pull/1459 gets merged, it should be rather straight-forward to package dub packages on debian?

Anyhow, installing binaries to ~./local/bin and ~/.local/lib indeed sounds like sth. we should really try to add support for in dub.

[ximion]

@s-ludwig is still a bit wary about complicating dub with the logic of figuring out where to install system binaries as this heavily depends on the distro.

It's not that complicated. Actually I think it's the easiest issue to resolve. All distros follow the FHS1 for the most basic directories. Here is a full implementation in Python that Meson uses: https://github.com/mesonbuild/meson/blob/master/mesonbuild/mesonlib.py#L490-L538 It's important though that these default paths can be overridden by the Linux distributor individually.

Once #1459 gets merged, it should be rather straight-forward to package dub packages on debian?

That depends... I can't promise anything, I would actually need to test that, since my last attempt was years ago. I do remember though that dub was writing to $HOME during a regular compilation process for some caching. That is also a forbidden practice - while not a dealbreaker like dub not being able to use local packages, it would force us to add some annoying hacks. Some interoperability with other projects would also be neat, via e.g. pkg-config2 (but that's also not mandatory - it would allow projects not using dub to use libraries built with dub easily though). Fixing #838 will indeed fix the issue that ultimately killed dub usage for package building in Debian back in the days - thank you for that! :-)

Anyhow, installing binaries to ~./local/bin and ~/.local/lib indeed sounds like sth. we should really try to add support for in dub.

Those paths are completely non-standard though, although commonly used. Users can't use them without some manual adjustments (which IMHO is a good thing, so users don't accidentally call binaries from their home directories). We do have some discussion though to specify at least the bin directory in some way3.

[Timoses]

What about a simple configuration field in one of dubs settings.json? E.g. "installPath".

When it is not set and a user runs dub install it prints Specify installPath in settings.json or use --target=...

This would at least allow users to set up their environments so dub does the "right" (user-defined) thing. It could be further split up into binaryInstallPath and libraryInstallPath.

[andre2007]

Maybe it is not needed to install the binary to any folder on the file system but just create batch/bash scripts which executes dub run packageName

#!/bin/bash
dub run helloworld --vquiet -- \$@

For Linux the scripts can be created here ~/.dub/scripts/

Using this approach you can easily update the dub packages and the scripts points to the recent package version. User has the task to add the path ~/.dub/scripts/ to his environment variable PATH. Maybe the install command can warn the user, if this path is not in his environment variable PATH (similar to NPM).

Also please have a look at this dub package. https://github.com/d-language-server/dls @LaurentTreguier has written something similar and his project is a good candidate for a generic dub install command.

[schveiguy]

Was just pointed here by @wilzbach, see #1991 for something that might be less complex on dub's side and provide similar options.

[CyberShadow]

I suggest to simplify this proposal and make it orthogonal to the rest of Dub as follows:

1. Implement dub install to have the same effect as publishing to a Dub registry + dub fetch. Meaning, dub install puts the project from the current directory in the Dub package cache in the same way dub fetch does.
2. The directory where to install to can be selected with the existing --cache switch (e.g. --cache=system to install system-wide), and further defined with the recently-added DUB_HOME and DPATH environment variables.
3. Installing executables to some ~/.dub/bin directory is orthogonal to dub install, and should be done by both dub fetch and dub install. It doesn't need to be done at the same time.

I think that will address all concerns?