Support snap packaging dub projects and dub itself

[WebDrake]

Recently I've been playing with creating snap packages from D projects, specifically:

• supporting dub as a plugin for snapcraft, so dub projects can be easily turned into snap packages;

• creating a snap package for dub itself, to facilitate getting the latest releases more easily into Ubuntu and other Linux distros.

The draft work for all this can be found here (this initial PR will be finalized once dub v1.1.1 is released): https://github.com/WebDrake/dub.snap/pull/1/files

This issue is intended to summarize the work done so far, the work still to be done, and to coordinate on what the snapcraft plugin would need to support (and potentially, what dub could do to better support the plugin).

[WebDrake]

Of the work done so far, the essential components are as follows.

snapcraft dub plugin

The dub.py file defines a snapcraft plugin that permits the creation of snap packages from dub projects. These plugins essentially need to support build and install options. Since dub only supports the former, the install stage may require some extra work on the part of the person creating the snap package. In the long run, this plugin should be submitted upstream as part of snapcraft; in the short term, it can be copy-pasted into other snap package definitions.

The plugin currently gets a copy of dub by downloading the dub package available in the Ubuntu archives (in future a better option may be to download one of the prebuilt dub packages from https://code.dlang.org/download; more on this later). It does not download any particular D compiler; this is expected to be handled by the user of the plugin.

It currently supports a subset of the dub build command-line flags:

• --build= can be specified using the dub-build option (defaults to debug if not specified);
• --config= can be specified using the dub-config option (defaults to none if not specified);
• the build target can be specified using the dub-target option (defaults to none if not specified);
• the expected targetPath (as specified in dub.json) can be provided via the dub-target-path option (this does not affect any command-line flags invoked, but can help the plugin filter what components will be copied by the plugin's install stage).

snap package for dub

The snapcraft.yaml file defines a snap package for dub itself, using the dub plugin to help build it (since the build.sh script would be difficult to use with snapcraft). In the long run, this could in principle be included in the upstream source tree to allow a snap package to be built directly from the dub source itself, but it currently works as an "external" snap fetching the dub source from GitHub and building from that.

A brief summary of its contents:

• the apps: section defines the applications that will be exposed by the snap package (dub only) and the access they require to the system (access to the user's home directory, and access to the network).
• the parts: section defines the components included in the snap package:
  • dub: the source location is specified via a git repository and branch (a tag will be used instead once v1.1.1 is out). Since the dub plugin is being used to build this (using the Ubuntu dub package to bootstrap things), some of the dub-specific options are provided, which mean that dub will be invoked with the flags --build=release --config=application. The dub-target-path option is also used to ensure that the "install" stage will only copy files from the build/ directory; the organize: and snap: fields ensure that the dub executable is correctly placed, and that the dll files included with dub are excluded. Finally, the build-packages: section specifies the packages that must be present in order to carry out the build; ldc is included as the D compiler of choice (though we could just as well have used gdc).
• curl is included simply by including the Ubuntu libcurl-gnutls4-dev package in the snap.
• LDC is built and included in the snap to ensure that a D compiler is available. In the long run, this will be unnecessary, as it will be possible to define a snap interface via which a D compiler can be made available, but in the short term the snap-packaged dub will be able to use no other compiler, since its apparmor containment will prevent it from being able to access them.
• GCC and libc are included in order to allow LDC to work correctly. Again, this is a short-term solution to allow the package to work; once the interface for an externally-packaged D compiler is available, this can be removed.

constraints on use

Due to snap-package confinement, there are currently a number of limitations on what can be done with the snap-packaged dub:

• it is possible to build a D project that depends only on druntime, phobos, and other dub packages; the latter will be downloaded to /home/[username]/snap/dub/[release]/.dub/ instead of the regular home-directory location.
• It is not possible to build a project that needs to link against system libraries, because the current snap-package confinement prevents access to these host-system resources. Work on allowing this kind of access is an ongoing snapcraft project. This may be more a constraint on the snap-packaged LDC than on dub itself.
• As previously mentioned, the only D compiler that can be used is the LDC included in the package. (LDC was chosen because it is easier to include in a snap-package than any of the other D compilers; there is an easy to use cmake plugin.)
• Since no rdmd instance is included in the snap, the --rdmd option will not work.

Beyond this, I have not extensively tested the results; at this stage I'm more interested in what people think of the general idea, particularly what you think of the dub.py plugin and what extra features it might need to support.

Anyone running Ubuntu 16.04 or later can have a go at building the snap package by following the instructions in the project readme: https://github.com/WebDrake/dub.snap/blob/master/README.md

[WebDrake]

... ping? Don't know how many people this is of interest to, but I was hoping at least for some comments or feedback.

Would be particularly interested in any thoughts on the possibility of creating a dub install command that could be used to support the dub plugin for snapcraft.

[WebDrake]

Just to follow up here: fairly recently snap packages gained a third confinement option, classic, which allows the package to see and use the resources of the host Linux system on which it is installed. This has greatly simplified packaging dub, which now no longer needs e.g. a D compiler to be bundled alongside it.

See https://github.com/WebDrake/dub.snap/pull/3 for details.