Drive external build system

[rtbo]

Hi, Not a bug but a feature request (and not a small one). Dub is great as a PM, but not really good as a build tool. The main problem isn't speed, it is really the flexibility when it comes to complex build scenarios. I'd like Dub to be able to drive a build system like Meson, which has a great support for D already.

Is it something that Dub's maintainers would like as well? What in your opinion would be the best way to do it?

I would be in favor of the following functionality:

• extend preBuildCommands and similar to preBuildScript and provided it a D file that is compiled and run by Dub.
• add buildCommand and buildScript. If one is present, DUB hands over the compilation to these commands.

DUB would indicate by env vars where to perform the build or other details needed, and the tool can communicate back info to DUB on stdout or stderr.

I can spend some time working on this feature, but I'd like to make sure of the direction to take before. Thanks

[WebFreak001]

Hm I think this might be a little bit complex for the current DUB architecture and it would mean for packages on the registry that they can only be used with the build systems installed and setup on the user's systems.

I think it would be better to call DUB from your build system just to fetch dependencies and provide built dependency or sourceLibrary file paths. You can work using dub describe to get a variety of information and you can use dub build <package name> to build dependencies in isolation.

[rtbo]

Consuming Dub dependencies from Meson is not really difficult. What I would like essentially is to be able to build my Dub package with Meson, because Dub is unable to handle complex build scenarii.

it would mean for packages on the registry that they can only be used with the build systems installed and setup on the user's systems.

This is true but not so much a problem. The "toolchainRequirements" already exist to limit the use of recipes if some compiler or dub itself is not in a given version. It can be augmented to search for arbitrary program.

[WebFreak001]

so what's your current problems with running DUB from Meson? Would probably be useful to have a list of things that you want to have supported.

[rtbo]

What I can do with Meson and can't with Dub (or not easily):

• build C/C++ code as part of the build process and link it with D
• compile a tool that generates code, which is afterwards used in the build
• generate resources that are imported afterwards. E.g. compile shaders to SPIR-V in a directory found with -J flag.
• create a config.d on the fly with informations gathered during a configuration phase (e.g. activate versions depending on what libraries are installed on the system, this is often useful for bindings)
• consume pkg-config dependency
• doing everything I've listed off-tree without polluting the source folder, in a directory managed by the build-system.

Some of these tasks can be achieved with preBuildCommands but not easily and not always in a portable way. Some of these tasks simply can't be done with Dub. This is why I'd like the possibility to have Dub handing over the build to a program that is doing things right, but still distribute in the Dub registry.

[WebFreak001]

extend preBuildCommands and similar to preBuildScript and provided it a D file that is compiled and run by Dub.

You can use $DC -run program.d (with current version there is a bug #2192 with that, working on fixing that atm)

Some of these tasks can be achieved with preBuildCommands but not easily and not always in a portable way.

Portable as in platform specific? You can use the platform suffixes to restrict the commands to given os/compiler/architecture.

[Panke]

I also want to use Dub together with a different build system and dub build and dub describe make this possible (although it could be more straightforward). What I am wondering though is: When I build my stuff with meson, how can I make it available to people that only use dub? For simple stuff it's easy to just maintain an additional dub.sdl, but something like pkg-config for D would be great. Or just pkg-config.

[rtbo]

@Panke In the Meson wiki, it is shown how to generate a pkg-config file for a D library, so we can expect that Meson is also able to use such library afterwards. The DFLAGS are read through the --cflags switch of pkg-config. I'm not sure however if any other build system would do the same.

[rtbo]

You can use $DC -run program.d

I didn't know that, for sure it will help in many situations.

You can use the platform suffixes to restrict the commands to given os/compiler/architecture.

Yes, I'm aware. The following snippet is from one of my repos (and I have another repo to build all the linked C libraries). The reason for such extremity is that we cannot always expect people to have the right libraries at the right place, especially on Windows, so to provide lean and clean builds to everyone I didn't find another solution than this one. (Nowadays I would not bother supporting 32bits builds) Once again Meson would be of a great help.

    "sourceFiles-windows-x86-ldc": [
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/turbojpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/jpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/libpng16_static.lib"
    ],
    "dflags-windows-x86-ldc": [
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/turbojpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/jpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/libpng16_static.lib"
    ],

    "sourceFiles-windows-x86-gdc": [
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/turbojpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/jpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/libpng16_static.lib"
    ],
    "dflags-windows-x86-gdc": [
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/turbojpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/jpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/libpng16_static.lib"
    ],

    "sourceFiles-windows-x86_mscoff-dmd": [
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/turbojpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/jpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/libpng16_static.lib"
    ],
    "dflags-windows-x86_mscoff-dmd": [
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/turbojpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/jpeg-static.lib",
        "$PACKAGE_DIR/clibs/i686-pc-windows-msvc/lib/libpng16_static.lib"
    ],

    "sourceFiles-windows-x86_64": [
        "$PACKAGE_DIR/clibs/x86_64-pc-windows-msvc/lib/turbojpeg-static.lib",
        "$PACKAGE_DIR/clibs/x86_64-pc-windows-msvc/lib/jpeg-static.lib",
        "$PACKAGE_DIR/clibs/x86_64-pc-windows-msvc/lib/libpng16_static.lib"
    ],
    "dflags-windows-x86_64": [
        "$PACKAGE_DIR/clibs/x86_64-pc-windows-msvc/lib/turbojpeg-static.lib",
        "$PACKAGE_DIR/clibs/x86_64-pc-windows-msvc/lib/jpeg-static.lib",
        "$PACKAGE_DIR/clibs/x86_64-pc-windows-msvc/lib/libpng16_static.lib"
    ]

[Panke]

@Panke In the Meson wiki, it is shown how to generate a pkg-config file for a D library, so we can expect that Meson is also able to use such library afterwards. The DFLAGS are read through the --cflags switch of pkg-config. I'm not sure however if any other build system would do the same.

That's true, but does really answer my original concern: How do we make build artifacts that are not build with dub available to other dub users (the D ecosystem).

[rtbo]

How do we make build artifacts that are not build with dub available to other dub users (the D ecosystem).

If you build binaries with meson, you can place them in platform/compiler specific folder (added to git) and play with platform/compiler specific sourceFiles and dflags so that Dub can link to them when used as a dependency (Like in my previous message). This is what I did for img-d, except it was to provide C libraries along with D bindings source files. (but I don't think that Dub will complain if there is no D file to compile at all).

The point of this issue is to avoid this process entirely.

[kassane]

Like other users, I must agree with the lack of some features in the dub itself.

Currently, I'm using build.zig to build libsokol and run ldc2 sharing the configurations (targets, flags, libs) between both toolchains.

However, dub support will not be ruled out.

• https://github.com/kassane/sokol-d/issues/2