dub icon indicating copy to clipboard operation
dub copied to clipboard

Fix #595 Error when using an executable subConfiguration.

Open veelo opened this issue 5 months ago • 14 comments
trafficstars

veelo avatar Jun 20 '25 14:06 veelo

✅ PR OK, no changes in deprecations or warnings

Total deprecations: 0

Total warnings: 0

Build statistics:

 statistics (-before, +after)
 executable size=5191704 bin/dub
-rough build time=59s
+rough build time=57s
Full build output
DUB version 1.40.0, built on Jun  7 2025
LDC - the LLVM D compiler (1.41.0):
  based on DMD v2.111.0 and LLVM 20.1.5
  built with LDC - the LLVM D compiler (1.41.0)
  Default target: x86_64-unknown-linux-gnu
  Host CPU: znver3
  http://dlang.org - http://wiki.dlang.org/LDC


  Registered Targets:
    aarch64     - AArch64 (little endian)
    aarch64_32  - AArch64 (little endian ILP32)
    aarch64_be  - AArch64 (big endian)
    amdgcn      - AMD GCN GPUs
    arm         - ARM
    arm64       - ARM64 (little endian)
    arm64_32    - ARM64 (little endian ILP32)
    armeb       - ARM (big endian)
    avr         - Atmel AVR Microcontroller
    bpf         - BPF (host endian)
    bpfeb       - BPF (big endian)
    bpfel       - BPF (little endian)
    hexagon     - Hexagon
    lanai       - Lanai
    loongarch32 - 32-bit LoongArch
    loongarch64 - 64-bit LoongArch
    mips        - MIPS (32-bit big endian)
    mips64      - MIPS (64-bit big endian)
    mips64el    - MIPS (64-bit little endian)
    mipsel      - MIPS (32-bit little endian)
    msp430      - MSP430 [experimental]
    nvptx       - NVIDIA PTX 32-bit
    nvptx64     - NVIDIA PTX 64-bit
    ppc32       - PowerPC 32
    ppc32le     - PowerPC 32 LE
    ppc64       - PowerPC 64
    ppc64le     - PowerPC 64 LE
    r600        - AMD GPUs HD2XXX-HD6XXX
    riscv32     - 32-bit RISC-V
    riscv64     - 64-bit RISC-V
    sparc       - Sparc
    sparcel     - Sparc LE
    sparcv9     - Sparc V9
    spirv       - SPIR-V Logical
    spirv32     - SPIR-V 32-bit
    spirv64     - SPIR-V 64-bit
    systemz     - SystemZ
    thumb       - Thumb
    thumbeb     - Thumb (big endian)
    ve          - VE
    wasm32      - WebAssembly 32-bit
    wasm64      - WebAssembly 64-bit
    x86         - 32-bit X86: Pentium-Pro and above
    x86-64      - 64-bit X86: EM64T and AMD64
    xcore       - XCore
    xtensa      - Xtensa 32
   Upgrading project in /home/runner/work/dub/dub/
    Starting Performing "release" build using /opt/hostedtoolcache/dc/ldc2-1.41.0/x64/ldc2-1.41.0-linux-x86_64/bin/ldc2 for x86_64.
    Building dub 1.40.0+commit.20.gb85ed8f7: building configuration [application]
     Linking dub
STAT:statistics (-before, +after)
STAT:executable size=5191704 bin/dub
STAT:rough build time=57s

github-actions[bot] avatar Jun 20 '25 14:06 github-actions[bot]

Thanks for tackling this! Feel free to ping me if you need anything and when it is ready for review.

Geod24 avatar Jun 20 '25 15:06 Geod24

Thank you @Geod24. I am struggling a bit with test\issue1651-custom-dub-init-type, which is failing on macOS and ubuntu. I am running Windows locally. That particular test only contains a .zip file, and run-unittest.d seems to skip it on Windows. Any idea how I can reproduce the failure locally on Windows?

veelo avatar Jun 20 '25 15:06 veelo

Got it. To reproduce on Windows:

cd test\issue1651-custom-dub-init-type
%DUB% remove custom-dub-init-dubpackage --non-interactive
%DUB% init -n custom-dub-init-type-sample --format sdl -t custom-dub-init-dubpackage --skip-registry=all --registry=file://D:\SARC\dlang\dub\test\issue1651-custom-dub-init-type -- --foo=bar

veelo avatar Jun 20 '25 15:06 veelo

Got it. To reproduce on Windows:

cd test\issue1651-custom-dub-init-type
%DUB% remove custom-dub-init-dubpackage --non-interactive
%DUB% init -n custom-dub-init-type-sample --format sdl -t custom-dub-init-dubpackage --skip-registry=all --registry=file://D:\SARC\dlang\dub\test\issue1651-custom-dub-init-type -- --foo=bar
% DUB="${PWD}/../bin/dub" dub run --single ./run-unittest.d -- issue1651-custom-dub-init-type.sh

(note the .sh at the end)

the-horo avatar Jun 20 '25 15:06 the-horo

Though it looks like run-unittest.d is skipping .sh tests on windows unconditionally, maybe making it run them if bash is found in PATH is a better idea

the-horo avatar Jun 20 '25 15:06 the-horo

Though it looks like run-unittest.d is skipping .sh tests on windows unconditionally, maybe making it run them if bash is found in PATH is a better idea

Even better would be if there were no shell scripts in the test suite ;-) I prefer D scripts.

veelo avatar Jun 20 '25 16:06 veelo

diff --git a/source/dub/package_.d b/source/dub/package_.d
index 18c5d485..dcc9cdc2 100644
--- a/source/dub/package_.d
+++ b/source/dub/package_.d
@@ -714,7 +714,7 @@ class Package {
 		}
 
 		// generate default configurations for root package if none are defined
-		if (m_info.configurations.length == 0 && m_parentPackage is null) {
+		if (m_info.configurations.length == 0) {
 			// check for default app_main
 			string app_main_file;
 			auto pkg_name = m_info.name.length ? m_info.name : "unknown";
diff --git a/source/dub/project.d b/source/dub/project.d
index e64cccad..199249a5 100644
--- a/source/dub/project.d
+++ b/source/dub/project.d
@@ -749,7 +749,13 @@ class Project {
 				} else {
 					auto subconf = pack.package_.getSubConfiguration(c, packages[dp].package_, platform);
 					if (!subconf.empty) setConfigs(only(subconf));
-					else setConfigs(packages[dp].package_.getPlatformConfigurations(platform));
+					else {
+						setConfigs(packages[dp].package_.getPlatformConfigurations(platform));
+						if (depconfigs[dp].length == 0) {
+							// Try with the executables too
+							setConfigs(packages[dp].package_.getPlatformConfigurations(platform, true));
+						}
+					}
 				}
 
 				// if no valid configuration was found for a dependency, don't include the

got the testsuite to pass for me though maybe you could make it a little bit more readable

the-horo avatar Jun 21 '25 09:06 the-horo

Kudos to @the-horo. Great job, much appreciated!

veelo avatar Jun 21 '25 23:06 veelo

@the-horo I have tried making it more readable, but I'm not sure. Is this better?

diff --git a/source/dub/project.d b/source/dub/project.d
index 199249a5..8c636b51 100644
--- a/source/dub/project.d
+++ b/source/dub/project.d
@@ -747,13 +747,12 @@ class Project {
                                if (auto pc = packages[dp].name in m_overriddenConfigs) {
                                        setConfigs(only(*pc));
                                } else {
-                                       auto subconf = pack.package_.getSubConfiguration(c, packages[dp].package_, platform);
-                                       if (!subconf.empty) setConfigs(only(subconf));
+                                       if (auto subconf = pack.package_.getSubConfiguration(c, packages[dp].package_, platform))
+                                               setConfigs(only(subconf));
                                        else {
-                                               setConfigs(packages[dp].package_.getPlatformConfigurations(platform));
-                                               if (depconfigs[dp].length == 0) {
-                                                       // Try with the executables too
-                                                       setConfigs(packages[dp].package_.getPlatformConfigurations(platform, true));
+                                               foreach (allow_non_library; [false, true]) {
+                                                       setConfigs(packages[dp].package_.getPlatformConfigurations(platform, allow_non_library));
+                                                       if (depconfigs[dp].length) break;
                                                }
                                        }
                                }

EDIT: No I don't think this is better, as allow_non_library would be shadowing the getPackageConfigs argument. Let's leave it as is.

veelo avatar Jun 22 '25 00:06 veelo

I'm happy, this is ready for review.

veelo avatar Jun 22 '25 00:06 veelo

@the-horo I have tried making it more readable, but I'm not sure. Is this better?

I'm not entirely sure as I don't even fully understand what that code is supposed to do. Conceptually, it picks the configurations for a package that has been pulled in as a dependency when a configuration override is not present. But is the behavior of this documented anywhere or are we just guessing what the code should do.?

My idea was that, when looking for a dependency, you want to try any of the library-ish configurations and fallback to the executable configurations if you couldn't find a match. I don't know if it would also work if you didn't go as far as to use a fallback, but simply iterate the dependency configs in the right order, something like library_configs ~ executable_configs and then leave the rest of the code to clean up unused edges in the dependency graph.

One edge case that came to mind would be:

  • pkgA depends on pkgDep => should default to pkgDep[library]
  • pkgB depends on pkgDep[application]

I think that this should lead to a failure as changing what pkgA gets as a result of some other package's config is hard to keep track of. I think that the current code allows this so maybe the solution would be:

When dub looks for a dependency without a configuration being specified it tries to pick

  1. The first non-executable-target config
  2. In the absence of the former, the first executable-target config

But that sounds a little bit restrictive and it would interfere with:

  1. pkgA depends on pkgDep => pick pkgDep[library]
  2. pkgB depends on pkgDep[fancy-library] => pick pkgDep[fancy-library] which would result in multiple configurations being pulled in the graph which would fail so, maybe it's better for dub to pick:
  1. Any non-executable-target config
  2. In the absence of the former, any executable-target config

I'm curios what @Geod24 thinks about this or if he has any suggestions.

the-horo avatar Jun 22 '25 05:06 the-horo

One edge case that came to mind would be:

  • pkgA depends on pkgDep => should default to pkgDep[library]
  • pkgB depends on pkgDep[application]

I think that this should lead to a failure as changing what pkgA gets as a result of some other package's config is hard to keep track of.

I don't see why you would have a dependency to an application configuration unless you're using it for building / generating code / etc... In this case, the code is not included in the final binary, so the dependency is "local" - I don't see why two packages, depending on the same package, would conflict, as long as at least one of the configuration is application.

I haven't fully reviewed the code yet though.

Geod24 avatar Jun 22 '25 12:06 Geod24

I don't even fully understand what that code is supposed to do.

You are not alone...

veelo avatar Jun 22 '25 16:06 veelo

I haven't fully reviewed the code yet though.

Friendly ping.

veelo avatar Jun 30 '25 18:06 veelo

@dkorpel is there a deadline for Dub PR merges in order for them to be bundled with 2.112 compilers? It would be great if this one could be included.

veelo avatar Jul 08 '25 11:07 veelo

Master will be tagged this week for a release candidate, then there's still some time to merge to stable for the 2.112.0 release.

dkorpel avatar Jul 08 '25 11:07 dkorpel