`dub describe --data=import-paths` produces invalid compiler flags

[pbackus]

trafficstars

System information

• dub version: 1.26.1
• OS Platform and distribution: Debian GNU/Linux 10.10
• compiler version dmd-2.097.2

Bug Description

The output of dub describe --data=import-paths contains options that are not recognized by the D compiler. Attempting to use that output as part of a dmd command line results in the following error:

Error: module `'` is in file ''-I/home/paul/tmp/test_project/source/'.d' which cannot be read

The problem is excessive quoting. dmd sees '-I/home/paul/tmp/test_project/source/' in its arguments, not -I/home/paul/tmp/test_project/source/, which prevents it from recognizing the argument as an option rather than a filename.

How to reproduce?

1. Create a new, empty dub project:

    $ mkdir test_project && cd test_project && dub init -n
   

2. Attempt to compile using dmd and dub describe ---data=import-paths:

    $ dmd $(dub describe --compiler=dmd --data=import-paths) -i source/app.d
   

Expected Behavior

The program compiles successfully.

[WebFreak001]

The --oq argument should exist at least according to the LDC help:

--oq  - Write object files with fully qualified names

I don't see it in the dmd help page online, so the whole reason might be that you are emitting these flags for LDC.

However I also don't think this should be part of the import paths description.

[pbackus]

Good catch. Adding --compiler=dmd makes that flag disappear.

[pbackus]

The quoting is still a problem. I've update my initial comment with the new error message and reproduction steps.

[WebFreak001]

ah yeah the output is probably rather made for copy-pasting, so that's why the quotes are there. The $() probably (?) puts it all in as a single string instead of properly unescaping.

There are also other modes for dub generate if you want to use it outside of bash, like --data-list or --data-0, see dub describe --help

[pbackus]

ah yeah the output is probably rather made for copy-pasting, so that's why the quotes are there

Perhaps—although some other options like --data=dflags do not include quotes. I suspect the current behavior is more of an accident than an intentional design.

In any case, I think dub describe's output should be designed for use in scripts, since that is where it is most useful. If someone wants to copy+paste it by hand, they can always add quotes themselves.

The $() probably (?) puts it all in as a single string instead of properly unescaping.

$() does field splitting unless it is enclosed in double quotes (i.e., "$()"). However, it does not do quote removal:

$ echo $(echo \'foo\')
'foo'
$ echo 'foo'
foo

This is because quote removal never applies to quotes generated by command or variable expansions, only to quotes that the user actually typed on the original command line. As the POSIX standard (linked above) puts it:

The quote characters ( <backslash>, single-quote, and double-quote) that were present in the original word shall be removed unless they have themselves been quoted. [Emphasis added]

The equivalent paragraph from the bash manual is more explicit about this:

After the preceding expansions, all unquoted occurrences of the characters \, ', and " that did not result from one of the above expansions are removed. [Emphasis added]

Source: https://www.gnu.org/software/bash/manual/bash.html#Quote-Removal

There are also other modes for dub generate if you want to use it outside of bash, like --data-list or --data-0, see dub describe --help

I am aware of the workarounds. I am currently using --data-list + sed in my script to generate an argument list that dmd will accept. But I do not think the burden should be on dub's users to do this. It should do the correct thing by default.