rake
rake copied to clipboard
Published
20 hours ago •
ruby
ruby
sh_show_command: Quotes are missing for an extra option including a space.
Summary
In the following rake command to compile a native extension with rake-compiler gem, where the extra options whose value (--with-cflags='-Wundef -Werror') includes a space, in this case -Wundef -Werror is printed without quotes. And the command is different from the one actually executed and it's not reproducible.
$ bundle exec rake compile -- --with-cflags='-Wundef -Werror'
...
/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags=-Wundef -Werror
...
I think quotes (double quotes or single quotes) are needed like the command below.
/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags="-Wundef -Werror"
Note that @kou helped me to report this issue here at https://github.com/rake-compiler/rake-compiler/issues/215#issuecomment-1563493206.
My environment
I am using the following my environment on Fedora Linux 37. I think this issue doesn't depend on the specific version of the Ruby. I would share it with you just in case.
$ ruby -v
ruby 3.3.0dev (2023-05-25T12:45:21Z wip/mkmf-verbose-r.. cbeba2ed16) [x86_64-linux]
$ which gem
~/.local/ruby-cbeba2ed16/bin/gem
$ gem -v
3.5.0.dev
$ which bundle
~/.local/ruby-cbeba2ed16/bin/bundle
$ bundle -v
Bundler version 2.5.0.dev
Reproducing steps
Here are the reproducing steps with the ruby/openssl: https://github.com/ruby/openssl first.
$ git clone https://github.com/ruby/openssl.git
$ cd openssl
$ bundle install --standalone
$ bundle list | grep rake
* rake (13.0.6)
* rake-compiler (1.2.2)
Ran the rake with the rake-compiler gem. The following command is to compile the native extension setting the -Wundef -Werror as compiler flags. You see the command propagates the -Wundef -Werror to the gcc command lines. It's important to run with the rake-compiler latest stable version 1.2.2. Because it fixes https://github.com/rake-compiler/rake-compiler/issues/215 related to this issue. The environment variable MAKEFLAGS reserved in the GNU make is to print the compiler command lines.
$ MAKEFLAGS="V=1" bundle exec rake compile -- --with-cflags='-Wundef -Werror'
mkdir -p tmp/x86_64-linux/openssl/3.3.0
cd tmp/x86_64-linux/openssl/3.3.0
/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags=-Wundef -Werror
checking for rb_io_maybe_wait(0, Qnil, Qnil, Qnil) in ruby/io.h... yes
...
gcc -I. -I/home/jaruga/.local/ruby-cbeba2ed16/include/ruby-3.3.0+0/x86_64-linux -I/home/jaruga/.local/ruby-cbeba2ed16/include/ruby-3.3.0+0/ruby/backward -I/home/jaruga/.local/ruby-cbeba2ed16/include/ruby-3.3.0+0 -I../../../../ext/openssl -DRUBY_EXTCONF_H=\"extconf.h\" -fPIC -Wundef -Werror -o openssl_missing.o -c ../../../../ext/openssl/openssl_missing.c
...
cp tmp/x86_64-linux/openssl/3.3.0/openssl.so tmp/x86_64-linux/stage/lib/openssl.so
$ echo $?
0
The problem is the the line /home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags=-Wundef -Werror printed at sh in the lib/rake/extensiontask.rb. Because the --with-cflags=-Wundef -Werror is different from the one actually executed, and not reproducible. The argument --with-cflags's value is -Wundef -Werror. It should be printed as --with-cflags="-Wundef -Werror" with quotes.
Debug
I debugged with debug gem. Added the gem "debug" to the ruby/openssl's Gemfile.
$ pwd
/home/jaruga/git/ruby/openssl
$ vi Gemefile
$ bundle update
Next set the breakpoint before the sh method to print and execute the command.
$ cd /home/jaruga/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake
$ pwd
/home/jaruga/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake
$ diff -u extensiontask.rb.orig extensiontask.rb
--- extensiontask.rb.orig 2023-05-26 20:36:06.207487491 +0200
+++ extensiontask.rb 2023-05-26 21:14:35.794188802 +0200
@@ -215,6 +215,9 @@
end
chdir tmp_path do
+ require 'debug'
+ binding.break
+
sh *cmd
end
end
Went back to the ruby/openssl, and cleaned the built files.
$ pwd
/home/jaruga/git/ruby/openssl
$ rm -rf tmp lib/openssl.so
And ran the command again.
$ MAKEFLAGS="V=1" bundle exec rake compile -- --with-cflags='-Wundef -Werror'
...
(rdbg) f # frame command
=> 219| binding.break
=>#0 block in define_compile_tasks (2 levels) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake/extensiontask.rb:219
(rdbg) p cmd # command
=> ["/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby", "-I.", "-r.rake-compiler-siteconf.rb", "../../../../ext/openssl/extconf.rb", "--", "--with-cflags=-Wundef -Werror"]
...
The lib/rake/file_utils.rb#sh calling Rake.rake_output_message sh_show_command cmd. And the cmd.join " " in the sh_show_command causes this issue. Is it a bug?
(rdbg) n # next command
[77, 86] in ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb
77| env = cmd.first
78| env = env.map { |name, value| "#{name}=#{value}" }.join " "
79| cmd[0] = env
80| end
81|
=> 82| cmd.join " "
83| end
84| private :sh_show_command
85|
86| def set_verbose_option(options) # :nodoc:
(rdbg) f # frame command
=> 82| cmd.join " "
=>#0 FileUtils#sh_show_command(cmd=["/home/jaruga/.local/ruby-cbeba2ed16/bin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb:82
(rdbg) p cmd # command
=> ["/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby", "-I.", "-r.rake-compiler-siteconf.rb", "../../../../ext/openssl/extconf.rb", "--", "--with-cflags=-Wundef -Werror"]
(rdbg) p cmd.join " " # command
=> "/home/jaruga/.local/ruby-cbeba2ed16/bin/ruby -I. -r.rake-compiler-siteconf.rb ../../../../ext/openssl/extconf.rb -- --with-cflags=-Wundef -Werror"
I would show you the backtrace command just in case.
(rdbg) f # frame command
=> 82| cmd.join " "
=>#0 FileUtils#sh_show_command(cmd=["/home/jaruga/.local/ruby-cbeba2ed16/bin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb:82
(rdbg) bt # backtrace command
=>#0 FileUtils#sh_show_command(cmd=["/home/jaruga/.local/ruby-cbeba2ed16/bin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb:82
#1 FileUtils#sh(cmd=["/home/jaruga/.local/ruby-cbeba2ed16/bin..., block=nil) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils.rb:51
#2 block in define_compile_tasks (2 levels) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake/extensiontask.rb:221
#3 [C] Dir.chdir at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/fileutils.rb:240
#4 FileUtils#chdir(dir="tmp/x86_64-linux/openssl/3.3.0", verbose=#<Object:0x00007fd7eb4ad510>, block=#<Proc:0x00007fd7ea893108 /home/jaruga/va...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/fileutils.rb:240
#5 Rake::FileUtilsExt#chdir(args=["tmp/x86_64-linux/openssl/3.3.0"], options={}, block=#<Proc:0x00007fd7ea8930b8 /home/jaruga/va...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/file_utils_ext.rb:35
#6 block {|t=<Rake::FileTask tmp/x86_64-linux/openssl/...|} in define_compile_tasks at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-compiler-1.2.2/lib/rake/extensiontask.rb:217
#7 block {|act=#<Proc:0x00007fd7efbd8b10 /home/jaruga/va...|} in execute at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:281
#8 [C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:281
#9 Rake::Task#execute(args=#<Rake::TaskArguments >) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:281
#10 block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:219
#11 [C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#12 Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::FileTask tmp/x86_64-linux/opens...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#13 block {|p=<Rake::FileTask tmp/x86_64-linux/openssl/...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
#14 [C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#15 Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::FileTask tmp/x86_64-linux/opens...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#16 block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
#17 [C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#18 Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task copy:openssl:x86_64-linux:...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#19 block {|p=<Rake::FileTask tmp/x86_64-linux/openssl/...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
#20 [C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#21 Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task copy:openssl:x86_64-linux:...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#22 block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
#23 [C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#24 Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile:openssl:x86_64-lin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#25 block {|p=<Rake::Task copy:openssl:x86_64-linux:3.3...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
#26 [C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#27 Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile:openssl:x86_64-lin...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#28 block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
#29 [C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#30 Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile:x86_64-linux => [c...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#31 block {|p=<Rake::Task compile:openssl:x86_64-linux ...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
#32 [C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#33 Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile:x86_64-linux => [c...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#34 block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
#35 [C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#36 Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile => [compile:x86_64...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#37 block {|p=<Rake::Task compile:x86_64-linux => [comp...|} in invoke_prerequisites at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:243
#38 [C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#39 Rake::Task#invoke_prerequisites(task_args=#<Rake::TaskArguments >, invocation_chain=LL(<Rake::Task compile => [compile:x86_64...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:241
#40 block in invoke_with_call_chain at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:218
#41 [C] Monitor#synchronize at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#42 Rake::Task#invoke_with_call_chain(task_args=#<Rake::TaskArguments >, invocation_chain=LL()) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:199
#43 Rake::Task#invoke(args=[]) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/task.rb:188
#44 Rake::Application#invoke_task(task_string="compile") at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:160
#45 block {|task_name="compile"|} in top_level (2 levels) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:116
#46 [C] Array#each at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:116
#47 block in top_level at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:116
#48 Rake::Application#run_with_threads at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:125
#49 Rake::Application#top_level at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:110
#50 block in run at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:83
#51 Rake::Application#standard_exception_handling at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:186
#52 Rake::Application#run(argv=["compile", "--", "--with-cflags=-Wundef ...) at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/lib/rake/application.rb:80
#53 <top (required)> at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/gems/rake-13.0.6/exe/rake:27
#54 [C] Kernel#load at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/bin/rake:25
#55 <top (required)> at ~/var/git/ruby/openssl/bundle/ruby/3.3.0+0/bin/rake:25
#56 [C] Kernel.load at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli/exec.rb:58
#57 Bundler::CLI::Exec#kernel_load(file="/home/jaruga/var/git/ruby/openssl/bundle..., args=["compile", "--", "--with-cflags=-Wundef ...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli/exec.rb:58
#58 Bundler::CLI::Exec#run at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli/exec.rb:23
#59 Bundler::CLI#exec(args=["compile", "--", "--with-cflags=-Wundef ...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli.rb:492
#60 Bundler::Thor::Command#run(instance=#<Bundler::CLI:0x00007fd7eb4ed160 @_initi..., args=["rake", "compile", "--", "--with-cflags=...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/vendor/thor/lib/thor/command.rb:27
#61 Bundler::Thor::Invocation#invoke_command(command=#<struct Bundler::Thor::Command name="exe..., args=[["rake", "compile", "--", "--with-cflags...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/vendor/thor/lib/thor/invocation.rb:127
#62 #<Class:Bundler::Thor>#dispatch(meth="exec", given_args=["rake", "compile", "--", "--with-cflags=..., given_opts=nil, config={:debug=>true, :shell=>#<Bundler::Thor::S...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/vendor/thor/lib/thor.rb:392
#63 Bundler::CLI.dispatch at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli.rb:34
#64 Bundler::Thor::Base::ClassMethods#start(given_args=["compile", "--", "--with-cflags=-Wundef ..., config={:debug=>true, :shell=>#<Bundler::Thor::S...) at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/vendor/thor/lib/thor/base.rb:485
#65 Bundler::CLI.start at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/cli.rb:28
#66 block in <top (required)> at ~/.local/ruby-cbeba2ed16/lib/ruby/gems/3.3.0+0/gems/bundler-2.5.0.dev/libexec/bundle:45
#67 Bundler.with_friendly_errors at ~/.local/ruby-cbeba2ed16/lib/ruby/3.3.0+0/bundler/friendly_errors.rb:117
#68 <top (required)> at ~/.local/ruby-cbeba2ed16/lib/ruby/gems/3.3.0+0/gems/bundler-2.5.0.dev/libexec/bundle:33
#69 [C] Kernel#load at ~/.local/ruby-cbeba2ed16/bin/bundle:25
#70 <main> at ~/.local/ruby-cbeba2ed16/bin/bundle:25
