Can't set shell_path to nil

[sleonard617]

Greetings!

I am building a proof of concept installation of version 1.7. Our cluster uses LSF10.1 and all batch jobs must execute within a docker container. Due to this and the required esub and elim scripts involved, the "-L" flag to bsub is incompatible with our system. OodCore::Adaptor::LSF natively sets this as "/bin/bash".

When looking to the ood_core code base, this appears to be easily handled by setting the "shell_path" attribute to "nil".

https://github.com/OSC/ood_core/blob/master/lib/ood_core/job/adapters/lsf/helper.rb 93 args.concat ["-L", script.shell_path.to_s] unless script.shell_path.nil?

Using test.rake, I can set this and it works as I'd expect: with "shell_path: nil" -L is dropped and the test completes successfully.

However, when configuring Jupyter Notebook (or any other app for that matter), when I attempt to set the value in the submit.yml.erb as such:

script: shell_path: "nil"

The value is treated as a string directly. This is true with or without quoting:

App 18192 output: [2020-10-29 16:31:01 -0500 ] INFO "execve = [{"LSF_BINDIR"=>"/opt/ibm/lsfsuite/lsf/10.1/linux2.6-glibc2.3-x86_64/bin", "LSF_LIBDIR"=>"/opt/ibm/lsfsuite/lsf/10.1/linux2.6-glibc2.3-x86_64/lib", "LSF_ENVDIR"=>"/opt/ibm/lsfsuite/lsf/conf", "LSF_SERVERDIR"=>"/opt/ibm/lsfsuite/lsf/10.1/linux2.6-glibc2.3-x86_64/etc"}, "/opt/ibm/lsfsuite/lsf/10.1/linux2.6-glibc2.3-x86_64/bin/bsub", "-cwd", "/home/shawn.m.leonard/ondemand/data/sys/dashboard/batch_connect/sys/jupyter/output/7f8141d6-819b-4e14-b298-508a13fc9767", "-J", "sys/dashboard/sys/jupyter", "-W", "60", "-L", "nil", "-o", "/home/shawn.m.leonard/ondemand/data/sys/dashboard/batch_connect/sys/jupyter/output/7f8141d6-819b-4e14-b298-508a13fc9767/output.log", "-a 'docker(jupyter/datascience-notebook:latest)'", "-G compute-ris", "-M 1GB", "-R 'select(mem=1GB)'"]"

┆Issue is synchronized with this Asana task by Unito

[johrstrom]

Hey I worked with Shawn on this in Slack. In LSF he needs to get rid of the -L option for shell path completely but can't.

I think the issue is here, that we force it so there's no way to 'unset' it. https://github.com/OSC/ondemand/blob/7aa05f00f7d809763105edc1b15301efa80fa654/apps/dashboard/app/models/batch_connect/session.rb#L473-L477

But even before that, we compact the options so if they could set nil, it'd be deleted here. https://github.com/OSC/ondemand/blob/7aa05f00f7d809763105edc1b15301efa80fa654/apps/dashboard/app/models/batch_connect/session.rb#L308

[ericfranz]

@sleonard617 instead of using:

script:
  shell_path: "nil"

You want

script:
  shell_path: null

For YAML, null is how you end up with nil:

irb(main):003:0> YAML.load "---\nfoo: null\nbar: 1"
=> {"foo"=>nil, "bar"=>1}
irb(main):004:0> YAML.load "---\nfoo: nil\nbar: 1"
=> {"foo"=>"nil", "bar"=>1}

[ericfranz]

Which is a great example of how difficult ERB rendered YAML can be!

[sleonard617]

Apologies. I had tested with empty string but not explicitly with key word "null". my bad

[sleonard617]

Apologies. I'll have to reopen this. In my zeal to figure out the inner workings I'd left a patch in /var/www/ood/apps/sys/dashboard/app/models/batch_connect/session.rb that disabled "shell_path".

Setting "shell_path: null" in the yml.erb does not actually work as session.rb will put a default right back in place.

[johrstrom]

Hey @sleonard617 where u ever able to get around this? I'm being reminded of this for other reasons and now wondering why we didn't suggest an initializer like so:

# /etc/ood/config/apps/dashboard/initializers/lsf_path.rb
module BatchConnect
  class Session
    def shell_path
      nil
    end
  end
end