ondemand

ondemand copied to clipboard

Hello,

I found there's might be a bug that handles the path, when user uses "Job Composer-> From Specified Path". By dafault OnDemand will use $HOME as the DATAROOT for "myjobs" app. So when user use $HOME(like ~/) as the "Source path" for the "From Specified Path), it will cause  infinitely looped path of ~/ondemand/data folder rsynced. OOD only checks the size of the "Source folder", and uses "du" command timeout fo safe copy check, which can not absolutely avoid this situation.

Th finitely looped path rsynced looks like the following:

/home/user1/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/ondemand/data/sys/myjobs/projects/default/2/

I have prepared a solution for this issue, by adding 3 lines of codes in app/models/filesystem.rb, to block the use of $HOME by users and give warning message:

 #
 # Exclude user's $HOME folder, avoid infinitely looped path rsynced, when the the DATAROOT is not set, or set to ~/ondemand/data/sys/myjobs/
 #
 if ENV['HOME'].to_s == Pathname.new(path).expand_path.realpath.to_s
    return false, "Please DO NOT use your HOME folder as your Specified Source path! Try to use other paths, sub-paths, like ~/mywork, etc."
 end
 # FIXME: consider using http://ruby-doc.org/stdlib-2.2.0/libdoc/timeout/rdoc/Timeout.html

The patching codes can be found here in my fork of OnDemand:

https://github.com/prod-feng/ondemand/commit/b03f091169835a8757846cfc8c7c5abf213af309

Best,

Feng

Follow up:

Added some more codes to handle more general situation when the DATAOOT is set.

#Check if the Source path is a subfolder of the OOD_DATAROOT. If so, stop it.

    if Pathname.new(ENV['OOD_DATAROOT']).fnmatch?(File.join(path,'**'))        return false, "Please DO NOT use #{path} as the Source path! This will cause infinitive loop of files/paths copying when OOD stages the folder for your job. Please choose other path!"     end

Please see the link here : https://github.com/OSC/ondemand/commit/cf4cb9bb2224c4d38ee8a04ba3e71c792c6670d7

Thanks for the ticket and the followup. I'll take a look at this this week.